EVALUATION OF THE LOCAL GOVERNMENT REFORM PROJECT IN MACEDONIA Submitted to: United States Agency for International Development Under Contract No. Eval IQC AEP-I-00-00-00023-00 Prepared by: Richard Kobayashi, Chief of Party Alan Edmund Submitted by: Development Associates, Inc. 1730 North Lynn Street Arlington, VA 22209-2023 May 2002 Evaluation of the Local Government May 2002 Reform Project in Macedonia TABLE OF CONTENTS Page No. Executive Summary....................................................................................................................... i Acronyms...................................................................................................................................... iv I. Findings............................................................................................................................... 1 A. Historical Program Overview and Purpose of the Evaluation ............................................. 1 B. The Appropriateness, Quality, and Impact of Technical Assistance Delivered................... 4 Municipal Management And Citizen Participation ........................................................ 4 Citizen Information Centers............................................................................................ 4 Training And Technical Assistance ................................................................................ 6 Summary......................................................................................................................... 7 C. The Role of National Organizations.................................................................................... 7 D. Political and Administrative Context.................................................................................. 9 Summary....................................................................................................................... 11 E. Assessment of Progress Toward Achieving Measurable Results...................................... 11 Summary....................................................................................................................... 13 F. Other Donor Assistance ..................................................................................................... 13 Summary..................................................................................................................... 14 Lessons Learned and Recommendations.................................................................... 14 Summary......................................................................................................................24 APPENDICES Appendix A Cover Letter and Scope of Work................................................................. A-1 Appendix B Team Composition and Methodology......................................................... B-1 Appendix C Performance Measurement Plan.................................................................. C-1 Appendix D List of People Interviewed .......................................................................... D-1 Appendix E Municipalities by Population....................................................................... E-1 Appendix F Political Context........................................................................................... F-1 Appendix G Program and Political History Timeline...................................................... G-1 Appendix H Questions Used to Guide Interviews........................................................... H-1 Evaluation of the Local Government i May 2002 Reform Project in Macedonia EXECUTIVE SUMMARY The Evaluation Team carried out an evaluation of the Local Government Reform Project in Macedonia in March 2002. The LGRP had completed two and a half years of activity at the time of the evaluation. The LGRP is based on a standardized Local Government Assistance approach used by USAID throughout CEE. Conditions in Macedonia underwent significant change after the commencement of the project. The ethnic conflict (2001) had the effect of placing the program under stress, and also raised the issue of decentralization to a very prominent role in Macedonian and in donor politics. This is the result of the Ohrid Framework Agreement executed in August 2001 in which the major Macedonian political parties agreed to significant decentralization as the vehicle to maintain Macedonia as a unitary state. The United States and the EU brokered this agreement and are signatories. As a result the LGRP, as USAID’s decentralization program, plays an important role in fostering the steps that are preconditions to preserving Macedonia as a unitary state. The Evaluation Team made three overarching recommendations: • USAID should maintain continuity in the LGRP’s programmatic approach and staff. It is an effective program and the stakes in Macedonia are simply too high to risk a gap in USAID’s support of decentralization. • USAID should maintain the flexible program structure and management style that can respond to opportunities in the dynamic context that characterizes Macedonian politics and society.. • Decentralization and its implementation will be a slow process. For USAID to have a major impact on both policy development and implementation the planning horizon needs to be at least five years. The Evaluation Team made the following major recommendations: • The LGRP is providing an appropriate level and type of assistance, in all of its four program areas. All aspects of the program provide a high level of professional assistance. • ZELS, the municipal association, is the only Macedonian political advocacy organization articulating the need and rationale for decentralization. It and the donor community are the constituencies for decentralization. USAID should support ZELS financially and with technical assistance as long as it is an effective advocate for decentralization. Self￾sustainability should not be a criterion for support, as the policy advocacy role is critical now and for the foreseeable future. Self-sustainability will occur when the municipalities have more money and macroeconomic conditions improve. Institutional support on a sustained basis is required until major decentralization laws are passed and implemented. • To be effective USAID/LGRP needs to continue to strongly support both Policy Reform and Association Development. Both areas remain fragile and require continuous support of expert expatriate staff. However, much of LGRP’s future work can be carried out by Development Associates, Inc. Evaluation of the Local Government ii May 2002 Reform Project in Macedonia its excellent and well trained Macedonian staff. Citizen Participation and Municipal Management programs will need the international experience that expatriates can bring to bear in the development and monitoring of new and refined programs, but not in day to day management. USAID/LGRP should consider assisting the Macedonian staff of DAI to form a firm that would subcontract to DAI to provide program services. This would help staff gain business skills, enhancing the possibility for a self-sustaining municipal technical assistance capability in Macedonia. • Existing CICs, which have been in existence for an average of six months, should be more intensively utilized by cities. They can become management analysis units for the municipalities and help shape citizen’s concerns into plans for systemic reform of service delivery. Additional training and support are required for CIC staff to grow into this role, but the pay off potential is high. A similar approach can be applied to the incipient Citizen Advisory Boards to public enterprises. • USAID/LGRP should consider developing an association of public enterprise managers along the lines of AFO. The objective would be to support line managers who provide most of the direct services in a municipality. The association could serve as an effective vehicle for education, training and technical assistance. A needs assessment should be performed that should include evaluation of any potential that the existing commercially organized association, MAKKOM, might play. • USAID/LGRP should work toward developing prototypes of functional decentralization in two municipalities and in two competencies. This requires willing Ministries as well as municipalities. The experience would inform the decentralization process. • In the interim period, before functional decentralization is implemented, USAID’s training and technical assistance efforts should focus on topics which are both important and not dependent on decentralization, e.g., Local Economic Development, Communal Enterprise Management, and Management Analysis Training for CIC staff. • USAID should assess the needs of smaller municipalities. There are 44 municipalities with populations under 5,000, which represent approximately 6% of the population, and 21 municipalities with populations between 5,000 and 10,000 comprising 11% 0f the population. .Because these municipalities contain a significant part of the population and have received a very low level of donor support they warrant attention from the LGRP. Fundamentally they have the same needs as larger municipalities, but approaches that are appropriate and effective in municipalities of this scale need to be developed. LGRP should take steps to improve the understanding of the needs and capacities of smaller municipalities, with emphasis on gaining an understanding of how Joint Administration, Circuit Riders and Contracting for services from large municipalities might help them maintain a political identity, while assuring efficient service delivery. A better understanding of smaller municipalities has the potential to enable LGRP to influence the formulation of the Law on Territorial Division. In addition some of LGRP’s efforts should be tailored to benefit smaller municipalities. • Monitoring and Evaluation should play a more important role in the design and management of LGRP projects, particularly in the Municipal Management, Citizen Development Associates, Inc. Evaluation of the Local Government iii May 2002 Reform Project in Macedonia Participation and Association Development aspects of the program. This effort should be focused both on enhancing program management and supporting USAID’s Intermediate Results framework. • City Councils play an important role in municipal governance, but have received scant attention by LGRP and other donors. As their decision-making responsibilities are likely to increase with decentralization, training and development programs should be developed for them. An initial needs assessment is recommended, using the Chamber of Councilors in ZELS’ governing structure as a starting point. • Local finance officers are employed by the Ministry of Finance in some municipalities, and directly by municipalities in others. All will continue to work in a unified system of municipal and state finance, even after decentralization. In collaboration with AFO and the Ministry of Finance, a needs assessment to identify useful types of pre￾decentralization training should be performed. This would set the stage for a cadre of municipal finance professionals to develop skills jointly – strengthening professional relationships. • Expatriate Assistance, in collaboration with Macedonian LGRP staff, should engage in new program development. The following are suggested as starting points for discussion: • Development of a Model Cities program, where donor aid is concentrated and highly coordinated; • Assessment of the feasibility of a Municipal Development Loan Fund, in collaboration with the World Bank; • Development of a locally administered revolving loan fund for SMEs, partly capitalized by the municipality; and • Development of strategies to increase participation by the private sector in the provision of public services. SUMMARY The recommendations build on the foundation created by the LGRP in the first 30 months of its existence. The major themes are, 1) maintaining momentum, 2) increasing municipal management capacity by more intensively using the resources and knowledge of citizen participation organizations, 3) strengthening ZELS, and 4) investing in new areas with potential for significant impact, e.g., prototype decentralization, private sector involvement and economic development. Evaluation of the Local Government iv May 2002 Reform Program in Macedonia ACRONYMS AFP Association of Finance Officers CAB Citizen Advisory Board CE Communal Enterprise, a local public authority which provides basic municipal services CEE Central and Eastern Europe CIC Citizen Information Center Circuit Rider an employee of local self government who is contracted or employed by several units and performs duties for each of them CMHS Center for Mental Health Services CSHI Community Self Help Initiative DAI Development Alternatives, Inc. ESI European Security Initiative EU European Union EU PHARE EU technical assistance agency FA Framework (Ohrid) Agreement FWA Framework Agreement GTZ German technical assistance agency GOM Government of Macedonia IR Intermediate Result(s) Joint Administration under Macedonian law, an arrangement among two or more local self- governments to provide services to one another, with special management mechanisms Leak detection use of an electronic device to locate underground leaks in water mains and house service pipes LED Local Economic Development LGA Local Government Assistance LGRP Local Government Reform Program LLSG Law on Local Self Government LLSGF (draft) Law on Local Self Government Finance MAKKOM A commercial company MOF Ministry of Finance MoFin Ministry of Finance MOU Memorandum of Understanding PHARE use as is OSCE Organization for Security and Cooperation in Europe Resource Cities a USAID program which pairs US and foreign cities for technical information exchange SAMHSA Substance Abuse and Mental Health Services Administration SEE Southeast Europe University Self-Sustaining maintained, nourished, or kept going through internal means only SO Strategic Objective SME Small and Medium Enterprises Sustainable,Sustainability maintained, nourished or kept going through internal or external means TA Technical Assistance ZELS Association of the Units of Self Government of Macedonia Evaluation of the Local Government 1 May 2002 Reform Project in Macedonia I. FINDINGS A. HISTORICAL PROGRAM OVERVIEW AND PURPOSE OF THE EVALUATION The Local Government Reform Project (LGRP), USAID’s program to support local government development and decentralization in Macedonia, commenced activities in October 1999. The Contractor for the project is Development Alternatives Incorporated (DAI). Prior to the start of the project, the Government of Macedonia (GOM) had passed a number of laws that attempted to clarify the role of local government in context of Macedonia’s unitary state structure. Major milestones during this period included the passage of: • 1995 – Original Law on Local Government • 1996 – Current Law on the City of Skopje • 1996 – Current Law on Territorial Division • 1997 – Macedonia ratifies the European Charter on Local Self-Government • December 1998 – Ministry of Local Self-Government (MoLG) established. The LGRP was established on the general model used in other Central and Eastern European nations by USAID, and contractor services were procured using the Local Government Assistance(LGA) Indefinite Quality Contract established for contracting for these kinds of projects. The major components of the project: Policy Reform, Citizen Participation, Municipal Management and Association Development were analogous to those carried on elsewhere in the CEE region. However the context in which the LGRP was carried out, quite quickly became significantly different from the conditions in other CEE countries. The program commenced on the heels of the Kosovo crisis of 1999 that placed Macedonia on the world stage. The crisis resulted in increased donor activity and (as it turned out) increased the possibility of fragmentation and conflict within Macedonia. Approximately fifteen months into the project (February 2001), the first terrorist incident occurred, taking place in the Western Macedonian city of Tanusevci. This was followed by a conflict that increased in intensity through much of the summer of 2001. The armed conflict between ethnic - Albanian militarized forces and the Macedonian security forces was resolved in August 2001 by the execution of the Ohrid Framework Agreement. This agreement was executed by the leaders of the two major Albanian political parties and the two major Macedonian political parties, in addition to the President of the Republic of Macedonia. As a result of these political developments since the initiation of the project, Macedonian local government development is viewed, not simply as a means to familiarize citizens with local democracy and enhance services. It is viewed as a vehicle to mitigate the irredentist tendencies of some ethnic Albanian political actors and citizens in areas with large Albanian populations. In short, decentralization is viewed as a way of granting a larger political voice to minority majority municipalities, where much of the ethnic Albanian population resides. Also, ethnic Macedonians feared that decentralization had the potential for federalization and fragmentation of the country. Avoiding the trend toward dissolution and fragmentation that characterized the Balkans for much Development Associates, Inc. Evaluation of the Local Government 2 May 2002 Reform Project in Macedonia of the 1990s, was an incentive for donor governments and the Macedonian Government to take this approach. During the peak of the conflict period July 2001 – October 2001, American staff of the LGRP were evacuated and the work program was carried out by Macedonian staff. DAI established a temporary office in Greece and managed program activities from there. The main points of the Framework Agreement relevant to the LGRP and to USAID’s policy objectives are that: 1. Macedonia would remain a unitary state; 2. The Albanian language would be an official language and that citizens of Albanian ethnicity could secure education and government services in the Albanian language. The relevant section of the FWA is Section 6: Education and Use of Languages1 which states in part: 6.5 Any other language spoken by at least 20% of the population is also an official language, as set forth herein. In the organs of the Republic of Macedonia, any official language other than Macedonian may be used in accordance with the law, as further elaborated in Annex B. Any person living in a unit of local self government in which at least 20% of the population speaks an official language other than Macedonian may use any official language to communicate with the regional office of the central government with responsibility for that municipality; such an office will reply in that language in addition to Macedonian. Any person may use any official language to communicate with a main office of the central government, which will reply in that language in addition to Macedonian. 6.6 With respect to local self government, in municipalities where a community comprises at least 20% of the population of the municipality, the language of that community will be used as an official language in addition to Macedonian. With respect to languages spoken by less than 20% of the population of the municipality, the local authorities will decide democratically on their use in public bodies. 3. The Government would decentralize major competencies related to municipal government, as defined in the European Charter of Local Self-Government. This was to be defined further in the law on Local Self Government. This law, which provides a framework for decentralization rather than a detailed transfer of powers, was enacted in January 2002, and provided that municipalities would have competence in 13 major areas. Of special note are the following: 1 The other sections of Section 6 deal with language in the following contexts: primary, secondary and university education, the official language of the country, criminal and civil proceedings, and official personal documents. Development Associates, Inc. Evaluation of the Local Government 3 May 2002 Reform Project in Macedonia • Urban and Rural Planning • Protection of the Environment, nature and space regulation • Local Economic Development • Communal Activities • Social Welfare and Child Protection • Health Care In addition, the Framework Agreement included a commitment to enact a Law on Local Government Finance by the end of the term of the present Assembly to “ ensure that the units of local self-government have sufficient resources to carry out their tasks, under the revised Law on Local Self-Government. It also provided that a Law on Territorial Division would be enacted by the end of 2002, likely to be based on census results. This was a precondition to significant donor activity, and during the time that the Evaluation Team was in Macedonia, a donors conference was held in Brussels. This conference was triggered by the passage of the Law on Local Self-Government. Significant announcements of aid were made at the conference. Work has been proceeding on the Local Self-Government Finance Law, but the timing of passage appears uncertain at this time. The general perception of key actors in Macedonia is that significant substantive work on devolution of major competencies will wait until after the Local Government Finance Law is passed, and most likely until after parliamentary elections that, while not currently scheduled for a specific date, will occur in the Fall 2002. The LGRP project is unique among LGA projects in that it is the only project which is serving in direct support of a political agreement, namely the Ohrid Framework Agreement, that the United States brokered (in collaboration with the EU) and to which it is a signatory. This condition elevates the significance of the LGRP as USAID’s local government development vehicle in Macedonia. The Table in Appendix 6 shows key political developments in Macedonia and major LGRP milestones on the same multi-year time schedule. The context illustrated by the Table 1 serves as one of the main points of reference in evaluating the LGRP. Expectations by citizens for practical results from the Framework Agreement are high, according to people interviewed by the Evaluation Team. Accordingly the pressure for tangible short term results is acute. It is widely believed that short term results in cities which are visible indicators of the positive impact of the elements of the Framework Agreement are necessary in order for the signatories to maintain credibility with the citizenry. While information to evaluate the LGRP must be drawn from the experience of the program since October 1999 the main purpose of the evaluation is to help USAID –Macedonia consider options for follow-on assistance to support local government development in Macedonia. Accordingly the main focus of the Report is on options for a follow-on program which will commence after September 30, 2002 when the period for performance under the current Task Order ends. Development Associates, Inc. Evaluation of the Local Government 4 May 2002 Reform Project in Macedonia The overarching consideration in planning USAID’s future efforts in Municipal sector development in Macedonia is explicit recognition that More effective and responsive local government in Macedonia is not merely an SO. It is a condition that is essential to implementation of the Framework Agreement, an agreement designed to bring the stability and confidence prerequisite to addressing widespread economic insecurity. Excerpts from USAID’s Strategy 2002-2005 as well as other significant commentary on the relationship of decentralization to sustained peace are included in Appendix 7. B. THE APPROPRIATENESS, QUALITY, AND IMPACT OF TECHNICAL ASSISTANCE DELIVERED MUNICIPAL MANAGEMENT AND CITIZEN PARTICIPATION Local officials interviewed were favorable toward the approach used to implement management improvements and citizen participation measures at the municipal level. This was almost universal. Due to the rapidly evolving decentralization program and its associated uncertainties, local officials described LGRP as a beginning and appropriate for the circumstances of the last few years. In addition, they consistently stressed the need for Training and Technical Assistance that would help implement decentralization. While the recommendations section of this report offers options for further strengthening by LGRP of truly self-governing municipalities, the Evaluation Team determined that early training and technical assessment efforts were primarily effective in building the confidence of individual municipalities and helped to induce demand for local management capacity improvement among Macedonian municipalities. Mayors and other local officials, as well as the GOM and donors, stress and the need for training and technical assistance, as various kinds competencies were devolved to municipalities, presumably with adequate resources to carry them out. There was a “wait and see” attitude among local officials concerning the kinds of training and technical assistance that would be useful and appropriate, as the competencies were spelled out in law and scheduled for implementation. The most important and urgent aspects of the training and technical assistance, in the minds of most mayors, were the continued provision of assistance to ZELS, and the careful crafting of the laws to carry out a successful devolution, particularly in finance, health, education, social services, sports and culture. CITIZEN INFORMATION CENTERS Without exception, the CICs are useful and affordable to municipal governments. By screening complaints and answering inquiries, they free up time for the mayors, in particular, to work on major projects or issues. The costs of the centers are low, and in no case are those costs a burden for local governments. The capital costs are borne by others or are minor, and the seconded staff from municipal administrations do not generally represent additions to local payrolls. The Evaluation Team found that the CICs are managed and staffed by competent, knowledgeable, and cheerful employees, who are able to process complaints and requests for information effectively for their mayors. They are able to work with municipal departments and Development Associates, Inc. Evaluation of the Local Government 5 May 2002 Reform Project in Macedonia other service providers in determining when a particular service activity will be carried out, or what the process for obtaining a certain license is. In some cases the centers have taken on responsibilities, or special projects, which go beyond the collection and processing of citizen inquiries or complaints. The team found instances of CICs’ carrying out surveys, or processing certain licenses, or providing support to local NGOs. There are successful examples of CICs carrying on two-way communications between citizens and their local administrations. The CICs, on average, are only a few months old, and therefore no standard “role” for them is apparent to citizens in participating municipalities. Whether by design or by local political leadership preference, some CICs seem able to take on duties assigned by the mayor that go beyond information gathering. A prime example is the traffic routing satisfaction survey done in Veles. Another is Kavadarci, where a citizen satisfaction survey is now being completed. This is a positive development, and this flexibility is one of the program features that the LGRP seeks. CIC internal reporting systems and their staff training programs are standardized. Networking capabilities among CICs are in the process of being established, and information sharing through meetings and telephone conversations appears to be well established. The Evaluation Team found, however, that there is no systematic way in which mayors or city councils use the CICs. In no case, in this early stage of CIC institutionalization have municipalities substantially changed their operating procedures because of CIC influence. Interim measures have been taken, such as issuing taxi licenses by the CIC (Veles), but taxi licenses are already in the Mayor’s purview, and he has delegated the function to the CIC, which is also in his purview. Some substantial changes in procedure and jurisdiction will have to wait until competencies are reassigned by law, notably in the area of development and building permits. However, CICs can play a more significant role in fostering improvements in services provided by public enterprises and by other local service providers. CICs, in this early stage, do perform the function of building a relationship between citizens and their mayor, and they do systematically collect useful information about how well the municipality and regional ministry offices offer transparent municipal and regulatory services. In some cases, the CICs have been called upon to perform special services, such as NGO assistance, emergency employment assistance, and survey research. The assessment team has found that, in order for CICs to take root as institutions that significantly impact the quality of local municipal management and also substantial two-way communication between local government and citizens, future program design should reflect the following approaches: • More concrete use of CIC program information by mayors and councils, in order to maximize CIC inputs as management and policy making tools, without diluting their effectiveness or causing mistrust of CICs on the part of the public. There is ambiguity, at present, in whether services will increase in net quality, or whether the CICs simply provide an avenue for the “squeaky wheel” citizens to benefit, at the expense of those who are silent or go through traditional channels; Development Associates, Inc. Evaluation of the Local Government 6 May 2002 Reform Project in Macedonia • Networking of CICs on an expedited basis, to prepare for the massive growth of local government responsibility and the resultant confusion, is perhaps only a year away; • More analysis of data that is collected for the purpose of arguing for certain competencies to be devolved, most importantly community development and permitting, where many complaints and inquiries arise; • More communication with municipal councils, as they seek increased responsibilities independent of mayors; and • More use of CIC staff time in assembling notices and posting information about municipal services, issues, and events. Publicizing of Council decisions, upcoming meetings, etc. should be routinized. CICs appear to be under-patronized at this time for the staff allotted. Now seems to be the time to employ the staff in promotional activities and for disseminating information on actions of all elected officials. TRAINING AND TECHNICAL ASSISTANCE Mayors and other local officials were profuse in their praise of the training to date. The mayor of Bitola was critical of the mixing of sophisticated municipalities with less sophisticated ones, but he was not critical of the training content LGRP has produced. The mayors and enterprise heads considered one particular program, the leak detection component, to be especially responsive to their needs and highly useful in containing local utility costs. LGRP staff had analyzed critical areas of local service deficiencies, and had concluded that the use of leak detectors would provide for the most measurable and effective method of improving services in ways that the citizens would immediately recognize. Local officials also lumped the ZELS-sponsored meetings on the LLSG together with training and technical assistance--characterizing those sessions as the most significant in terms of impacting the municipal government role. These events enhanced the collective power and prestige of the mayors as a group. The Evaluation Team concluded that the early training in citizen participation, leadership, policy advocacy, and organization development offered to municipalities was useful, based upon an accurate perception of overall local need, and appropriate for the time in the content offered. Affordability was not a factor, as training and technical assistance did not require local investment in capital or staff. The Evaluation Team found, primarily through anecdotal information and program descriptions, that the recent training and technical assistance projects provided a broad array of information and skills to municipalities. The focus of interventions at the local level has been on skills development or technical training and advice, focused on producing short term results that will improve the relationship of citizens to local government, by increasing transparency and/or providing the prospect of better service. Development Associates, Inc. Evaluation of the Local Government 7 May 2002 Reform Project in Macedonia SUMMARY Based on survey inputs, the Evaluation Team found that LGRP training and technical assistance efforts have created a demand for municipal training products. The overall impact of this experience is that local governments, in general, are aware of the training products being offered, and eagerly want to participate in the programs. In particular, study tours, CICs, leak detection, and enterprise finance were popular. Local Economic Development training is getting underway, and all municipalities contacted expressed a strong interest in taking advantage of the content, and in creating a position responsible for economic development coordination and advocacy. It is too soon to say whether this activity and interest represent the beginning of a sophisticated and sustainable market for training and technical assistance, or whether an incipient “cadre” of local government experts has been formed around the country. Local officials are going to avail themselves of whatever training is offered, that is of even passing interest, to develop and maintain relationships with the donor community--which is currently the only source of significant discretionary funding available to municipalities. While at this stage it appears that training and technical assistance programs are supply driven (determined by donor perception of needs), the assessment team found that, in each sample community, there is a broad based constituency for training and technical assistance, which is becoming more sophisticated. Training and technical assistance components of the LGRP have been appropriate and useful. They have been suitable for the Macedonian context during the period that the program has operated in Macedonia. C. THE ROLE OF NATIONAL ASSOCIATIONS The efforts of the LGRP in developing local associations have concentrated on the Association of Macedonian Cities (ZELS) and the Association of Finance Officers (AFO). This is a pattern similar to the approach taken by many USAID supported local government development projects in CEE, where ZELS and AFO have fundamentally different characters: • ZELS is an organization of cities whose main mission is political – advocating for local self-government interests. It also provides a level of direct member services. The ZELS Standing Committee which functions as its Board of Directors is comprised of 15 mayors and 15 city council chairmen. All major parties and regions are represented. Significant development of ZELS as a modern municipal organization did not begin until after the October 2000 municipal elections, when an activist group of newly elected mayors gained control of the organization and proceeded to modernize it. Accordingly, the current ZELS organization and level of sophistication is the result of less than two years of effort by ZELS membership and USAID. • AFO is an organization of individual members whose main mission is the professional development of its members. It also supports the development of national – local financial rules, mechanisms and procedures, through participation in technical working committees and also serves as a technical resource to ZELS. While AFO is a municipal finance officers organization, an emerging membership policy will enable any financial officers involved in municipal finance to become members no matter whether their employer is a local government unit or a national government ministry. Development Associates, Inc. Evaluation of the Local Government 8 May 2002 Reform Project in Macedonia AFO has been going through a transitional process and LGRP support has diminished recently as a consequence of the president’s difficulty in relinquishing his post, since leaving the municipal finance area. During the Evaluation, the current leadership advised that new officers would be elected shortly. This was confirmed by LGRP staff. This appears to be a growing pains problem, and once the leadership transition is complete LGRP should step up its assistance to AFO, which can play a valuable role in implementing decentralization. ZELS has a long history as a municipal organization, but as an ineffective one. According to people interviewed for this evaluation, ZELS was viewed as a Mayor’s club and as a captive of the Government, with no particular policy agenda. ZELS appears to be making rapid progress toward becoming a full fledged municipal service and advocacy organization. USAID support and technical assistance are highly regarded by ZELS officers and staff, who consistently state that the progress ZELS has made has been made possible by USAID support. ZELS has a broadly representative Standing Committee, which functions as the Board of Directors and plays an active role in policy making and implementation--and it appears, based on interviews carried for this evaluation, to have good communication with its members. ZELS has adopted a dues policy which assesses 0.4% of City Budgets as a membership fee. According to the Executive Director, currently 53% of the members pay their dues and this yields 35% of the total dues assessed. This occurs because a higher proportion of smaller communities pay their dues than do the larger communities. The Executive Director stated that, while these numbers may look low, they are the highest they have ever been. Annual dues revenues, if all were paid, would be approximately 50,000 Euros. The Executive Director stated that revenues could go as high as 70% this year 2002, and noted that a significant jump in revenue would occur if Skopje paid its dues, which represent 10% of the total. She stated that ZELS’ President was negotiating on this matter with Skopje’s mayor. The key issue facing ZELS and the major donors which support it, is ensuring that it has sufficient capacity to play an active and constructive role in the political and technical decisions which affect decentralization. In essence, ZELS has to build the following types of capacity simultaneously: • Political Advocacy skills of its leadership; • Internal Organizational and Management capacity; • Technical Skills in each of the competencies, so that its staff can take part in technical discussions on specific areas of decentralization. And so that its leadership has the benefit of technical support to its political advocacy role; and • Service provision to members. Not all members see the direct benefit of the policy advocacy. ZELS needs to develop the capacity to provide expert TA and information to members. The Executive Director is thinking along this line, and would like to have a legal expert on staff to advise cities. Development Associates, Inc. Evaluation of the Local Government 9 May 2002 Reform Project in Macedonia In the foreseeable future, the ability to provide this level of service and advocacy cannot be sustained by the current dues base, and cannot be sustained even if all dues are collected. D. POLITICAL AND ADMINISTRATIVE CONTEXT The overarching framework for decentralization efforts is the Ohrid Framework Agreement signed in August 2001. While the Ohrid agreement raised the importance of decentralization in the political decision making processes of the country, decentralization efforts were underway previously. For example, the LGRP was designed and implemented well before the Ohrid agreement, as were several other donor programs aimed at decentralization and strengthening municipalities. The adoption of the European Charter on Local Self- Government is viewed as a significant milestone in national policy development by local officials and by European donors, who describe it frequently in terms of an EU accession requirement. While the Framework Agreement raised the profile of decentralization and held out the promise of local control of major functions, implementation is likely to take time. The general feeling among local officials interviewed was that the Law on Local Self-Government would not be effective, until the Law on Local Government Finance is passed and implemented. In the Framework Agreement this was specified as a milestone to be achieved before the current parliamentary session is completed. Parliamentary elections are anticipated in the fall. There is significant uncertainty about the passage of this law by this Parliament, donors interviewed all stated that the law would not be likely to pass until winter 2003. On paper, Macedonia is committed to decentralization. However there is a significant gap between the commitments and implementation. This gap will only be narrowed by consistent efforts by local political actors, coupled with sustained donor support for decentralization. Decentralization in Macedonia will be a process which will take place over a period of years. The risk in moving too slowly is that accomplishments will fall short of the expectations fostered by the Framework Agreement and that consequently the risks to the survival of Macedonia as a unitary state will be increased. For a fuller discussion of these risks please see Appendix F. The major political forces advocating decentralization are ZELS and the donor community. According to reports, the donor community, acting in concert, made significant aid commitments contingent on the passage of the Law on Local Self Government. This monetary pressure on the Government is widely believed to have been the catalyst for the passage of the LLSG in February. During the evaluation period (on March 12th) the donors announced a significant aid package for Macedonia at a much heralded donors conference in Brussels. The next major piece of legislation in the decentralization pipeline is the Law on Local Government Finance. While the LLSG establishes a structure, the LLGF will give municipalities resources to carry out their competencies. What is unclear at his time is whether the Macedonian Government, either now or after the elections expected in the fall, will have the political will to devolve national revenue sources to localities and/or grant municipalities new revenue sources. One way to think about the climate for decentralization is to consider the incentives that the national government has for the devolution of revenue. Except for ZELS and ethnic minority political parties, there appears to be little internal advocacy to devolve revenue. In a poor country like Macedonia, it is reasonable to expect national revenues to be devolved piecemeal, with the pace of devolution affected by economic growth. In this context, passage of a LLGF, which provides a comprehensive framework for state-municipal fiscal relations and makes initial progress in revenue assignment, Development Associates, Inc. Evaluation of the Local Government 10 May 2002 Reform Project in Macedonia is an important milestone. A key measure of the political climate for decentralization will be whether an effective Law on Local Government Finance is passed, within a reasonable time, after the formation of a Government following the fall elections. Members of the Donor community, including USAID, through the Policy Reform component of the LGRP, are working diligently at the technical level, on the LLGF. Similarly the LGRP support for ZELS as an advocacy organization is building public and legislative advocacy capabilities. These capabilities were in evidence during the passage of the LLSG, and can be expected to be harnessed to the passage of the LLGF. Evaluating the political climate for decentralization can best be determined by whether actual decentralization is taking place. The LLSG was an important first step and the passage of the LLGF will be another step. The careful monitoring of the actual implementation of these laws will provide indicators of whether decentralization is supported by the political climate. While the passage of the LLSG occurred several months later than specified in the Ohrid Agreement, it was enacted. Monitoring how its provisions are implemented by Ministries, will provide a tangible indication of the Government’s orientation. The Ohrid Framework Agreement which accelerated decentralization, was signed just over six months ago. It has changed the political climate, but whether it has changed it enough for decentralization to proceed at a pace fast enough to satisfy critical elements of the public, remains an open question. According to the European Security Initiative Report, a major constraint to decentralization is the perception of winners and losers among the major ethnic groups. While Mayors, from all parties, appear unified in their desire for decentralization, Macedonian ethnic-based political parties tend to view decentralization as holding the potential for significant losses in public sector jobs and power, as these ethnic parties tend to be over represented in Albanian majority communities. In a country with a poor economy this can be a severe constraint to decentralization. Whether sufficient pressure from mayors, the donor community and non-Macedonian ethnic interests is sufficient to counter this obstacle is not yet known. The political context for decentralization and the concomitant political will to implement decentralization laws, is expected to remain a limiting factor. Whatever the results of the fall elections, continued enhancement of the advocacy capabilities of ZELS, and support for technical policy reforms are all important and must continue. A useful lesson can be drawn from USAID’s experience in Slovakia. Until the 1998 election, the Slovak government was dominated by former Prime Minister Meciar, a very conservative, populist style, former communist political leader. During the mid-90s, USAID created, staffed and sustained the Local Self-Government Assistance Center, which implemented a program of training, institution building and policy reform, similar to the LGRP. This effort assisted in the development of strong municipal advocacy and policy reform capabilities. When Meciar’s party was soundly defeated in 1998, much of the technical work on policy reform was ready, and a cross section of political and senior administrative actors were well prepared to implement a local government reform and fiscal decentralization agenda. Significant progress has been made, since 1998, in decentralization and local development. USAID’s Local Government Development efforts in Slovakia were deemed successful and terminated in about 1998 or 1999. Development Associates, Inc. Evaluation of the Local Government 11 May 2002 Reform Project in Macedonia The principal lesson from the Slovak experience is that policy reform efforts, carried out in adverse conditions, can pay off when there is a change in government. Mayors were elected in 2000 for a four-year mandate which expires in 2004. It was members of the class of mayors who took leadership roles in recreating ZELS. Parliamentary elections are scheduled in 2002, and it is assumed that by the winter a new government will be formed. It would be useful to use the direction and pace of decentralization efforts between now and the next municipal elections, scheduled for 2004, as a measure of the political climate for devolution. Within this time horizon, the growing capabilities of ZELS, the impact of donor technical assistance, and the political behavior of a new Government can be assessed, in the context of an evolving political decision making process. This is an appropriate timeline for evaluating whether its investments are worth sustaining. SUMMARY USAID’s Policy Reform has given local government interests and USAID a seat at the table in the formulation of decentralization reforms. This is a relatively new phenomenon and needs to be built upon to support incremental progress. Credibility with senior Macedonian government officials is an important but intangible accomplishment of the LGRP, and can serve as the basis for a significant contribution to the policy debate on decentralization and its implementation. It is particularly noteworthy that LGRP staff have preliminary plans to play an active contributing role in the implementation of LLGF. In addition, the plans to utilize an educational finance specialist, with experience in the financial aspects of decentralizing education, represents an appropriate strategy to first understand and then influence the Government’s thinking on this technical issue. A similar approach is contemplated for other competencies to be decentralized. USAID, through the LGRP, is positioned to have significant policy influence in the development and implementation of decentralization. E. ASSESSMENT OF PROGRESS TOWARD ACHIEVING MEASURABLE RESULTS The assessment team found that the process of decentralization is proceeding on its own schedule, influenced primarily by the country’s legislative agenda, the social and ethnic dynamics which are embodied in the Framework Agreement, and the pressures for reform that are being applied by the international community. As such, it is not possible to measure the success of the LGRP with just the set of criteria found in the Intermediate Results framework; in fact, except for one measure, they are irrelevant in measuring the contribution of the LGRP to achieving USAID’s Intermediate Results for this program area. While in Macedonia, USAID staff requested that, in carrying out this task, the Evaluation Team concentrate on approaches to measuring impact, which could be used in future phases of the LGRP. The current measures follow: Development Associates, Inc. Evaluation of the Local Government 12 May 2002 Reform Project in Macedonia Indicator Technique(s) Used 2.3.a Public trust in local government in target municipalities Attitude surveys about satisfaction levels 2.3.1.a Local government spending as a percent of all general government expenditures Mathematical calculations from government documents 2.3.2.a Public participation in local government decision making Surveys about successful attempts to have Local Self-Government solve a problem 2.3.3.a Percent of municipal association members indicating they are satisfied with their association Attitude surveys conducted by the subject associations None of these performance indicators measure the effect of USAID/LGRP program activities on achieving USAID objectives. They are all proxy indicators that cannot isolate USAID/LGRP effects from other effects. The ZELS members attitude survey is probably the best proxy measure, given the very high degree of USAID/LGRP support for ZELS. The two surveys include so many influences which are not related to USAID/LGRP, that the data are not useful. The measure of local government spending, as a percentage of total government spending, is difficult in the absence of qualifying commentary. Increases in the national budget, e.g., defense or security spending could cause the percentage to plummet, and very modest increases in local own-source revenue could be masked easily in this measure. The following types of measures, more directly related to USAID/LGRP activity are suggested with the caveat that more analysis is required before measures along these lines are considered: • Absolute increases in own-source revenue in Target Cities, where LGRP focus is on increasing own-source revenue. • Increases in tariff revenue and/or tariff collection rates by Public Enterprises in Cities with CABs. • The number of documented systemic improvements in municipal service provisions, through interventions by CICs. • The volume of private or public investment in Economic Development fostered by Cities participating in the LED program. (the numbers are likely to be small, but the trend may be important). This is a proxy measure which would have to be refined to isolate the impact of the LGRP. • The rate of increase in the number of dues paying members of ZELS, as evidence of member satisfaction. This is a proxy measure, but may be realistic. • Small non-statistical surveys or focus groups in Target Cities of participants in the citizens advisory groups of various kinds (CIC, CAB, LED). Measurement of some type in each city could be possible. Similarly a cross section of citizens who have used CICs or CABs for information or input could be surveyed. Development Associates, Inc. Evaluation of the Local Government 13 May 2002 Reform Project in Macedonia All of these lines of thinking about measures require more programmatic effort to design and implement by LGRP than is the current case; but the benefit is that the kinds of data secured could be readily used in adjusting the programs, rather than in simply reporting. None of these measures will be effective in measuring the effects of Policy Reform activities, since the benefits of this type of work are always diffuse and in the future. A productive approach might be to help municipalities identify their goals and design performance measures that help client municipalities measure their progress. To the extent that these goals are aligned with those of USAID/LGRP the measures will help assess how LGRP has enabled local governments to meet explicit goals, which are shared by USAID/LGRP and the city(ies). This approach may have utility in the Municipal Management and Citizen Participation component, and in the Association Development component of the program as well. While objective measures are important, USAID’s daily supervision of the LGRP, and accurate anecdotal information, also provide useful evaluative data. Judgments based on close observation by seasoned development professionals who are familiar with the Macedonian context and programmatic risks are likely to be more valuable than numerical scores on proxy measures. SUMMARY The assessment team recommends that USAID/LGRP develop performance measurement techniques for the next phase of the program--which rely less on survey research and more on the objective measurement of outcomes that can be traced back to program activities. This will require monitoring and evaluation to be built into program design at the outset. F. OTHER DONOR ASSISTANCE A large number of donors are active in the municipal sector in Macedonia. Because of the small size of the country and the long tenure of many donor officials coordination has been quite extensive. Senior staff have well developed professional relationships. The Evaluation Team witnessed a highly professional coordination meeting among EU PHARE consultants, LGRP staff and USAID, at the kick off of PHARE’s project to assist the Ministry of Local Self Government. USAID appears to play a major role in Macedonia because of the size of its program relative to others, its combined bottom up/top-down approach, and its long term and consistent presence in Macedonia. Most of the donor activity in the Country is well known to USAID/LGRP, which serves as a frequent convener of donors. Programs in the pipeline, or which might inform LGRP activities and which can buttress LGRP efforts are: • GTZ’s Public Enterprise Program, which provides TA and commercialization advice. • World Bank’s Water Loan to Skopje, which is likely to have a major tariff reform and management component. Development Associates, Inc. Evaluation of the Local Government 14 May 2002 Reform Project in Macedonia • World Bank’s Public Sector Adjustment Loan, which, in future tranches, is likely to focus on the municipal sector. • EBRD’s water loan activity in several cities, which is also likely to have a major tariff reform and management component. • The activity now being initiated by the European Agency for Reconstruction, which has just fielded a team in Macedonia and plans to conduct a small infrastructure program, focused on municipalities. This program is in the planning stage. The major area for influence in the donor community appears to be during the planning stages of donor activities. While a high level of professional coordination among contractors and implementing agencies exists, most implementing agencies are bound tightly to their Terms of Reference, once program activity commences. The major influence that USAID/LGRP could bring to bear on donor activity would be to play a more active role, probably as a convener, among major donors, during the period when projects are in the formulation stage. This is likely to become more important as donor activity increases, to implement projects announced at the donor conference in Brussels on March 12th. SUMMARY USAID’S/LGRP extensive experience in Macedonia enables it to play an influential role in donor coordination, a role which is likely to become more important with the step-up in donor activity. USAID/LGRP’s role in donor coordination is effective, but this activity will require more attention in the future. LESSONS LEARNED AND RECOMMENDATIONS The goal of preserving Macedonia as a unitary state is the driving force behind decentralization in Macedonia, a condition that distinguishes the LGRP from its sister programs in other countries in the CEE. The recommendations that follow are written in response to this condition. During the evaluation period, the Evaluation Team observed the Mayors, who lead ZELS and participate in its governance, publicly advocate for the general interests of municipalities. In addition, project documents, Macedonians interviewed, and donor staff, all shared the view that ZELS’ leadership had been recognized and was making a specific point of communicating, at all levels, that the interests of municipalities were identical, in terms of relations to the State. It appears that over the last two years, ZELS has matured and is on the path to becoming an effective spokesman for the municipal community. Perhaps more importantly, the Mayors who speak for and are represented by ZELS are politicians – from all parties – who are known and recognized in their home cities; collectively they are developing an effective political voice. Their collective public political persona, as advocates for improved services and resources appears to be growing. The main thrust of USAID’s efforts, in the near future, should be to enhance and support this developing phenomenon, which bodes well for the country as a whole. While EU PHARE and others plan to enhance the MoLSG, as the main vehicle for controlling implementation Development Associates, Inc. Evaluation of the Local Government 15 May 2002 Reform Project in Macedonia devolution programs, the donor community recognizes that ZELS alone is the institution which informs the decentralization process in terms of local political and social conditions. As decentralization is implemented and the Mayors have real resources and competencies, they can fine tune service delivery to their communities’ needs thus enhancing the value of municipal government to citizens. USAID needs to consider that the progress to-date in Local Development is fragile and will need strong support from USAID for a significant period—support which builds on the staff competencies and programmatic experience of the first three years of the LGRP. In the Macedonian case, as in medicine, continuity of care is important. Lessons Learned Discussions of the value of the experience of the LGRP and the implications of that experience are presented throughout the report, and particularly in the Recommendations section.. The following “lessons learned” are not discussed elsewhere in the report, so are described below: • The LGRP, using the basic USAID LGA approach, tailored to Macedonian conditions, has been successful in strengthening the overall effectiveness of municipal governments. More importantly, a knowledge level and institutional structure which can support continuous improvement has been established. • The major lesson is that the multi-faceted simultaneous top down / bottom up approach has been successful in Macedonia. • It is clear that ZELS would not have its current capacity without USAID intervention. Similarly the sophistication of mayors and local leaders would not be at its current level, without the intensive LGRP efforts in training, technical assistance and development. In short, the LGRP has played a major role in taking the municipal community in Macedonia to a threshold level, from which rapid development can take place, if the government implements a well thought out decentralization program. These accomplishments occurred, despite the difficult circumstances surrounding the Kosovo crisis of 1999 and the conflicts of 2001. A major factor in sustaining LGRP efforts has been the high quality of local staff hired by the contractor. Local staff managed to sustain implementation during difficult periods and have developed and maintained a high level of credibility with municipal officials. The LGRP has been as successful as it has been because the program structure and USAID’s management were sufficiently flexible to adjust to changing circumstances and to learn from the early stages of the program. Program management appears to have been opportunistic, in that unforeseen opportunities were seized, e.g., new leadership among Mayors, as the result of the 2000 elections, development of Macedonian staff during the evacuation, pressures generated by the Ohrid Agreement.. The flexibility and opportunistic nature of the program should be incorporated into any extension of LGRP activities. Development Associates, Inc. Evaluation of the Local Government 16 May 2002 Reform Project in Macedonia Recommendations There are three overarching recommendations: 1. USAID should maintain continuity in the LGRP’s programmatic approach and staffing. The complex web of relationships and trust that has been developed, over the last three years, could not be easily reestablished after a prolonged gap in the program. In particular, the Association Development and Policy Reform Tasks would suffer from discontinuities. A significant gap would limit USAID’s ability to capitalize on its successes in Citizen Participation and Municipal Management. A gap would result in loss of momentum. 2. In planning for the future, USAID needs to maintain a flexible program structure that can respond to opportunities, not apparent at the time the program is designed. 3. In planning for the future, USAID needs to consider that the pace of reform in Macedonia depends on factors outside of its control, such as major legislative enactments, macroeconomic conditions and elections. These factors have a significant effect on the prospects for self-sustaining development among its clients. Decentralization will be a slow process. For USAID to have a major impact on both policy development (laws) and on the actual implementation of decentralization, it needs to plan for a sustained commitment to Macedonian local government reform. The planning horizon for USAID should be at least five years. The Recommendations are divided into two categories: General and Specific. General Recommendations These recommendations apply to the LGRP program as a whole. They set an overall framework for a distinctive approach to a multi-year extension of the LGRP. 1. The Macedonian staff of the LGRP can assume greater responsibility for all facets of program implementation. In particular, the Citizen Participation and Municipal Management components can be very well led by current local staff. They appear diligent, imaginative, committed and responsive to their municipal clients and to the objectives of the program. Expatriate staff can be used to buttress local staff, and help provide the perspective of international experience, but they will not be required to manage day-to-day program operations. Taking this approach strengthens the Macedonian team and will enhance the self￾sustainability prospects for a locally based municipal technical assistance team. One avenue to consider is fostering the creation of a Macedonian corporate entity that would subcontract to USAID’s contractor. This would provide business skills development, as well as visibility, both important for long-term business viability. DAI’s current mentoring program for local staff creates a positive environment for this approach. 2. The pace of decentralization cannot be predicted. USAID’s most significant contributions to decentralization is its effort in Policy Reform and development of Local Government Advocacy. These efforts should be sustained and strengthened, as they are the program activities which can have the largest long term systemic impact. Development Associates, Inc. Evaluation of the Local Government 17 May 2002 Reform Project in Macedonia 3. An important training and technical assistance activity for USAID to be involved in, over the next three years, is implementation of the LLGF and other decentralization laws. But USAID needs an interim strategy that continues to develop municipal capacity during the period of uncertain length, until decentralization laws are passed. 4. ZELS should be sustained financially by USAID, as long as it is effective in advocating decentralization. The stakes in Macedonia are too high for dues-based sustainability to serve as a criteria for supporting ZELS. Financial self-sustainability will occur when the organization’s local credibility is enhanced and when cities have more revenue. ZELS is simply the only clear and unified voice in the political arena advocating decentralization – a key USAID policy interest. Specific Recommendations These recommendations are program specific and are intended to establish a framework for considering specific elements of an extended LGRP. 1. Existing CICs should be more intensively utilized by cities. Instead of being largely information and referral centers, they can be proactive catalysts for permanent service improvement. CICs have modern offices, trained staff and increasingly sophisticated understanding of local public administration. USAID/LGRP should capitalized on this by fostering an environment in which CIC’s roles are expanded. Specifically, efforts should be undertaken so that: • CICs can be used as a Research and Analysis organization by making better use of the pattern of complaints they receive in order to foster systemic changes in the way services are delivered. Starting now, LGRP can prepare CIC staff to perform this role in the more complex municipal environment envisioned after decentralization; and • CIC staff can take on specific analytical requests of the Mayor and serve in a broader staff resource role, than is presently the case. The method and approach of forming a CIC is now well developed. The network of existing CICs, with modest support from LGRP or other donors, can be expanded with little drain on LGRP resources. These should focus on supporting the network of CICs and assisting CICs to inform and support municipal and public enterprise service improvement; in essence, communicating citizen need for systemic service improvement to local governments and helping fashion responses. Developing this capacity may require training CIC staff in management analysis and assisting them in initial analytical work, a task which could be performed by LGRP’s Municipal Management staff. 2. It is in USAID’s policy interest to continue to play a major strengthening role with ZELS. Without a strong ZELS, there is no political advocacy for decentralization in Macedonia. Concerning sustainability, USAID should frame consideration of support for ZELS in the context of the Ohrid Framework Agreement and US policy support for the decentralization described in the agreement. That is, US policy considers sustaining Macedonia as a unitary state a top priority, and decentralization as a means to achieve this goal. In this context, USAID’s support of ZELS should not be constrained by questions of self-sustainability, Development Associates, Inc. Evaluation of the Local Government 18 May 2002 Reform Project in Macedonia rather USAID should take the responsibility of sustaining ZELS, perhaps in cooperation with other major donors, for a period of 3 – 5 years or longer. The stakes are simply too high for ZELS to fail or to perform at a mediocre level. Long-term self-sustainability will come significantly later in the decentralization process and will depend on two factors: 1) Significant enhancement of local revenues, and 2) improvement in macroeconomic conditions. ZELS serves as the cornerstone of the strategy to foster decentralization, not only for USAID but for all donors. It remains fragile and in a developmental stage. ZELS’ main goal is to serve as a political advocacy group for municipal interests. It also needs to be a member service organization, because its members need technical support and because, for many members this is the only service they see. Effective membership services along with advocacy is key toward maintaining ZELS’ momentum and credibility with members. Both functions, advocacy and member services, need to be strengthened in the short-term. According to interviewees, the quality of direct member service, as opposed to policy advocacy, is a constraint to achieving broader based support from its members. The addition of key staff, such as a lawyer or other technical expert who could provide direct support to members, should be supported. The rate of increase on member dues payments should be tracked as an indicator of member satisfaction with the organization, but not used as an indicator of LGRP program effectiveness, as there are too many externalities to use the measure for this purpose. In addition, ZELS’ advocacy role should be strengthened. USAID should support ZELS on a technical level, by providing technical experts in each of the competencies which are to be decentralized. In developing this approach, USAID should consider funding a Macedonian staff member with subject matter knowledge, to support each of the decentralization committees that ZELS is establishing. This staff would be on ZELS’ payroll and receive technical support from the short-term or intermediate term expatriate expert(s) working on decentralization of a particular competency. Teaming a local ZELS based staff person with an expatriate expert would foster knowledge transfer and enhance the prospects for sustainability. The Evaluation Team has several immediate action recommendations concerning ZELS and USAID support for it. Building on its current program, USAID should provide sufficient institutional support, so that ZELS has, 1) adequate administrative support, 2) technical specialists to support ZELS policy advocacy in the main competencies to be devolved, and 3) the capability of providing direct member services. To do this, USAID should either make a direct grant(s) to ZELS, or enable DAI to do so. Technical Assistance from LGRP to support implementation of these resources should be continued. Open and competitive hiring practices similar to the one used to hire the current Executive Director, should be utilized. To the extent possible, these steps toward institutional support should be taken now, with a view to continuing them in subsequent years. The Evaluation Team envisions USAID support to ZELS to enable it to provide the following: Development Associates, Inc. Evaluation of the Local Government 19 May 2002 Reform Project in Macedonia • Technical staff to support policy analysis and advocacy in the competencies to be devolved. Urban Planning and Local Finance appear to be top priorities now, with Education and Health contemplated in the near future; • Member Services Staff to provide direct member support services in critical areas. Municipal Law appears to be a priority; • Administrative staff and systems to manage, coordinate and control resources and operations. Several interviewees indicated the need for ZELS to enhance its accounting and financial control systems to enable it to be a credible grant recipient. This should be done quickly. Consideration should be given to providing the resources to support a local accounting firm managing ZELS accounts, in the short term, while developing a permanent financial system. Institutional support commitments to ZELS should not be contingent on its development of western style financial controls, because of the time delays involved. Rather these function must be developed concurrently. Consideration of using DAI’s local accounting firm to provide interim support to ZELS may be appropriate; and • As the only bona fide municipal association in the country, ZELS is starting to be beset by offers to serve as a partner for various programs of training and development, aimed at the municipal sector. While attractive, these opportunities have the potential to stretch ZELS too thin--reducing focus on its advocacy and member service activities. The opportunities can represent a significant risk to ZELS’ effectiveness. USAID/LGRP assistance should help ZELS vet these opportunities. The institutional support recommended above, should reduce the need for ZELS to generate overhead from such activities. One approach USAID/LGRP may consider is assisting ZELS develop a training and technical assistance arm, incorporated as its own legal entity, to carry out such functions. If established, it should be financially autonomous and organized in a way which does not draw financial or human resources from ZELS. This is not a time for ZELS to squander core resources on entrepreneurial activity. 3. The CAB program, now in its infancy, should serve as a primary source of information in developing a Technical Assistance program targeted at Communal Enterprise organizations. In fact, most direct municipal services are delivered by communal enterprises that, while under the political control of the City, are independent organizations. Cities, in general, perform few direct services. Most of their tasks are regulatory and they typically have few employees. On the other hand CEs are the organizations which provide water, clean streets, and a wide range of public works services. If the LGRP is to enhance citizen perception of local government as an effective service provider, placing emphasis on CEs is appropriate. Two aspects of a CE development program are envisioned: • Developing an Association of Public Enterprise Managers. This would follow the path pioneered by AFO, and serve primarily as a member education and professional development organization. A needs analysis should be carried out during the remaining months of the current LGRP program with a view toward making this an association development activity in the next phase of the program. There is an existing association (MAKKOM) which has received very mixed reviews, according Development Associates, Inc. Evaluation of the Local Government 20 May 2002 Reform Project in Macedonia to interviewees. It is organized as a commercial company and participation is limited. The needs analysis should evaluate whether there is any benefit to building on the existing organization. One approach to developing an organization may be to use a former officer of the American Public Works Association to assist in the needs assessment. This approach was followed in Slovakia with positive results. • Developing a Technical Assistance program for CEs, which builds on the preliminary work already performed by LGRP (CABs), but which is focused specifically on activities which enhance CE performance, in ways citizens can observe. Target CEs should be those with CABs, or which have undergone other developmental experiences. It would be more intense than the recent efforts and should include some provision for the acquisition of capital equipment, perhaps in the $50,000 range. The provision of a capital “allowance” would enable the TA to be provided on a Memorandum of Understanding (MOU) basis, in which LGRP and the CE have explicit reciprocal obligations that result in visible service improvement. The support of the Community Self-Help Initiative (CSHI) should be incorporated, where appropriate, in the development of this program. 4. In the decentralization process, the most likely scenarios are that existing Ministry staff serving a city or region, are assigned to City Administrations, and accordingly will report to the Mayor. Budget resources for these functions will flow through the City Government, based on the particular decentralization law and the LLGF. This will give Mayors and other city staff a broader range of management and supervisory responsibilities, and make the municipal finance and personnel functions more complex. USAID should develop training and TA programs to assist municipalities in the following: • Developing management practices which enable effective coordination of a wider range of functions and increased responsibilities. • Developing financial management practices that support and control a wider range of functions and wider range of revenue sources. One non-traditional, but potentially effective way to approach this, is to identify a line ministry (e.g. health, education, or transportation) and two or three municipalities that are willing to serve as prototypes for decentralization. In essence, decentralization would be “modeled” in these two prototypes, and the learning from the pilot would help smooth implementation in the two pilot cities and ministries. It would also serve as the basis for planning national implementation. Developing this would require finding two cities and two Ministries in which participation would be viewed positively. For this approach to work, program design, identification of partners, and understanding the incentives for participation would be key to an effective project. Taking this approach provides the opportunity to “test” the administrative obstacles and benefits of decentralization – both for municipalities and for Ministries. The staff of the Ministry of Finance expressed interest in a project which would develop a prototype of the kinds of human and systems skills which would be required, after the passage of the LLGF, and at the onset of decentralization of competencies. Ministry staff Development Associates, Inc. Evaluation of the Local Government 21 May 2002 Reform Project in Macedonia mentioned that some local finance officers are now Ministry employees and others are city employees, but that all needed training in the systems that the new laws will require. The learning from the prototypes could serve as the basis for implementation training and even serve to inform government staff planning implementation. Design of a prototype project(s) could be performed before the end of the current contract, and implemented in the next phase. 5. Since the timing of decentralization is not predictable, USAID’s local government training efforts, except for the recommendation above and others that are closely tied to advancing the policy reform agenda, should focus on areas which meet two criteria: 1) Functions not dependent on decentralization (e.g. management analysis training for CIC staff), and 2) Topics of high priority to local officials, e.g., LED and Communal Enterprises. LGRP has modest current investments in these areas and demand is high. In addition, both topics have the potential to create modest, but visible, improvements in the short term. 6. Many Macedonian municipalities are small; 45 of the 124 municipalities have populations under 5,000. The population of these municipalities represents approximately 6% of the country’s population. It is an open question as to whether these municipalities are viable as administrative entities, especially, in the context of decentralization. The general view appears to be that many, but not all of these municipalities will be combined with larger neighbors. In addition, there are 28 municipalities with populations from 5,000-10,000. These comprise approximately 11% of the population. The Evaluation Team recommends that the LGRP take steps to improve the understanding of the needs and capacities of smaller municipalities, that is, those under 10,000 in population. A project along the following lines is recommended: • Identification of current patterns of Joint Administration, resource and personnel sharing; • Identify the manner in which Ministries provide services on the list of competencies to be decentralized to smaller communities; and • Based on the information gathered and assessed, develop strategies that may maintain smaller municipalities as independent political entities, but effective and efficient municipal governments. These might include: − Model forms of Joint Administration; − Model arrangements for managing competencies to be devolved; − A circuit rider program which enables a finance specialist, architect or engineer, to be shared by several communities; and − Arrangements for contracting for service with Ministries. Development Associates, Inc. Evaluation of the Local Government 22 May 2002 Reform Project in Macedonia In developing approaches to maintain smaller municipalities, the LGRP should be aware that low administrative capacity appears to be one argument for reducing the number of municipalities. The analysis recommended above, could serve to rebut or confirm arguments for disestablishing small communities, as well as for serving as the basis for a small municipality strengthening program. This task could be performed before the end of the current LGRP program. OSCE staff in Skopje are very familiar with this issue. They should be consulted in the development of any program focused on smaller municipalities. In the short term, LGRP could identify “best practices” currently in use by small municipalities and disseminate them via the ZELS newsletter and/or a best practices conference. 7. In the future, monitoring and evaluation should take a more central role in the LGRP program activities. One reason for this is that impacts should be easier to measure, as Training and Technical Assistance becomes more focused on technical skill building, as opposed to general management and leadership development. Monitoring and evaluation should be incorporated into program design, in a way that gives LGRP staff accurate information quickly information they can use to fine tune programs and as a basis for continuous program improvement. The monitoring and evaluation should not be designed solely for the purpose of reporting on an IR basis to USAID. 8. The Southeast Europe University, newly established in Tetavo, has a program of study in Public Administration. According to University staff, the Public Administration program has two main foci: Central Government and Local Government. Undergraduate students will be ready for practical learning placements in the 2003-2004 academic year. In addition, SEE does have the capability to develop and provide short courses and seminars for especially targeted groups, such as Mayors, Councilors or municipal professionals. SEE has the capability to provide modern instruction in Macedonian, Albanian and English. It also has the ability to train trainers. The Evaluation Team recommends that LGRP identify appropriate ways to utilize SEE as a training resource. In 2003 a few pilot training programs could be carried out for LGRP by the University, in order to test its capabilities and begin to forge long-term relationships with the municipal community. Very significant donor resources, including USAID funds, have been committed to SEE. Ways to harness this investment to a municipal development agenda should be fully explored, as the potential exits for this institution to be a significant player in the development of Human Resources for municipal administration over the long-term. 9. City Councils and their individual members have a very significant role to play in major municipal policy decisions. This role will become stronger and more defined, as the LLSG is implemented and competencies are devolved. While extensive policy leadership training has been provided to Mayors and other officials, no significant training has been provided to this key group of elected municipal decision-makers. A number of municipal officials, including Mayors, voiced the opinion that it was important to secure training for Councils and Councilors in the duties of the Council, and on municipal governance generally. Several Mayors voiced the opinion that having Councilors educated on their roles in government would enhance effective working relationships. USAID/LGRP should perform a training needs-assessment using the Chamber of Councilors in the ZELS governance structure as a starting point. Typically, future Mayors are among Council members and Councilors who self-select for training programs and tend to be leaders on Councils and in their Development Associates, Inc. Evaluation of the Local Government 23 May 2002 Reform Project in Macedonia municipalities. The needs-assessment could be completed before the end of the current LGRP contract. 10. The current situation among municipal finance officers is that some are direct employees of municipalities and some are employees of the MoFin. Discussions with AFO leaders indicated that they envisioned an organization that was open to all professionals working in the arena of municipal finance – no matter who the employer was. This appears to be a useful and productive approach. In a small unitary state like Macedonia, all municipal finance officials will work in the same highly integrated financial system that is controlled by a uniform set of rules established by the MoFin. Discussions with MoFin officials reinforced this view. USAID/LGRP should assess the training needs of officials dealing with municipal finance, with the goal of determining whether a joint approach among AFO, MoFin and locally employed municipal finance officials can be developed. One attitude encountered is that nothing can be done until the passage of the LLGF. Another is that certain skills, procedures and rules are certain to be required, and that training in these would be valuable now. The needs assessment will have to clarify this. 11. Expert expatriate staff members in Policy Reform and Association Development components of the LGRP will be required for the foreseeable future. In the Policy Reform area, the decentralization issues are complex and will take a long time to sort out and solve. LGRP needs to be a player in these decisions and, to do so, needs to bring international expertise to the table, as it is currently doing. In Association Development, ZELS, in particular, needs on going development assistance from an experienced expatriate. Because it is critical to the politics of decentralization, sustained support as well as funding is required. An expatriate is also needed to support the development of the Public Enterprise Management Association and to work with the AFO and MoFin to develop prototypes for joint training. 12. Citizen Participation and Municipal Management are well led by Macedonian staff. These programs will need expatriate assistance in program design, monitoring and evaluation. Macedonian staff can benefit from the international perspective in these areas and coaching; but once designed, programs will not need direct program management by expatriates. The areas that need design work have already been mentioned in the recommendations on CICs and CABs. This work can be performed with expatriate assistance, before the end of the current contract. 13. Expatriate assistance can be used in developing one or two new program areas for the LGRP. These areas might include: • Development of “Model Cities”, where donor assistance is concentrated and highly coordinated, based on local commitments. A competition for participation might be developed and the finalists, perhaps two cities, would receive intensive on-site technical assistance and money from LGRP and others, in exchange for an aggressive commitment to support modernization and service improvement. • Assessment of the feasibility of a Municipal Development Loan Fund, in cooperation with the World Bank. Development Associates, Inc. Evaluation of the Local Government 24 May 2002 Reform Project in Macedonia • Development of a locally administered (and partly municipally capitalized) revolving loan fund for Small and Medium Enterprises (SMEs). This could build on the Prisma experience. • Development of increased participation by the private sector and private sector techniques in the provision of public services. This would build on the work being carried out in this area by GTZ, as well as on initial work on cost recovery that is prerequisite to a World Bank Water Loan to Skopje. 14. A recurrent theme among those interviewed was that the LGRP training and technical assistance was of high quality, partly due to the international perspective presented by the training staff and consultants. To build upon that experience, in a cost effective way, USAID might consider the application of its Resource Cities program. The program has proven to be successful in small countries, whose capital cities have formed partnerships with U.S. cities of similar character. The partnerships foster exchanges of experts, program materials, and short-term or middle term technical assistance. Skopje appears to be a strong candidate for such a Resource Cities twinning arrangement, in part because much of the analysis of its administrative structure and service delivery needs has already been carried out. While Skopje currently has a Sister Cities arrangement with Tempe, Arizona, a Resource Cities relationship with an American city or county of similar makeup to Skopje would complement the Sister Cities twinning. That is, Sister Cities programs emphasize cultural and commercial ties, while Resource Cities emphasizes service delivery and governance information exchanges. SUMMARY The recommendations all build on the foundation created by the LGRP to date. The major themes are, 1)maintaining momentum, 2) increasing municipal management capacity by more intensively using the resources and knowledge of Citizen Participation organizations, 3) strengthening ZELS, and 4) investing in new areas with potential for significant impact, e.g., prototype decentralization, private sector involvement, economic development.. LGRP Macednonia Report-Final Draft-EvalQC-07.doc Evaluation of the Local Government May 2002 Reform Project in Macedonia APPENDICES Appendix A Cover Letter and Statement of Work ..........................................A-1 Appendix B Team Composition and Methodology.........................................B-1 Appendix C Performance Measurement Plan..................................................C-1 Appendix D List of People Interviewed ..........................................................D-1 Appendix E Municipalities by Population.......................................................E-1 Appendix F Political Context...........................................................................F-1 Appendix G Program and Political History Timeline......................................G-1 Appendix H Questions Used to Guide Interviews...........................................H-1 Development Associates, Inc. Evaluation of the Local Government A-1 May 2002 Reform Project in Macedonia U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT REGIONAL SERVICES CENTER for Europe and Eurasia December 20, 2001 Ms. Patricia McPhelim Checchi/Louis Berger Joint Venture 1899 L Street, NW, Suite 800 Washington, DC 20036 Subject: Request for quotations (RFQ) No. 165-02-14 Evaluation of Local Government Reform Project in Macedonia Ref.: IQC No. AEP-I-00-00-00022-00 Dear Ms. McPhelim: The United States Government, represented by the U.S. Agency for International Development (USAID) Mission in Macedonia, has asked me to issue the subject solicitation for a proposed Task Order under the referenced IQC to provide technical assistance as fully described in the attached Statement of Work. Under the “Fair opportunity to be considered” Clause (Article F.6), personnel qualifications and schedule of availability will determine selection. Accordingly, please submit personnel resumes and a schedule electronically to “silles@usaid.gov” by January 11, 2002. The financial magnitude for this activity is approximately $160,000. The name and the address for the proposed Task Order Ombudsman is stated below. The ombudsman will review complaints, if any, from offerors and ensure they are afforded a fair opportunity to be considered, consistent with the procedures in the IQC contracts. Development Associates, Inc. Evaluation of the Local Government A-2 May 2002 Reform Project in Macedonia Margaret Dula Ombudsman for Acquisition and Assistance Office of the Procurement Executive USAID Ronald Reagan Building 1300 Pennsylvania Avenue, NW Washington, D.C. 20523-7700 Tel.:(202)712-5162; Fax:(202)216-3380; E-mail: MDula@usaid.gov Issuance of this solicitation does not constitute a commitment on the part of the US Government to award a contract nor does it commit the government to pay for costs incurred in the preparation and submission of a proposal. Please note that you are not authorized to proceed with any of the work described in the SOW until you have received either a fully executed task order or written authorization from the Contracting Officer. If you have any questions concerning the procurement, please feel free to contact Ms. Szilvia Illes at (36-1) 475-4626, telefax (36-1) 475-4988 or 475-4991 or by e-mail at silles@usaid.gov. Sincerely, Catherine A. Mallay Regional Contracting Officer USAID/RSC/RCO - Budapest Encl.: a/s Development Associates, Inc. Evaluation of the Local Government A-3 May 2002 Reform Project in Macedonia ATTACHMENT I - STATEMENT OF WORK STATEMENT OF WORK FOR EVALUATION OF THE LOCAL GOVERNMENT REFORM PROJECT IN MACEDONIA I. OBJECTIVE The United States Agency for International Development in Macedonia seeks the services of a Contractor to conduct an evaluation of the USAID-funded Local Government Reform Project (LGRP) for Macedonia, implemented by Development Alternatives Inc. The evaluation will assist USAID management to evaluate the progress of the LGRP in achieving results and to make recommendations for future year activities. The evaluation should provide USAID, LGRP, and Macedonian counterparts with an evaluation of: • LGRP activities to date, especially the appropriateness, quality, and effectiveness of the technical assistance delivered to the partner local governments and other institutions participating in the LGRP; • the progress toward achieving results as defined in indicators, measures, and targets, specified in the performance monitoring plan , to judge the performance of LGRP; • correlation of LGRP implementation design in achieving the goals laid out in the SO 2.0, I.R.2.3 results framework; . • record successes, obstacles and lessons learned about getting impact and results, and make recommendations to USAID to improve the design and implementation of its local government activities to maximize the positive impact of it. II. ACTIVITY BACKGROUND The Local Government Reform Project (LGRP) is USAID’s program to support Macedonia’s decentralization efforts and strengthen local government. The objective of the three-year LGRP effort is to help build a local government that is effective, responsive and accountable so that Macedonia citizens will live better. USAID believes that well-managed local government is crucial for the long-term stability of democracy and the continuation of economic development. Local government reform and decentralization are one of the top priorities of the Government of Macedonia. LGRP’s efforts will support these priorities by improving local governments’ capacity to manage their resources more effectively. The LGRP consists of four components, designed to complement and cooperate with each other. 1) Policy reform component provides technical assistance in drafting and amending laws in the local government area, as well as in implementation of them, once they are enacted; 2) Citizen Participation component provides assistance that involves institutionalizing citizen involvement in local government in the delivery of public services, providing information Development Associates, Inc. Evaluation of the Local Government A-4 May 2002 Reform Project in Macedonia and assistance to citizens at citizen information centers, and development of effective relationships between local government, the media and NGOs. 3) Municipal Management component provides assistance to local governments in building their capacity to effectively execute devolved responsibilities in areas of local economic development, human resource development, budget/finance in the delivery of public services, public service performance standards, improved local tax collection, and information management. The program works with targeted cities to develop generalized models for management innovations that can then be adopted by a larger number of Macedonian municipalities. 4) Association Development component assists the two major municipal associations, the National Municipal Association (ZELS) and the Association of Finance Officers (AFO) to improve their ability to serve their members with information and other products that contribute to more effective local government, and advocacy of municipal interests in the policy process. The Local Government Reform Project is a Task Order No. 801, under the IQC EEU-I-99- 00012-00, between USAID and the Development Alternatives Inc. signed on September 29, 1999. Initially this was a two-year activity, with a completion date of September 30, 2001 and an option for a third year extension. The activity was extended for an additional, third year, till September 30, 2002. The total amount obligated for this activity is $ 6,800,000.00. The overall objective of this program is to develop more effective responsive and accountable local government. This can be accomplished by creating an enabling legal environment and developing policies and institutions that will support decentralization; establish effective municipal management techniques; develop sustainable ”two-way” communication mechanisms between citizens and local governments; and, strengthen the national municipal associations. III. PURPOSE OF EVALUATION Over the last two years, USAID has implemented a rather comprehensive local government support program in Macedonia. This program provided assistance to assist local governments become more effective, responsive and accountable. The conditions of the country required that the Mission uses both “top-down” and “bottom-up” approaches. This means assisting the central government in reforming the laws, building the capacity at the local level, as well as strengthening municipal associations that present a linkage between the two levels. As already mentioned, this program has been implemented for two years now, but has not been systematically evaluated to determine whether the used approach is the most effective, given the constraints of national governments, and the needs of local communities. USAID intends to continue this program beyond the currently estimated completion date of this particular activity and, therefore, will undertake this assessment to determine how USAID can best develop and implement this program in future. To be more specific, the purpose of the evaluation is to (1) assess the progress of the Program toward achieving its stated program objectives (as defined in the Task Order and the Amendments) and to document its successes, obstacles to getting desired results, and lessons learned; (2) to analyze the key problems and the overall conditions of the Development Associates, Inc. Evaluation of the Local Government A-5 May 2002 Reform Project in Macedonia local government sector in Macedonia at this time; and (3) to consider options for follow on assistance to support the local government sector’s development. Tasks Listed below are tasks that the team should perform with a list of specific questions. In a sense, these are the kinds of questions the USAID activity managers should be asking as they develop and implement activities. The assessment team will use these questions as a guide to develop the final research questions. It will not be necessary to respond directly to each of these questions in the assessment report. Questions to be addressed in the assessment report are found in the lessons learned section below. For each task the team should take into consideration the internal and external factors (Macedonia crises 2001, Kosovo crises 1999, local elections 2000) which may have affected program implementation and document the effect these circumstances may have had on the program’s ability to achieve stated objectives. The team should formulate conclusions and recommendations for each of the tasks given below. They will serve for a potential design of follow on assistance in this area. 1. The Appropriateness, Quality, and Impact of the Technical Assistance Delivered The team should evaluate the appropriateness, quality and impact of the approach used in terms success in achieving the desired impact, and the sustainability of the program in the local government sector once USAID funding will cease. The issue is to determine, to what extent USAID’s contractor developed approaches and techniques for improving local government management that are effective, responsive to demand, useful, and affordable. A special emphasis should be given to the techniques developed under the Municipal Management and Citizen Participation components. • Has the program developed approaches and techniques that are desired, useful, affordable, and cost effective? Did the local governments find the approaches and techniques developed by USAID-supported local government programs useful and affordable? How have the programs demonstrated that the use of these approaches and techniques is cost effective, sustainable and enhances management capacity? • Was there sufficient emphasis on the sustainability of project achievements? Is there any evidence of sustainability, what is it and how measured? • Did the approaches and techniques developed address key local concerns for improving local government management? How were the key concerns determined? • What changes to improve local government management have been brought about through the use of these approaches and techniques? What observable and measurable results have there been? Has the proper implementation and use of the approaches and techniques developed led to local government that is more effective and more responsive to the needs and desires of citizens? Development Associates, Inc. Evaluation of the Local Government A-6 May 2002 Reform Project in Macedonia • What means were used to monitor and evaluate the appropriateness, quality, and impact of the assistance delivered? Have USAID and the implementer developed an approach or a system to monitor and evaluate effectively the appropriateness, quality, and impact of the assistance delivered? • What number of local governments used the approaches and techniques developed by the USAID activities? Were activities designed to encourage use by a large number of local governments? If not, why not? • Is the methodology of working with limited number of local governments sufficient to attain impact? • How did or how can USAID encourage larger numbers of local governments to use the approaches and techniques developed by local government programs? • What is the overall impact of the technical assistance and training? What is the depth and breadth of the results achieved? Is there national impact? • What types of public relations or marketing work best? • Are a growing number of local governments requesting assistance from the program? Are they using the assistance to improve the way they operate? What steps might the implementer take to increase local government involvement and have national impact? Are local governments willing to use their own resources to procure services to improve their management? If not, why not? • Is there evidence that a market is emerging for the kinds of technical assistance activities offered by the local government programs? How extensive is this market? What steps have been taken and/or what steps could be taken to encourage the further development of this market? How can the programs effectively market their products? 2. The Role of Local Organizations The team should evaluate whether the local government activities have been developed collaboratively with local governments and local organizations and if they are desired, useful, and affordable. The team should also explore the possibility for use of local expertise (academicians, think tanks, research institutions, etc). A special emphasis should be given to the two major municipal associations and the USAID assistance delivered to those organizations. • Have local organizations developed a capacity to play a role in local government affairs? What is the role that they see for themselves? Do they have the capacity to play this role? • What can USAID do, if anything to ensure greater sustainability of these organizations in future programs? What are the obstacles to sustainability? What is the potential for sustainability without USAID assistance and support? If the organizations are not sustainable, should USAID continue to try to make them sustainable, or should USAID work with them for project results only? Development Associates, Inc. Evaluation of the Local Government A-7 May 2002 Reform Project in Macedonia • Have local organizations developed activities nationwide? • How has USAID contributed to improving the functioning of the two municipal associations? • Has the credibility of the associations among its members and the central government officials increased? • Does USAID’s vision of the role of associations correspond to the vision associations have of themselves? To what degree is it realistic to think that the associations can play this role, and they can become sustainable without USAID assistance? • Has the association played any role during the ethnic crises in Macedonia? 3. Political and Administrative Context The team should evaluate USAID activities in the local government area, especially activities to promote legislative reform. It should take into consideration whether the enabling political environment for effective decentralization exists. To what extent local government leaders and local government associations have influence over the national politics and policy on decentralization and local government management improvement. The team should look at the effects of the ethnic crises in the country to the decentralization process in general, and the dynamics for its realization. • What has been the impact of the activities to promote decentralization and policy reform? • What role can USAID play to attain the greatest impact in policy reform? Wht is the appropriate role of USAID vis a vis other Donors or international organizations, in the next period? • Assuming key laws are passed, is there sufficient political will and support to implement the laws? • To what extent did the political environment aid or hinder successful program development and implementation? What other in-country circumstances might aid or hinder program development? • What political factors support programs aimed at improving local government program? • What political factors hinder programs aimed at improving local government management? • At what point does the political context make it not worthwhile to implement a local government programs? • How does the status of decentralization and administrative reform affect the development and implementation of local government activities? How much “IMPACT” can USAID expect until reform legislation is implemented? Development Associates, Inc. Evaluation of the Local Government A-8 May 2002 Reform Project in Macedonia • Can local government leaders and local government associations serve as effective advocates for more effective decentralization and improved local government management? Is the program effective in bringing local and central government together for policy change? 4. Lessons Learned The team will describe how the successes and/or failures of LGPP could contribute to the design and implementation of future USAID local government programs. The team will provide answers to the following general questions: • What results have been achieved? • What are the principal lessons learned from the LGRP? • How could the Contractor have implemented the program more effectively? • How could USAID have identified more effectively customer needs and interests? • What changes, if any, should be made in overall project design and strategy to ensure its contribution to achieving LGRP targets? • Is it possible or desirable to replicate the activities in other countries? More specifically, the team will describe what types of changes might be made to LGRP management and/or activities in the coming year that would help LGRP to meet its goals and objectives. These changes should range from general to specific – e.g., LGPP should work with fewer local governments or LGRP should hold more informational and training workshops for local governments and the professional staffs of partner and non-partner local governments. Nevertheless, the changes recommended should be those that could be reasonably expected to have an effect on LGRP results in the last year of program implementation. 5. Assessment of Progress Towards Achieving Measurement Results The Evaluation Team is encouraged to review the results and sub-results of the SO 2.0, I.R. 2.3 results framework as it prepares this section. USAID/Macedonia has developed a SO 2.0, I.R 2.3 results framework to measure progress toward achieving the strategic objective. (See the results framework) The indicators and measures in the DAI LGRP contract are reflected, since this is the only activity contributing to I.R 2.3. After reviewing LGRP activities, the assessment team (primarily the team leader) will determine to what extent the LGRP’s activities have contributed to achieving results as defined in the I.R 2.3 of the S.O 2.0 results framework. USAID recognizes that it may be too early in the process of decentralization to conclude if results have been achieved or not. Nonetheless, the team is encouraged to make its best estimates and prognostications based on the objective external and internal factors. More specifically, with the LGRP activities in mind, the assessment team will provide answers to the following questions, providing factual information to support the responses: Development Associates, Inc. Evaluation of the Local Government A-9 May 2002 Reform Project in Macedonia • To what extent has the LGRP achieved its stated objectives with respect to indicators, measures, and targets listed in the contract? To what extent will it achieve them? To what extent has the local government program achieved the Strategic Objective objectives? • Does the way in which the indicators, measures, and targets are defined lend itself to accurate measure of results? • To what extent is there, or will there be, an improved policy and legal framework to support local government functions? • To what extent has public participation in local government decision making increased? Is this due to USAID program? To what extent will it increase? • To what extent has the capacity of local governments to deliver services and manage local resources effectively increased? To what extent will it increase by the end of LGRP and after? • To what extent has the capacity and role of municipal associations increased? Is this due to USAID program? To what extent will it increase by the end of LGRP and after? • To what extent have the local organizations become more sustainable? To what extent will they become more sustainable and if not what are the obstacles? 6. Other Donor Assistance The team should, to the extent possible, report on assistance being provided by other donors. • Is there collaboration and coordination with other donors providing assistance in the local governance area. If yes, to what extent and how effective it is? • Does a division of areas for assistance exist? Are there overlapping activities? Are there synergies in the program implementation? IV. METHODOLOGY An outside Evaluation Team consisting of two U.S. evaluation experts with expertise in the area of local governance and one Macedonian logistics support person (see section VI below for team skills required) will conduct this evaluation. A) Before undertaking fieldwork in Macedonia, team members shall familiarize themselves with previous and current documentation about the project. USAID/Macedonia and DAI will ensure that this documentation is available to the team immediately after the contract is signed. The literature includes: • RFP Scope of Work Development Associates, Inc. Evaluation of the Local Government A-10 May 2002 Reform Project in Macedonia • DAI Proposals • The task Order and the Amendments • LGRP Workplans • LGRP Quarterly Reports • Key manuals or tools developed by LGRP • Written briefing materials on the program • Other referential or historic documents which might be identified by DAI or USAID USAID/Macedonia and DAI will provide the contractor with input and guidance in setting up a schedule of interviews and site visits, but the responsibility is with the Contractor. The schedule will be defined as much as possible before the U.S. team member arrives in Macedonia and will be finalized as soon as possible after the team arrives in Macedonia (not later than close of business February 6th). The draft schedule is to be submitted to USAID and DAI Macedonia for review and comments at the initial team planning meeting and a final schedule will be submitted for approval before work is commence (not later than close of business February 6th). Prior to his/her departure to the field, the U.S. evaluator will spend three days in Washington D.C. conducting interviews and reviewing background program documents with relevant USAID officials in the Europe and Euroasia Bureau and DAI home office, and other relevant organizations to gain better understanding of the situation in Macedonia and the Local Government Program. B) Following the U.S. phase of the evaluation, the U.S. evaluator will proceed to Macedonia for four weeks of fieldwork. The U.S. evaluator will brief the Macedonian member of the evaluation results of the U.S. phase of the evaluation. Prior to the U.S. team member’s arrival, the Macedonian team member will receive a list of names from USAID and DAI, for the team to meet with. In general, the recommended institutions and organizations that the team should meet with are the following: • Ministry of Local Self-government • Ministry of Finance and other relevant Ministries • Association of Mayors (ZELS) • Association of Finance Officers • Pilot and target cities that LGRP has provided assistance (the list will be provided) • Other relevant USAID projects (PRIZMA, CSHI, DemNet, etc) • Think tanks, Academicians and other research institutions • Other Donors providing assistance to the local government sector (EU, World Bank, UNDP, SIDA, Swiss Government, GTZ, etc) • A sample of non-partner local governments The Evaluation Team is encouraged to identify additional Macedonian local governments and institutions to visit based on its review of materials and its determination of where useful examples might be found. The first day in country the Evaluation Team will meet with USAID and DAI to establish clear expectations about the outcomes of the evaluation and go over the goals, schedule and methodology of the evaluation. The team will be required to meet with USAID/Macedonia Development Associates, Inc. Evaluation of the Local Government A-11 May 2002 Reform Project in Macedonia halfway through the four weeks of fieldwork and brief USAID orally on their progress and findings to date. At the end of the fourth week the team will also be required to give a final exit briefing to USAID/Macedonia. V. DELIVERABLES The team will prepare in the field a draft final report. The following sections shall be included in the report: • An Executive Summary – ( 3- 4 pages) a document containing a clear, concise summary of the most critical elements of the report, including the recommendations. • A Table of Contents • An Assessment Report (no more than 25 pages), which discusses the major findings and the related issues and questions raised in Section III. In discussing these findings, the assessment shall also address the following: — Purpose and study questions of the assessment; — The economic, political, and social context of the LGRP; — Evidence/findings of the study concerning the assessment questions; — Briefly stated conclusions drawn from the findings (including lessons learned); and — Recommendations based on the assessment’s findings and conclusions. • Evaluation Report Appendices, including: — A copy of the assessment scope of work; — Team composition and study methods (1 page maximum); — USAID/Macedonia’s SO 2.0 I.R 2.3 results framework; — A list of documents consulted, and of individuals and agencies contacted; and — More detailed discussions of methodological or technical issues as appropriate. A draft report will be submitted to USAID/Macedonia prior to the Evaluation Team leader's departure from Macedonia. USAID/Macedonia will provide the Evaluation Team leader with comments within 10 days of the draft report submission. The Evaluation Team shall incorporate all comments and submit a final report to USAID/Macedonia within 10 days of receipt of comments. The USAID/Macedonia Activity Manager/CTO will be responsible for review and approval of the final report. The Evaluation Team leader shall be responsible for report production and will provide the final deliverables to USAID/Macedonia on a diskette (in Microsoft Word 97), plus 7 printed and bound copies. The Evaluation Team leader shall provide 3 copies to PPC/CDIE/DI in accordance with normal AID/W requirements. VI. TEAM COMPOSITION AND PARTICIPATION The team should comprise of two U.S. based evaluation experts, one Macedonian logistics person and two Interpreters. Development Associates, Inc. Evaluation of the Local Government A-12 May 2002 Reform Project in Macedonia One U.S. based expert – Team Leader should have at least 10 years of experience in the development and/or implementation of local government development. Previous experience in municipal service, strategic management for local governments, local economic development, citizen participatory approaches to local government decision making processes is highly desired. Familiarity with local government programs and issues in Macedonia preferred. Prior experience in conducting evaluations of USAID local government programs preferred. Professional experience in Macedonia or in similar countries of Central and Eastern Europe is desirable. He/she must be fluent in English and have excellent writing and presentation skills One U.S. based expert should have at least 10 years of experience in the development and/or implementation of local government programs. Previous experience in policy evaluation, fiscal decentralization and evaluating local government programs is highly desired. Familiarity with local government programs and issues in Macedonia preferred. Prior experience in conducting evaluations of USAID local government programs preferred. Professional experience in Macedonia or in similar countries of Central and Eastern Europe is desirable. He/she must be fluent in English and have excellent writing and presentation skills. One Macedonian logistics person should have experience in logistical organization and coordination. This should include: organizing and coordinating meetings, making hotel reservations, organizing transportation, etc. Knowledge of USAID, Macedonian institutions and other Donors in Macedonia is desirable. He/she should be fluent in Macedonian and should have good working knowledge of English language. Two interpreters will have as their primary responsibility to accompany the U.S. evaluators and translate at meetings. In addition, however, they may be asked to prepare short summaries in English of important studies done in Macedonia or Albanian on the local government sector. VII. SCHEDULE AND LOGISTICS The task order is expected to be awarded February 1, 2002, with work to commence shortly thereafter. USAID/Macedonia and DAI will help the team in setting up the schedule of interviews and site visits. The schedule will be defined as much as possible before the U.S. team member arrives in Macedonia. Upon award of the TO, USAID/Macedonia will review the proposed schedule and provide comments at the initial team planning meeting. A final schedule will be submitted for approval before work begins. Upon the U.S. team members’ arrival in country the team shall hold an initial briefing with USAID and DAI. Halfway through the four weeks of fieldwork the team shall brief USAID orally on their progress and findings to date. At the end of the fourth week, prior to U.S. team member’s departure for the U.S. o/a March 2, 2002 the team shall debrief USAID on its findings, conclusions and recommendations. USAID will provide oral comments at the debriefing and may follow up with written comments after the U.S. team members’ return to the U.S. Development Associates, Inc. Evaluation of the Local Government A-13 May 2002 Reform Project in Macedonia A draft report will be submitted to USAID/Macedonia prior to the Evaluation Team’s departure from Macedonia. USAID/Macedonia will provide the Evaluation Team leader with comments within 10 days of the draft report submission. The Evaluation Team shall incorporate all comments and submit a final report to USAID/Macedonia within 10 days of receipt of comments. The USAID/Macedonia Activity Manager/CTO will be responsible for review and approval of the final report. VIII. ESTIMATED LEVEL OF EFFORT U.S. Evaluation Expert 24 days fieldwork Team Leader 5 days in US preparation 10 days follow up and report preparation 4 days of travel U.S. Evaluation Expert 24 days fieldwork 5 days in US preparation 10 days follow up and report preparation 4 days of travel Macedonian Logistics person 24 days fieldwork 5 days preparation in Macedonia 5 days follow up and report preparation Interpreter 48 days fieldwork in Macedonia IX. EVALUATION CRITERIA U.S. Evaluators 75 points – total Experience evaluating development assistance programs (USAID or others) 30 points Knowledge of local government and decentralization issues 35 points Experience working in Central and Eastern Europe (CEE) 10 points The Contractor shall provide names, current e-mail addresses and telephone numbers of references of previous experience in the above areas. References shall be checked for quality of work. Schedule of Availability 10 points – total Cost effectiveness 15 points – total Development Associates, Inc. Evaluation of the Local Government A-14 May 2002 Reform Project in Macedonia X. SPECIAL PROVISIONS Duty Post Skopje, Macedonia Access to Classified Information The Contractor shall not have access to any Government classified material. Logistical Support The team is responsible for providing its own office and computer facilities. Office space shall not be provided, except as available. The team should plan to work outside the USAID premises. Team members will be responsible for providing office supplies and equipment. Secretarial services are the responsibility of the contractor. XI. REPORTING Along with delivering the assessment report to USAID/Macedonia, the following reporting is required: For G/EGAD/EM: one hard copy by mail to: John B. Crihfield G/EGAD/EM Office of Emerging Markets Center for Economic Growth and Agricultural Development Bureau for Global Program U.S. Agency for International Development 1300 Pennsylvania Avenue, N.W. Washington, D.C. 20523 202-712-1288 For CDIE: one hard copy and one electronic copy on diskette to: U.S. Agency for International Development PPC/CDIE/DI, Attention: Acquisitions 1300 Pennsylvania Avenue, NW Ronald Reagan Building M.01-010 Washington, D.C. 20523 Evaluation of the Local Government B-1 May 2002 Reform Project in Macedonia APPENDIX B TEAM COMPOSITION AND METHODOLOGY Richard Kobayashi, Team Leader, has over thirty years of state and local government development experience. Since 1993 he has served as a consultant to municipal development projects sponsored by USAID in Central and Eastern Europe. Previously he served as the policy advocate for municipal interests in Massachusetts state government, development director for a poor industrial city, and planner for a very large metropolitan water agency. He was a Loeb Fellow at Harvard’s Graduate School of Design where he studied Urban Policy. Alan Edmond, Team Member, has been an international development consultant since 1991, specializing in decentralization policy, monitoring and evaluation, and local government reform primarily in Central and Eastern Europe, Egypt, South Asia, and the Caribbean. His second area of specialization is anti-corruption training and technical assistance. Prior to 1991 Mr. Edmond was a city manager in three U.S. municipalities. After receiving a Master’s degree in Public Administration, his career began as a personnelist and municipal trainer for a state government agency. The team carried out this assessment using a variety of techniques. They met with USAID and DAI in Washington and analyzed key program documents prior to traveling to Macedonia in late February. During March they conducted a field audit of program results to date, using structured interviews to gain inputs from a large number local officials, many from DAI pilot cities, but also from several mayors and other officials from non-participating municipalities. The team analyzed four CICs, using role play and intensive questioning in some instances to gain a feel for the competence and thoroughness of local CIC staff. All major donor organizations and their implementing employees or contractors contributed information on the ways in which the LGRP complements their programs, and several USAID￾funded contractors described in detail the joint program efforts they have made in selected municipalities. The team performed a document review and met with each LGRP staff member to corroborate field findings or to seek clarification on work plan elements and projections of the issues to be faced over the next one to three years. The team concentrated on the degree to which the LGRP was meeting its work plan objectives, and it considered the ways in which program performance measures were designed, revised, and used to further program deliverables to Macedonian clients. Meetings with key ministry officials and municipal association staff and leadership contributed to the team’s understanding of the dynamics of the devolution process and the imminent changes in decentralization laws. The team met four times with USAID officials for the purpose of task clarification, progress reporting, and debriefing. The team delivered the final draft report to USAID prior to departure on March 28, 2002. Evaluation of the Local Government C-1 May 2002 Reform Project in Macedonia APPENDIX C PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 SO 2.0. MORE LEGITIMATE DEMOCRATIC INSTITUTIONS 2.0.a. Public perception of respect for democratic values Average percent of respondents to a probability survey of Macedonian adults who answered “4” or “5” (1=”often violated” and 5=”fully respected”) to 16 items listing freedoms and rights included in the Constitution of the Republic of Macedonia (e.g., freedom of press; right to vote and be elected, etc.). Disaggrega￾ted by gender, age, ethnicity, educational attainment USAID Survey of Citizen Attitudes 2001 survey: Q5: “To What extent are these rights and freedoms respected in our country? Use the five-point rating scale where 1 means that they are often violated ANNUAL Responsible person: Antoaneta Skartova 2000 39% (Margin of error: +/- 3%) T: 44— 46% A: 37% gender male: 36% female: 39% age 18-24: 33% 25-39: 30% 40-54: 40% 55+: 40% education primary or less: 35% secondary:3 6% higher: 39% ethnicity Mac: 42% Alb: 22% other: 41% T: 44- 48% T: 46- 50% T: Over 50% Development Associates, Inc. Evaluation of the Local Government C-2 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 This indicator measures the extent to which Macedonians find that such rights are broadly respected in their society. and 5 means that they are fully respected.” employ￾ment employed: 39% unemployed: 38% inactive: 35% place urban: 38% rural: 36% 2.0.b.Public perception of effective￾ness of key government institutions For government institutions: percentage of survey respondents that find the following institutions to be “very effective” or “effective to some extent”: (a) Parliament, (b) government, (c) local government, Separate percentages for (a), (b), (c), (d) and (e). In order for an overall target to be met, targets for at least 2 of the three institutions must be met. USAID Survey of Citizen Attitudes 2001 survey: for govern￾ment institutions: Q13: “How effective, do you think, are these institutions in solving problems in Annual Responsible person: Antoaneta Skartova 2000 Parl.: 40.2% Govt.: 39.9% Local Govt.: (Not collec￾ted) Judici￾ary: 43.3% NGOs: Actual: Parl.: 26.9% Govt.: 25.5% Local Govt.: 37.3% Judici-ary: 36.6% NGOs: 37.9% Target : Parl.: 30% Govt: 26% Local Govt.: 42% Judici ary: 40% NGOs T: Parl.: 35% Govt.: 35% Local Govt.: 46% Judiciary: 43% NGOs: T: Parl.: 42% Govt.: 42% Local Govt.: 52% Judiciary: 46% NGOs: Development Associates, Inc. Evaluation of the Local Government C-3 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 and (d) the judiciary. For (e) nongovernmen￾tal organizations (NGOs), percent responding that they find NGOs useful (“Very useful” or “4”). our country?” For Judiciary: Q18 “Do you think the courts are effective?” For NGOs, Q32: “To what extend to you consider NGOs useful?” (Not collec￾ted) (Brima data tables, p. 13-14) (BSC Estek data tables, Q13, Q18, Q32) This indicator is new to the PMP. No targets had been set, but for all institutions with ’00 and 01 data, percentag es declined in ’01. : 40% 42% 44% Development Associates, Inc. Evaluation of the Local Government C-4 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 IR. 2.1. INCREASED CITIZEN PARTICIPATION IN POLITICAL AND SOCIAL DECISION MAKING 2.1.a. Public participation in political activities Average percent of citizens responding that they have participated in several types of political action. Disaggrega￾ion by gender, age, ethnicity, educational attainment USAID Survey of Citizen Attitudes 2000 Brima report: pp. 25-26; 2001 survey: Q8: “I’m going to read you different forms of political action that people can take, and I’d like you to tell me, for each one, whether you have actually Annual Responsible person: Antoaneta Skartova 2000 18.75 (+/- 3%) T: 16- 20% A: 32.7% gender male: 37% female: 29% age 18-24: 31% 25-39: 36% 40-54: 34% 55+: 29% education primary or less: 25% secondary: 33% higher: 39% ethnicity Mac: 31% Alb: 39% other: 29% employment employed T: 25- 30% (The SO 2 team expects participa -tion to stabilize after civil strife of 2001.) T: 25- 30% T: 30-35% Development Associates, Inc. Evaluation of the Local Government C-5 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 done any of these things.” Thirteen possible actions include, for example, voting in parliament ary elections, attending protest rallies, and attending political meetings. : 38% unemploye d: 30% inactive: 28% place urban: 32% rural: 34% BSC Estek data tables, Q8) 2.1.b. Continuing initiatives at the local level Percent of citizen community groups that have already organized and implemented activities through CSHI or ISC and are Percent Count is non￾cumulative: A new count is conducted each year. CSHI and ISC Project Monitoring and Evaluation Reports Annual Responsible persons: Melita Cokreva and Sladjana Srbinovska 2000 0% T: 38% (31 of 82) A: 59% (48 of 82) CSHI: A: 38 of 71 T: 38% (50 of 131) CSHI: T: 40 of 120 T: 38% (81 of 211) CSHI: T: 70 of 200 Development Associates, Inc. Evaluation of the Local Government C-6 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 now continuing with the participatory planning and implementation process. Groups are counted if they (a) are seeking assistance from other donors, or (b) have organized local self-help contributions for another (follow-on) community activity. ISC: A: 10 of 11 ISC: T: 10 of 11 ISC: T: 11 of 11 2.1.c. Women’s participation in community￾level decision making Percentage of CSHI or ISC projects that are led by women. Number of projects that are led by one or more women (as identified in project documenta￾ion) divided CSHI and ISC M & E records Annual Responsible persons: Melita Cokreva and Sladjana Srbinovska 2001 A: 37% (72 of 196) CSHI: 15 of 71 ISC: 57 of 125 (see base￾line) T: 40% A: T: 44% A: T: 48% A: Development Associates, Inc. Evaluation of the Local Government C-7 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 by the total number of grant projects awarded in a year. (i.e., a woman is the head of the community organization or the project itself) IR 2.1.1. CITIZENS ATTITUDES TOWARD DEMOCRATIC PROCESSES/PRACTICES IMPROVED 2.1.1.a Public trust in NGOs Percentage of survey respondents who respond “definitely yes” or “rather yes” to “Do you trust the following institutions?” Probability survey of Macedonian citizens Disaggregate d by gender, age, ethnicity and educational attainment USAID Survey of Citizen Attitudes 2001: Q12, f-8 Annual Responsible person: Antoaneta Skartova and Melita Cokrevska 2000 54% T: 54% A: 48.8% gender male: 50% female: 47.5% age 18-24: 47% 25-39: 48% 40-54: 48% 55+: 52% education primary or T: 54% T: 56% T: 58% Development Associates, Inc. Evaluation of the Local Government C-8 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 less: 40% secondary:4 7% higher: 57% ethnicity Mac: 51% Alb: 39% other: 48% employment employed: 54% unemployed : 43% inactive: 50% place urban: 48% rural: 51% BSC Estek data tables, Q12 f-8) IR 2.1.2. IMPROVED OPPORTUNITIES FOR CITIZEN PARTICIPATION 2.1.2.a. Public participation Percentage of survey respondents Percent of interviewees USAID Survey of Citizen Annual Responsible 2000 17% (+/- T: 15-19% A: 19.6% 16- 20% 21-25% Over 25% Development Associates, Inc. Evaluation of the Local Government C-9 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 in civil society organiza￾tions that have joined a group or organization pursuing their interest. . Disaggrega￾tion by gender, age, ethnicity, educational attainment Attitudes 2000 Brima report: pp. 25-26; 2001 survey: Q8, d-8: “I’m going to read you different forms of political action that people can take, and I’d like you to tell me, for each one, whether you have actually done any of these things.” Thirteen possible person: Antoaneta Skartova and Melita Cokrevska 3%) gender male: 23% female: 16% age 18-24: 21% 25-39: 23% 40-54: 24% 55+: 11% education primary or less: 13% secondary:21 % higher: 25% ethnicity Mac: 17% Alb: 28% other: 16% employment employed: 28% unemployed: 19% inactive: 15% place urban: 19% rural: 22% Development Associates, Inc. Evaluation of the Local Government C-10 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 actions include, for example, voting in parliament ary elections, attending protest rallies, and attending political meetings. This indicator uses the percent responding “yes” for “joining a group or organiza￾tion pursuing your own interest.” BSC Estek data tables, Q8 d-8) Development Associates, Inc. Evaluation of the Local Government C-11 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 IR 2.1.3. IMPROVED CITIZEN ACCESS TO OBJECTIVE, UNBIASED INFORMATION 2.1.3.a. Media Sustainabili￾ty Index Summary score measuring the overall state of media development in a country. This indicator is IREX’s comprehensive score for Macedonia, which is an average of five scores (one for each of the attributes described in the next column. Scores may range from 0 (unsustainable, anti-free press) to 4 (sustainable media). Average Disaggre￾gated by five attributes (free speech, professional journalism, plurality of news sources, business management, supporting institutions) Focus group data collected and reported by IREX. Documen￾tary source: Media Sustainabililt y Index 2001. Summary score is on p. 5; Macedonia analysis is on pp. 139- 151; methodology is described on pp. 17- 19. Annual Responsible person: Melita Cokrevska 2001 1.73 (Benchma rk compariso ns for refe￾rence: Croatia = 2.44; Monte￾negro = 1.58). A: 1.73 Attri-bute scores: Free speech = 1.72 Profession al journalism = 1.89 Plurality of news sources = 2.17 Business manage￾ment = 1.33 Supporting institutions = 1.55 T: 1.45 (Score is expect ed to decline due to election -year politiciz aion of media and potenti al re￾ignition of violent conflict in spring of 2002) A: T: 1.80 A: T: 2.30 A: Development Associates, Inc. Evaluation of the Local Government C-12 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 IR 2.2. ADHERENCE TO THE RULE OF LAW ENHANCED (INDICATORS TO BE REVIEWED AND UPDATED AS NEEDED WHEN LEGAL REFORM SUPPORT ACTIVITY BEGINS) 2.2.a.Public perception of equal treatment under court procedures Percent of interviewees answering “Yes, they are always equally treated” or “They are often equally treated” in response to item described under “Unit of measure￾ment/disaggre￾gation.” Percent of interviewees Disaggrega￾tion by gender, age, ethnicity, educational attainment USAID Survey of Citizen Attitudes, Q19: “Do you think the people are equally treated under the court procedures?” Annual Responsible person: Antoaneta Skartova 2001 38% A: 38% gender male: 37% female: 39% age 18-24: 33% 25-39: 36% 40-54: 37% 55+: 44% education primary or less: 35% secondary:35% higher: 39% ethnicity Mac: 44% Alb: 14% other: 38% employment employed: 45% unemployed: 29% inactive: 29% T: 2.538% (No USAID interven -tions in this area are expecte d until Septem￾ber 2002) T: 41% T: Over 3.045% Development Associates, Inc. Evaluation of the Local Government C-13 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 place urban: 38% rural: 38% BSC Estek data tables, Q18) IR 2.2.1. MORE EFFECTIVE JUDICIARY 2.2.1.a. Time required to decide a civil case Average number of months across all cases in five target courts Cases included are all cases formally assigned for a hearing or trial Number of months This indicator will be better defined next year. TBD, under new legal reform support project TBD with implementing partner; likely annual. Responsible person: Antoaneta Skartova 2002 36 (esti￾mate) -- T: A: T: TBD T: 18 IR 2.2.2. MORE EFFECTIVE LEGAL PERSONNEL 2.2.2.a Percent of legal personnel trained Percent of judges, advocates and court staff who participated in continuing legal education Percent (Number of participants divided by total judges, advocates For number of participants : ABA/CEELI and legal reform Annual Responsible person: Antoaneta 2002 TBD with legal reform support imple￾menting partner T: A: T: TBD T: TBD Development Associates, Inc. Evaluation of the Local Government C-14 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 courses funded by USAID. Courses to be sponsored by the Bar Association or the Center for Continuing Education. and court staff) This indicator will be better defined next year. support partner For total legal personnel: Macedonia n Bar Associa￾ion, via ABA/CEELI Skartova and ABA/- CEELI IR 2.3. MORE EFFECTIVE, RESPONSIVE AND ACCOUNTABLE LOCAL GOVERNMENT 2.3.a. Public trust in local government in target municipali￾ties Percent of interviewees who respond “a lot” or “somewhat” to the question “How much confidence do you have in your local governmrnt” Q2Index of several questions averaged Percent of interviewees Disaggrega￾tion by gender, age, ethnicity, educational attainment For tracking against performanc e targets: USAID Municipal level survey (under IR 2.3) N=3,300 in 2001 Annual Responsible person: Afrodita Salja 2001 TBD43% A: 43% gender male: 41% female: 44% age 18-24: 45% 25-39: 45% 40-54: 39% 55+: 43% education primary or less: 45% secondary:3 T: 44- 49% A: T: 50- 54% 7% to 10% increase over baselineT: 55-60% Development Associates, Inc. Evaluation of the Local Government C-15 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 across 17 target cities. [List of questions to be added as an endnote] 17 target municipaliti es For comparison of attitudes in USAID – assisted sites with those nation￾wide: USAID Survey of Citizen Attitudes (Q12, f-2) 2000 41.7 (+/- 3%) 8% higher: 51% ethnicity Mac: 37% Alb: 62% other: 36% employment employed: 45% unemployed: 38% inactive: 46% place urban: 41% rural: 46% (Q2 on DAI survey) National level (or compariso n) 40.2% BSC Estek data tables, A: A: Development Associates, Inc. Evaluation of the Local Government C-16 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 Q12 f-2) IR 2.3.1. LOCAL GOVERNMENTS HAVE INCREASED RESPONSIBILITY AND FINANCIAL RESOURCES 2.3.1.a. Local government spending as a percent of all general Government expenditure s Local government spending divided by all general government spending including spending through special funds, e.g., environment Percent Municipal and GOM budget documents. DAI will calculate and deliver data to USAID. Annual Responsible person: Afrodita Salja 2001 4.5% A: 4.5% T: 4.5% T: 4.5% T: 9.0%Double the baseline (Passage of finance law is expected to lead to this increase.) IR 2.3.2. EFFECTIVE RELATIONSHIP BETWEEN CITIZENS AND LOCAL GOVERNMENT EXISTS 2.3.2.a. Public participation in local government decision making Percentage of survey respondents that have attempted to make local self government solve a problem Percent of interviewees Disaggrega￾tion by gender, age, ethnicity, educational attainment USAID Survey of Citizen Attitudes (Q 24) Annual Responsible person: Afrodita Salja 2000 19.8% (+/- 3%) T: 18- 20% A: 21.6% gender male: 27% female: 16% age T: 20- 22% T: 22- 25% T: Over 25% Development Associates, Inc. Evaluation of the Local Government C-17 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 18-24: 15% 25-39: 23% 40-54: 25% 55+: 20% education primary or less: 11% secondary: 23% higher: 29% ethnicity Mac: 19% Alb: 35% other: 17% employmen t employed : 29% unemploye d: 17% inactive: 13% place urban: 21% rural: 24% IR 2.3.3. MUNICIPAL ASSOCIATIONS SATISFACTORILY SERVE THE INTERESTS OF THEIR MEMBERS 2.3.3.a. Percent of Percentage of respondents to Percent of survey Association surveys Annual; data cover the 2000 ZELS: 32% ZELS: T: 35% ZELS: T: 55% ZELS: T: 60% ZELS: T: 65% Development Associates, Inc. Evaluation of the Local Government C-18 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 municipal association members indicating they are satisfied with their association annual association surveys. respondents with published baselines and results Parallel questions used and reported by both ZELS and AFO calendar year (January￾December) Responsible person: Afrodita Salja AFO: 84% (Note that AFO mem￾bership is volun￾tary, while ZELS members hip is not) A: 88% AFO: T: 86% A: 100% A: AFO: T: 88% A: A: AFO: T: 90% A: A: AFO: T: 90% A: ZELS: Improvement in the 10- 15% range AFO: Improvement in the 5% range IR 2.3.4. LOCAL GOVERNMENTS IMPROVE THE MANAGEMENT OF MUNICIPAL SERVICES 2.3.4.a. Bill collection rate for basic services in target public enterprises (e.g., water, garbage, etc.) Value of bills collected as a percent of the total charged, averaged among target public Percent. The nine target public enterprises use the same method for recording revenue Public enterprises’ financial data, collected and provided to USAID by DAI Monthly or quarterly, aggregated to annual basis, summarized for the calendar year (January to December). 2001 53% baseline T: 58% T: 63% T: 69% Development Associates, Inc. Evaluation of the Local Government C-19 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 enterprises. Data may be disaggregate d into: --enterprises having received USAID assistance for more than one year, and --(b) enterprises that have received USAID assistance for one year or less. Responsible person: Afrodita Salja IR 2.4. INCREASED CONFIDENCE IN GOVERNMENT INSTITUTIONS AND POLITICAL PROCESSES 2.4.a. Trust in Parliament Percentage of survey respondents that trust Parliament Individuals chosen using survey sampling procedures Disaggrega￾tion by USAID Survey of Citizen Attitudes Q12 f-3 Annual Responsible person: Antoaneta Skartova 2000 36.7% T: 34- 38% A: 27.7% gender male: 30% female: 26% T: 35- 38% T: 37- 39% T: Over 40% Development Associates, Inc. Evaluation of the Local Government C-20 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 gender, age, ethnicity, educational attainment age 18-24: 21% 25-39: 26% 40-54: 32% 55+: 30% education primary or less: 22% secondary: 27% higher: 32% ethnicity Mac: 31% Alb: 15% other: 18% employmen t employed : 34% unemploye d: 21% inactive: 23% place urban: 27% rural: 30% IR 2.4.1. PARLIAMENT FUNCTIONS IN A MORE EFFECTIVE AND VISIBLY REPRESENTATIVE MANNER 2.4.1.a. MPs must meet Percent of 85 NDI Annual 2000 15% T: 20% T: 5% T: 15% T: 25% Development Associates, Inc. Evaluation of the Local Government C-21 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 Percentage of single member district MPs that engage in regular outreach to constituents at least 3 of the 5 benchmarks for this scorecard. In order to be counted during a reporting year an MP must do three of the following: (1) produce at least one issue of a legislative newsletter and distribute it widely among constituents, (2) conduct at least one open, publicly announced meeting in the constituency that features a question and answer session between the MP and single￾member￾district members This indicator will be reviewed again next year, to correspond with the new Election Law Responsible person: Antoaneta Skartova A: NDI still hasn’t provide d the actual data. Development Associates, Inc. Evaluation of the Local Government C-22 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 constituents, (3) conduct constituent casework; (4) hold office hours on at lease a quarterly basis, (5) meet or consult with NGOs/CSOs or other community or neighborhood based groups on at least a quarterly basis or hold consultative meetings with groups of constituents (including NGOs) at least twice. This definition Development Associates, Inc. Evaluation of the Local Government C-23 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 will be revised if and when the constitutional mandate for single-member MPs changes. IR 2.4.2. ELECTIONS ARE MORE FREE AND COMPETITIVE 2.4.2.a. Free and Fair Elections Held This is a simple “pass/fail” test conceptually that in practice is applied by observers who reach a consensus and normally record that consensus in a report. Newspaper reports supplement such reports Elections held every other year USAID to make final judgment for reporting purposes on whether an election was free and fair or not. OSCE observers and newspaper articles When elections occur Responsible person: Antoaneta Skartova 1998 and 2000 Not free and fair; flawed pro-cess; out￾breaks of vio-lence [No elec￾tion this year.] T: Signifi￾cant overall improve￾ment over 1998 and 2000 elections [USAID election assis￾tance ends in 2003.] IR 2.4.3. POLITICAL PARTIES TRANSFORMED TO A PLATFORM AND PERFORMANCE BASIS Platform communica￾Number and percent of Number and percent of NDI and IRI activity Annual 2001 0 No electio (parlia- mentary T; x, x% (presidential elections) Development Associates, Inc. Evaluation of the Local Government C-24 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 tion by parties This indicator will be reviewed again next year. parties that (a) publish and distribute their platforms to members and other citizens during an election campaign, and (b) communicate their policy positions to voters. Criterion (a) is relevant only during election campaigns. parliamentary parties in existence during the reporting period that meet the criteria in the definition. Note: We restrict the coverage to parties present in parliament, since there are numerous smaller parties outside parliament; these would be difficult to collect data for. reports, based on ongoing discussions with party members and review of platform documents, news releases, party brochures, etc. NDI, IRI and selected members of the USAID SO 2.0 Team meet each fall to derive the total number Responsible person: Antoaneta Skartova ns this year elections) T: x, x% A: A: T: x, x% A: Development Associates, Inc. Evaluation of the Local Government C-25 May 2002 Reform Project in Macedonia PERFORMANCE MEASUREMENT PLAN (WITH TARGETS AND ACTUALS) FOR SO 2.0 USAID/Macedonia (Last updated on February 13, 2002) Baseline Targets and Actuals Perfor￾mance Indicator Definition Unit of Measure￾ment/ Disaggrega￾tion Data Source/ Data Collection Method Timing and Frequency of Data Collection/ Person Responsible Year Value 2001 2002 2003 2004 and percent, based on data from IRI and NDI. IR 2.4.4. COMMUNICATION BETWEEN ELEMENTS OF GOVERNMENT AND CITIZENS IMPROVED GOM’s communica￾tion effort This indicator will be better defined next year. Number of press releases issued by GOM during one year Number IRI Annual Responsible person: Antoaneta Skartova 2001 Evaluation of the Local Government D-1 May 2002 Reform Project in Macedonia APPENDIX D LIST OF PEOPLE INTERVIEWED USAID offices Skopje and Washington # Name Position Date 1. Diana Ponasik General Development Officer 27.02.2002 2. Afrodita Salja Project Management Specialist 27.02.2002 3. Ivica Vasev Program Assistant 27.02.2002 4 Bonnie Walter Senior Advisor, Washington 24.02.2002 5 Stacia George Macedonia Desk Officer 24/02.2002 6 Michael Keshishian Democracy and Governance Advisor, E&E/EEST/Washington 27.02.2002 LGRP in Skopje (DAI team) 1. Larry Birch Chief of Party 27.02.2002 2. Leah April Association Development Advisor 27.02.2002 3. Bill Althaus Municipal Management Advisor 27.02.2002 4. Bob Refuse Policy Reform Advisor 27.02.2002 DAI team (ZELS) 1. Nada Vuchinich 28.02.2002 DAI team (Policy Reform) 1. Branko Protic 28.02.2002 2. Liljana Ristevska 28.02.2002 LED workshop 1. Snezana Salamovska 1.03.2002 Andrijana Aleksovska 1.03.2002 Arberor Basha 1.03.2002 Hajredin Osmani 1.03.2002 Evantija Osmanovska 1.03.2002 Ljubomir Janev 1.03.2002 Ljupco Petkovski 1.03.2002 Zoran Manasiev 1.03.2002 Dusko Arsovski 1.03.2002 Taki Petreski 1.03.2002 Goran Angelov 1.03.2002 Vladislav Zupan 1.03.2002 Valentina Mitreva 1.03.2002 Nebi Jusufi DAI team (CICs) Vesna Atanasova 1.03.2002 DAI team (LED) Bekim Ymeri 1.03.2002 Bardyl Marku 1.03.2002 Vesna Vasilevska 1.03.2002 Development Associates, Inc. Evaluation of the Local Government D-2 May 2002 Reform Project in Macedonia ZELS Administration Goran Angelov ZELS president 21.03.2002 Dusica Perisic Executive Director 4.03.2002 Natasa Cvetkovska 4.03.2002 Ministry of Local Self-Government Faik Arslani Minister 4.03.2002 Elena Petkanovska 4.03.2002 Mihaela Stojakovska 4.03.2002 AFO Maksim Acevski President 4.03.2002 Evgenija Gramatikova Board Member 4.03.2002 M. Loga Board Member 4.03.2002 Euroconsultant (EU) Vasilis Vasilatos 5.03.2002 Paris Kokorotsikos 5.03.2002 Panayotis Karkatsoulis 5.03.2002 PRISMa Emmy Ramm Director 8.03.2002 GTZ Marina Naumovska￾Milevska Project Coordinator 11.03.2002 CSHI Michael Wallace Chief of Party 12.03.2002 OSCE Petra Andresen Economic and Development Advisor 12.03.2002 Andreas Rabb Advisor 12.03.2002 World Bank Evgenij Najdov Research Analyst 19.03.2002 UNDP Matilda Dimovska 19.03.2002 SEE University Denis Farington Secretary General 20.03.2002 European Commission Luisa Rizzo Program Officer 21.03.2002 Paivi Nikander European Agency for Reconstruction 21.03.2002 Ministry of Finance V. Savevska 22.03.2002 Trajko Spasovski M. Todevski 22.03.2002 Development Associates, Inc. Evaluation of the Local Government D-3 May 2002 Reform Project in Macedonia Visits to Municipalities Gostivar Xemail Redxepi Mayor 13.03.2002 Afrim Jakupi City Secretary 13.03.2002 Shpresa Architect and CAB 13.03.2002 H. Osmani LED officer 13.03.2002 Iljmiasan Dauti CIC 13.03.2002 Debar Imer Ologu Mayor in Skopje B. Kercishta Director - municipal enterprse in Skopje Veles Ace Kocevski Mayor 13.03.2002 Stevka Bogoevska Council 13.03.2002 Lili Velkova Budget officer 13.03.2002 Lj. Mancovska CIC 13.03.2002 M. Andonov AFO 13.03.2002 M. Markov AFO 13.03.2002 D. Damjanovski AFO 13.03.2002 R. Nikolov AFO 13.03.2002 Bitola Zlatko Vrskovski Mayor 13.03.2002 B. Kitanovski Secretary/CAB 13.03.2002 Snezana Salamovska LED 13.03.2002 S. Strkovski Council 13.03.2002 B. Stamatova AFO 13.03.2002 P. Josevska CIC 13.03.2002 Kocani Ljubomir Janev Mayor 15.03.2002 Z. Manasiev Advisor 15.03.2002 Georgi Hristov CE director 15.03.2002 Todor Kusevski CE employee 15.03.2002 Todor Pasovski Council 15.03.2002 Kavadarci G. Josifovski Advisor 15.03.2002 Jordan Milkov CE director 15.03.2002 Jovan Ristovski CE employee 15.03.2002 Todor Efremov CIC 15.03.2002 Strumica Kiril Janev Mayor 15.03.2002 Georgi Dimitrievski City Architect 15.03.2002 Julija Javazova Team member 15.03.2002 K. Salamanova LED/ Spokesperson of Strumica 15.03.2002 Development Associates, Inc. Evaluation of the Local Government D-4 May 2002 Reform Project in Macedonia Miravci Suncica Petkovska Mayor 15.03.2002 Gorgi Grebenarov Secretary 15.03.2002 Mite Basov NGO member (European Forum) 15.03.2002 Tetovo Murtezan Ismaili Mayor 20.03.2002 Small non-participating municipalities Vlasta Dimkovic Mayor of Staro Nagoricane 20.03.2002 Boris Gavrilov Mayor of Karbinci 20.03.2002 Nuzi Sahin Mayor of Centar Zupa 20.03.2002 Goce Velickovski Mayor of Rosoman 20.03.2002 Ilija Duklevski Mayor of Meseista 20.03.2002 Blazo Georgiev Mayor of Podares 20.03.2002 Evaluation of the Local Government E-1 May 2002 Reform Project in Macedonia APPENDIX E MUNICIPALITIES BY POPULATION Adapted from a list provided by GTZ. Municipality Population % of total Cumulative Cumulative as % of total TOTAL 1,933,385 100.00% 1 City of Skopje 537,210 27.79% 2 Kisela Voda 118,079 6.11% 118,079 6.11% 3 Kumanovo 89,626 4.64% 207,705 10.74% 4 Bitola 86,176 4.46% 293,881 15.20% 5 Centar 85,021 4.40% 378,902 19.60% 6 Prilep 71,124 3.68% 450,026 23.28% 7 Gazi Baba 67,664 3.50% 517,690 26.78% 8 Tetovo 65,565 3.39% 583,255 30.17% 9 Chair 63,375 3.28% 646,630 33.45% 10 Karposh 58,359 3.02% 704,989 36.46% 11 Veles 56,547 2.92% 761,536 39.39% 12 Ohrid - land 52,066 2.69% 813,602 42.08% 13 Shtip 46,791 2.42% 860,393 44.50% 14 Gostivar 43,567 2.25% 903,960 46.76% 15 Strumica 42,953 2.22% 946,913 48.98% 16 Gorche Petrov 37,961 1.96% 984,874 50.94% 17 Kavadarci 36,192 1.87% 1,021,066 52.81% 18 Struga 35,137 1.82% 1,056,203 54.63% 19 Kochani 31,764 1.64% 1,087,967 56.27% 20 Kichevo 27,443 1.42% 1,115,410 57.69% 21 Lipkovo 24,284 1.26% 1,139,694 58.95% 22 Radovish 23,049 1.19% 1,162,743 60.14% 23 Tearce 22,405 1.16% 1,185,148 61.30% 24 Zhelino 22,051 1.14% 1,207,199 62.44% 25 Kriva Palanka 20,695 1.07% 1,227,894 63.51% 26 Gevgelija 19,459 1.01% 1,247,353 64.52% 27 Negotino 18,955 0.98% 1,266,308 65.50% 28 Sveti Nikole 18,487 0.96% 1,284,795 66.45% 29 Debar 17,586 0.91% 1,302,381 67.36% 30 Resen - land 550 17,419 0.90% 1,319,800 68.26% 31 Delchevo 17,255 0.89% 1,337,055 69.16% Development Associates, Inc. Evaluation of the Local Government E-2 May 2002 Reform Project in Macedonia 32 Vinica 17,078 0.88% 1,354,133 70.04% 33 Negotino Poloshko 15,675 0.81% 1,369,808 70.85% 34 Brevenica 15,077 0.78% 1,384,885 71.63% 35 Studenichani 14,655 0.76% 1,399,540 72.39% 36 Ilinden 14,562 0.75% 1,414,102 73.14% 37 Shuto Orizari 14,301 0.74% 1,428,403 73.88% 38 Berovo 14,113 0.73% 1,442,516 74.61% 39 Saraj 13,734 0.71% 1,456,250 75.32% 40 Bogovinje 13,616 0.70% 1,469,866 76.03% 41 Novo Selo 12,917 0.67% 1,482,783 76.69% 42 Probishtip 12,884 0.67% 1,495,667 77.36% 43 Bosilovo 12,372 0.64% 1,508,039 78.00% 44 Kamenjane 12,243 0.63% 1,520,282 78.63% 45 Valandovo 12,049 0.62% 1,532,331 79.26% 46 Chegrane 11,746 0.61% 1,544,077 79.86% 47 Vasilevo 11,335 0.59% 1,555,412 80.45% 48 Kratovo 10,893 0.56% 1,566,305 81.01% 49 Dolneni 10,836 0.56% 1,577,141 81.57% 50 Rostushe 10,718 0.55% 1,587,859 82.13% 51 Zajas 10,014 0.52% 1,597,873 82.65% 52 Arachinovo 9,990 0.52% 1,607,863 83.16% 53 Kondovo 9,739 0.50% 1,617,602 83.67% 54 Krushevo 9,690 0.50% 1,627,292 84.17% 55 Sopishte 9,621 0.50% 1,636,913 84.67% 56 Labunishta 9,371 0.48% 1,646,284 85.15% 57 Oslomej 8,939 0.46% 1,655,223 85.61% 58 Bogdanci 8,886 0.46% 1,664,109 86.07% 59 Vrapchiste 8,794 0.45% 1,672,903 86.53% 60 Petrovec 8,073 0.42% 1,680,976 86.94% 61 Chucher-Sandevo 7,969 0.41% 1,688,945 87.36% 62 Makedonska Kamenica 7,854 0.41% 1,696,799 87.76% 63 Djepchishte 7,678 0.40% 1,704,477 88.16% 64 Demir Hisar 7,610 0.39% 1,712,087 88.55% 65 Dolna Banjca 7,503 0.39% 1,719,590 88.94% 66 Jegunovce 7,127 0.37% 1,726,717 89.31% 67 Murtino 7,039 0.36% 1,733,756 89.67% 68 Shipkovica 7,002 0.36% 1,740,758 90.04% 69 Veleshta 6,939 0.36% 1,747,697 90.40% 70 Krivogashtani 6,598 0.34% 1,754,295 90.74% 71 Delogozhda 6,555 0.34% 1,760,850 91.08% Development Associates, Inc. Evaluation of the Local Government E-3 May 2002 Reform Project in Macedonia 72 Centar Zupa 6,196 0.32% 1,767,046 91.40% 73 Makedonski Brod 6,077 0.31% 1,773,123 91.71% 74 Bistrica 5,737 0.30% 1,778,860 92.01% 75 Vrutok 5,705 0.30% 1,784,565 92.30% 76 Pehchevo 5,624 0.29% 1,790,189 92.59% 77 Obleshevo 5,401 0.28% 1,795,590 92.87% 78 Mogila 5,368 0.28% 1,800,958 93.15% 79 Staro Nagorichane 5,178 0.27% 1,806,136 93.42% 80 Karbinci 4,928 0.25% 1,811,064 93.67% 81 Demir Kapija 4,791 0.25% 1,815,855 93.92% 82 Orizari 4,685 0.24% 1,820,540 94.16% 83 Kuklish 4,650 0.24% 1,825,190 94.40% 84 Rosoman 4,400 0.23% 1,829,590 94.63% 85 Plasnica 4,399 0.23% 1,833,989 94.86% 86 Rankovce 4,376 0.23% 1,838,365 95.09% 87 Srbinovo 4,130 0.21% 1,842,495 95.30% 88 Zelenikovo 4,107 0.21% 1,846,602 95.51% 89 Drugovo 3,793 0.20% 1,850,395 95.71% 90 Gradsko 3,789 0.20% 1,854,184 95.90% 91 Konche 3,690 0.19% 1,857,874 96.09% 92 Podaresh 3,655 0.19% 1,861,529 96.28% 93 Star Dojran 3,649 0.19% 1,865,178 96.47% 94 Zletovo 3,608 0.19% 1,868,786 96.66% 95 Belchishta 3,393 0.18% 1,872,179 96.83% 96 Topolchani 3,377 0.17% 1,875,556 97.01% 97 Zrnovci 3,311 0.17% 1,878,867 97.18% 98 Vratnica 3,295 0.17% 1,882,162 97.35% 99 Mesheishta 3,025 0.16% 1,885,187 97.51% 100 Sopotnica 2,991 0.15% 1,888,178 97.66% 101 Lozovo 2,904 0.15% 1,891,082 97.81% 102 Chashka 2,856 0.15% 1,893,938 97.96% 103 Kukurechani 2,854 0.15% 1,896,792 98.11% 104 Novaci 2,778 0.14% 1,899,570 98.25% 105 Miravci 2,670 0.14% 1,902,240 98.39% 106 Vevchani 2,447 0.13% 1,904,687 98.52% 107 Cheshinovo 2,438 0.13% 1,907,125 98.64% 108 Dobrushevo 2,386 0.12% 1,909,511 98.77% 109 Zhitoshe 2,123 0.11% 1,911,634 98.87% 110 Klechevce 2,069 0.11% 1,913,703 98.98% 111 Samokov 2,046 0.11% 1,915,749 99.09% Development Associates, Inc. Evaluation of the Local Government E-4 May 2002 Reform Project in Macedonia 112 Blatec 1,994 0.10% 1,917,743 99.19% 113 Lukovo 1,929 0.10% 1,919,672 99.29% 114 Capari 1,811 0.09% 1,921,483 99.38% 115 Kosel 1,773 0.09% 1,923,256 99.48% 116 Vraneshtica 1,671 0.09% 1,924,927 99.56% 117 Orashac 1,628 0.08% 1,926,555 99.65% 118 Mavrovi Anovi 1,421 0.07% 1,927,976 99.72% 119 Bogomila 1,323 0.07% 1,929,299 99.79% 120 Izvor 1,250 0.06% 1,930,549 99.85% 121 Bach 953 0.05% 1,931,502 99.90% 122 Vitolishte 889 0.05% 1,932,391 99.95% 123 Konopishte 543 0.03% 1,932,934 99.98% 124 Staravina 451 0.02% 1,933,385 100.00% Evaluation of the Local Government F-1 May 2002 Reform Project in Macedonia APPENDIX F POLITICAL CONTEXT – USAID STRATEGIC PLAN AND ESI REPORT EXCERPTS The following text, too long to include in the main report, places decentralization in the context of Macedonia’s ethnic tensions. A recurring theme throughout the Evaluation Team’s assessment has been the importance of the local government sector in Macedonia to nurturing trust in governmental institutions in the country. USAID’s Strategic Plan 2002-2004 s in its description of IR 2.4 Increased confidence in Government Institutions and the political process characterizes the state as closer to a semi competitive partyist oligarchy rather than a true participatory democracy. A state in which a high degree of competition is evident only between the two leading ethnic political parties, competition which is based on the personalities of their leaders and the ability of those leaders to command loyalty, rather than on issues2 . It is in this political context that improvements to the effectiveness of municipal government offer a means, in the words of the Strategic Plan, “to start to shift the balance of power away from Government. Rebalancing power will enhance the legitimacy of democratic government.” 3 In a similar vein, the European Stability Initiative in a Discussion Paper dated February 20, 2002 titled “The Other Macedonian Conflict” 4 supports the thrust of the USAID strategy. This paper is focused on the overall development context for Macedonia and places local government in a critical role. The paper states that “Strengthening the capacity of local government in key areas such as urban planning, local infrastructure development, education, social welfare and health will require long term efforts on the part of the Macedonian government. Strengthening local government, integrating Albanians into the police forces, and public administration … Are all steps with significant budgetary implications. These are recurrent commitments which will require a serious view of how public revenues are shared among different 2 USAID/MACEDONIA Amended Strategic Plan FY2001-2004, July 2001 Page 51 3 Ibid. Page 49 4 European Stability Initiative The Other Macedonian Conflict – ESI Discussion Paper Page 13 (Supported by the German Foreign Ministry) Feb-20-02 Development Associates, Inc. Evaluation of the Local Government F-2 May 2002 Reform Project in Macedonia levels of government. If the Framework Agreement stalls for want of attention to these issues, it will likely set back the overall peace process…. The peace process depends upon an increase in public confidence in the capacity of the Macedonian state to address the widespread economic insecurity. This requires addressing the present gap between the state and the majority of its citizens, In a process in which local government must play a critical role. The alternative is further erosion of faith in public institutions and the consolidation of parallel institutions at the local level formed along ethnic lines. This would deepen ethnic distrust…” These documents reinforce what appears to be the paramount role well developed and well led local governments can play in developing Macedonian Evaluation of the Local Government G-1 May 2002 Reform Project in Macedonia APPENDIX G PROGRAM AND POLITICAL HISTORY TIMELINE Prepared by LGRP at the request of the Evaluation Team. Local Government Reform Project Project Chronology LGRP Milestone Timeline Current Events Pre October 1999 1995 – Original Law on Local Government enacted. 1996 - Current Law on the City of Skopje enacted. 1996 – Current Law on Territorial Division enacted. 1997 – Macedonia ratifies the European Charter on Local Self-Government. December 1998 – Ministry of Local Self￾Government (MoLG) established. Nasufi appointed Minister. LGRP starts operation. October 1999 MM & CP assessment of needs. November 1999 Government adopts Strategy for Local Govern- ment Reform. MM & CP assessment of needs. December 1999 MM & CP identification of pilot projects for pilot cities. January 2000 MM & CP identification of pilot projects for pilot cities. February 2000 March 2000 MoLG formally establishes three interagency working groups to prepare legislative elements of the Strategy for Reform. April 2000 Saiti appointed Minister of Local Government. LGRP advisors begin participating in the meetings of the MoLG’s working groups. May 2000 Working groups on Law on Local Government and Local Finance begin meetings. The first Chief of Party left the Project June 2000 Before end of the month: Working Group on Law on Local Government is de facto disestablished. Antonio Iskandar appointed Interim Chief of Party. Strategic Planning Workshop ZELS and AFO. ZELS and AFO sign MOU for Cooperation. July 2000 Development Associates, Inc. Evaluation of the Local Government G-2 May 2002 Reform Project in Macedonia ZELS and AFO Membership Survey Results Published August 2000 LGRP-Urban Institute Workshop on Fiscal Decentralization. The contract was modified to place more emphasis on Policy Reform September 2000 Larry Birch takes over as Chief of Party. Seminar organized by AFO5 budgeting techniques parts covered by MM October 2000 Local elections. LGRP plans and conducts FDI Forum jointly with UNDP and MoLG. LGRP workshop on local accounting, financial reporting, and auditing. November 2000 Fiscal Decentralization Initiative Forum for Macedonia. Work Begun to Reconstitute the Standing Committee for ZELS. December 2000 1st new ZELS Standing Committee Organizational Meeting Held. January 2001 Financial Management (Cost allocation) presentation with CE. AFO Strategic Plan, 2001 Budget and Workplan Approved by Members. By –Laws Amended. Budget Hearing in Kumanovo. February 2001 Terrorist incidents start at Tanuševci. Start up of Financial Management program in (Debar, Tetovo, K Palanka, Valandovo, Krushevo, Kumanovo). LGRP workshop on Law of the City of Skopje. LGRP workshop on local government credit market access. CIC Opens in Gostivar. March 2001 Terrorist incidents begin in Tetovo and spread to other areas. Leak detection equipment delivered to Kavadarci. ZELS 12 Regional Public Hearings on LLSG begins. Weekend meeting of Working Group on Local Finance Law approves detailed technical outline of the law. ZELS Action Agenda 2001 with National Association of Bulgaria held. CIC Opens in Shtip April 2001 Terrorist incidents increase in intensity. 5 Kumanovo used LGRP model for the Budget hearings in February 2001. Development Associates, Inc. Evaluation of the Local Government G-3 May 2002 Reform Project in Macedonia Meetings of the Working Group on Local Finance Law are suspended because of the terrorist crisis. Executive Director for ZELS hired. LGRP discontinues direct technical assistance to AFO. ZELS sub committee drafts language for new LLSG. ZELS adopts Public Statement on LLSG in Plenary Session. May 2001 Four Party Government of National Unity formed Faik Arslani becomes Minister of Local Government. LGRP adopts Confidence Building Initiative as a short-term work plan. Meetings of the Working Group on Local Finance Law are suspended because of the terrorist crisis. CIC Opens in Centar – Skopje. Two one day workshops on Media Training. CIC Staff Training on Customer Service. June 2001 First workshop on amendments to the Law on Local Government presented by experts engaged by the Council of Europe.6 All short-term USAID advisors are ordered to be evacuated from Macedonia. CIC Opens in Kavadarci. CIC Opens in Kumanovo. Presentation of distance accessing Municipality/CIC to Municipality/CIC via TCP/IP protocol7 First American staff evacuated. Manual for Internal Human resource development distributed to city secretaries of the core municipalities.8 July 2001 All non-essential American personnel are ordered to be evacuated from Macedonia. Remaining American staff evacuated By request of Kocani CE, CE was included on Financial Management-Cost Allocation program. August 2001 13 August – Ohrid Framework Agreement signed. MM & AFO made adaptation for the Municipal needs the Financial Management software module. CP/MM - Workshop on project proposal for CIC employees. September 2001 Second workshop on amendments to the Law on Local Government presented by experts engaged by the Council of Europe. NATO’S “Operation Essential Harvest” (collecting the NLA’s weapons) is completed. DAI is allowed to return one American from evacuation. 6 LGRP advisors assisted in the planning of the Council of Europe workshops and participated actively in them. 7 MM/CP organized workshop in Cooperation with UNDP/IT centers program and SOROS. Workshop was organized through ZELS. Ministry of Local Government participated in the workshop. About 50 different municipalities participated besides the fighting in north west parts of Macedonia 8 Seminar for the Modern personnel techniques for Macedonian municipalities was canceled due to evacuation of American staff Development Associates, Inc. Evaluation of the Local Government G-4 May 2002 Reform Project in Macedonia American staff return from evacuation. MM & AFO workshop in Veles UNDP-IT center9 CIC Opens in Veles. Two one day workshops on Media Training. ZELS/AFO Study Tour to US. October 2001 First workshop on the Law on Local Finance presented by experts engaged by the Council of Europe. 23 October – mandatory evacuation is lifted for all Americans. LGPR returns to normal work plan. LGRP advisors participate in weekend workshop in Ohrid on Law on the City of Skopje. LGRP short-term mission to develop a proposal for a municipal bond bank for Macedonia. LGRP short-term mission to complete drafting of the provisions of the local finance law relating to municipal budgeting, accounting, financial reporting, and auditing. CIC Opens in Bitola. November 2001 ZELS Expo, the first municipal fair in Macedonia, held at the city fairgrounds. LGRP advisor participates in meetings of IMF mission to discuss reform, the local finance law, and the donors’ conference with the ministers of Finance and Local Government. LGRP advisors participate with officials of the U.S. embassy and the European Commission on negotiations with the political parties and Government ministries on the amended Law on Local Government. December 2001 Ambassador Einik leaves. Second workshop on the Law on Local Finance presented by experts engaged by the Council of Europe. Minister of Finance announces that he is taking over responsibility for the local finance law from the MoLG. Financial Management(Cost Allocation) Software module installed in Strumica, Veles, Krushevo, Saraj. CIC Opens in Tetovo. Citizen Advisory Board in Public Enterprise constituted in Debar. CIC Network Meeting. ZELS/MOLG sponsor party for adoption of LLSG. ZELS Board/Executive Staff Retreat. January 2002 24 January – Amended Law on Local Self￾Government is enacted. 9 Simulation of Municipal Budget divided in 3 units (Mayor and Mayor Cabinet unit, city council, administration and managing bodies) and presentation of reports by defined cost centers Development Associates, Inc. Evaluation of the Local Government G-5 May 2002 Reform Project in Macedonia CICs opened in Debar and Karposh – Skopje. Public Hearings on Municipal Budgets hold in Saraj, Kavadarci, Kocani, Debar, Prilep, Bitola and Suto Orizari. Citizen Advisory Board in Public Enterprise constituted in Kavadarci. Two one day workshops on Media Training. Donor training coordination initialization meeting at LGRP. Local Economic Development seminar for 21 cities and city profile template design for 12 pilot cities. VPSN (Virtual parts and services network) presented to CE , CE association , GTZ (German program for commercialization of CE). ZELS begins Public Hearings on Finance Law and establishes joint sub-committee with AFO. February 2002 European Commission launches its Local Government Development Programme. Government officially approves assumption by the Ministry of Finance of responsibility for the local finance law. The political parties appear to reach agreement that no Parliamentary election will be scheduled before the fall and that the Ohrid Agreement understanding that the local finance law will be enacted before the Parliament adjourns will be disregarded. Larry Butler appointed as US Ambassador Designate. CIC Opened in Kriva Palanka. Public Hearings on Municipal Budgets hold in Kumanovo, Karposh and Gostivar. ZELS Policy Making workshop occurs. Financial Systems Review for ZELS to begin. March 2002 12-26 March - IMF mission on the local finance law. April 2002 World Bank mission on education reform. May 2002 June 2002 July 2002 August 2002 September 2002 Currently most-discussed period for Parliamentary election. October 2002 Tentative date for ZELS Expo II. November 2002 Mandate of the current Parliament expires. December 2002 Evaluation of the Local Government H-1 May 2002 Reform Project in Macedonia APPENDIX H QUESTIONS USED TO GUIDE INTERVIEWS Draft Discussion Guide for Municipal Interviews to Be Conducted in Sample Cities During the Week of March 10, 2002 The following sample questions are illustrative of the leadoff questions to be asked of municipal officials in the survey sample. Questions of this type are designed to open up a detailed discussion of the LGRP’s progress and success in each sampled municipality, and to elicit a description from local officials of the possibilities for LGRP expansion and refinement to meet their perceived needs. Depending on conditions unique to each sampled municipality, not all questions below will be asked at each interview, and follow-up questions will be made ad hoc. The effect of this approach will be the creation of a narrative account by local officials of the LGRP’s appropriateness, quality, impact, and potential for application in refined form over the long term. This is a draft of the survey administrators’ instrument, and therefore includes editorial comments for their use. Municipal Management General questions: 1. (Icebreaker) - How have you been involved in the LGRP? (we will know generally, so we could introduce a question like this by saying we know you have been involved in X,Y, and Z with LGRP, then have him/her describe the most meaningful experiences.) 2. What LGRP municipal management improvement programs have you or your staff attended, or which training and technical assistance projects have taken place in your city? 3. How would you rate each of them (High, Moderate, Low) in terms of being helpful to you and your staff in managing your city? 4. How were these programs originated (if you know)? Are you aware of a municipal training or technical assistance needs assessment that was done near the beginning of the LGRP? Please explain how your local needs were determined and if those needs are similar to those of municipalities like yours. Do you feel strongly that the training and technical assistance provided by LGRP here are applicable in all or most municipalities in Macedonia? 5. Do you feel that the LGRP programs which you participated in were relevant to your city’s needs, and were they at the correct level of complexity and technical content? Explain. Development Associates, Inc. Evaluation of the Local Government H-2 May 2002 Reform Project in Macedonia 6. Do your citizens know about the programs LGRP has held here, and if so, are there examples of how citizens have changed their opinions on how local government operates? 7. Does the City Council have a good knowledge of theses programs, and does it see any benefit to the Council in terms of their making policies? How has the LGRP been explained to Councilors, and how have they participated in it, if they have? Follow-up questions: 1. Describe, for each of the training and technical assistance projects, how the information provided to you was used, if it was relevant and of high quality, in improving your municipal operations and policy making functions. What does your municipal administration do more effectively now as a result of the LGRP programs? Explain. 2. As a result of any of these training or technical assistance programs, have you initiated any new municipal processes, initiatives, or policies, or have you opened any new offices or provided any new services? Explain. 3. Have you improved any existing services as a result of LGRP programs? Describe how those LGRP programs enabled you to get started in those new or improved programs, initiatives, etc., and if there was sufficient follow-up by the LGRP contractor to enable your success. If yes, what sorts of follow-up were done, and was it sufficient? If not, what would have helped you more? 4. What financial expenses has your municipality incurred in instituting these programs? How do you feel about the expenses as an investment in better municipal management, policymaking improvements, new or better services, etc.? 5. What amount of staff time was incurred in designing and starting the new or improved services, better policymaking function, etc.? Include training course attendance and the time devoted by you and any of your staff. 6. For the LGRP programs that required significant time or expense by the municipality, do you feel that the LGRP contractor presented all reasonable options for doing the new programs or making the improvements? 7. Are the new programs or improvements capable of being maintained over several years, and if so, what is the funding plan for them (donor assistance, self-funding, government support or subsidies, some combination of these, etc.)? 8. What are the major new competencies you think will be given to municipalities by law, and have the LGRP programs prepared you to accept new responsibilities in those areas? Or has the LGRP prepared you in a more general way to expand the size and scope of municipal government? 9. As the Law on Local Self Government takes hold, and as a Law on Local Self Government Finance is enacted soon, what kinds of municipal finance or other capacity- Development Associates, Inc. Evaluation of the Local Government H-3 May 2002 Reform Project in Macedonia building programs would help you the most? Please mention any that LGRP could develop or modify, and also any that could be developed by other donors or by Macedonian trainers, local academics, municipal associations, other NGOs, or anyone familiar with municipal capacity issues. Citizen Information Centers 1. Please describe how the concept of citizen information centers was introduced to you by the LGRP. 2. Are there other components of citizen participation that were introduced to you by the LGRP that you considered adopting? Did you adopt any? Explain. 3. What were the benefits of CICs that were explained to you as part of the early discussions with the contractor? Who was to benefit and how much investment was to be required of the municipality? Were those early estimates of needed resources accurate? 4. How many other CICs opened in Macedonia before yours? Did the LGRP contractor provide you with descriptions of the CICs’ benefits in other municipalities that were highly useful to you in deciding how to set yours up, manage it, and benefit from it? 5. Do you have “success stories” about how the CIC has benefited citizens in major ways, or in how the CIC has enabled you to make better policies or decisions for the municipality? 6. Were there any big mistakes(“failure stories”) that were made in the design of the CIC that have caused the municipality to redesign the CIC, or which caused the municipality to lose money or tie up staff unnecessarily? 7. What do you believe the public sees the purpose and role of the CIC to be? Did the LGRP provide you with enough information that you can comfortably explain the role and purpose of the CIC to the public and to the Council? 8. Explain the monitoring process you use to determine the effectiveness of the CIC over time. How was the monitoring process developed? Did the LGRP provide you with highly useful technical assistance in setting up the monitoring, and does the LGRP now assist you in collecting and analyzing the data and modifying your CIC programs accordingly? 9. In what ways do you share information with other municipalities which have CICs? Does the LGRP provide assistance to you to in the information sharing? 10. What NGOs are interested in the CICs? Do they participate in the running of the centers? Do you give them data, and what do they use it for? 11. What are the most frequent inquiries made by citizens? How do you organize information about complaints or inquiries, and what do you do with that information that helps you manage your self government better? Development Associates, Inc. Evaluation of the Local Government H-4 May 2002 Reform Project in Macedonia Representative Questions for Non-Participating Small Municipalities 1. Please tell me about the history of this municipality in terms of whether it once was linked with or part of another municipality. 2. Do you currently share services with another municipality or do you contract for some? 3. Which services do you provide for your residents? 4. How many central administrative staff do you have? Job titles? 5. How many service delivery staff do you have? 6. Where do you get information about the new laws and the decentralization process? Newsletters? Associations? Newspapers and television? 7. Do you expect to merge with any other municipalities when the new Law on Territorial Divisions is passed? What do you know about that draft law, and where do you get information on it? 8. I am here in Macedonia to evaluate the Local Government Reform Project of USAID. That project has been run by a contractor called DAI. Have you heard of USAID? Have you heard of DAI? 9. Have you or any of your staff attended any training sessions on municipal government skills or issues over the past year or two? What training, where, who went? Were the training sessions run by DAI or by some other company or foreign agency? 10. What were the benefits from the training program? Is there anyone here in the office now or nearby who attended? May I speak with him or her? 11. Assuming that you are going to continue as a municipality and not be merged with another, what do you see as the greatest needs of your municipality’s administration over the next few years? Are you aware of the new competencies you are going to have to manage, and how are you preparing for that? 12. What specific skills training or technical assistance do you need for your existing staff and for yourself and your Council? LGRP Macedonia Final Report.doc/EvaIQC-07