IMPACT EVALUATION OF THE COMMUNITY REVITALIZATION THROUGH DEMOCRATIC ACTION PROGRAM (CRDA), SERBIA LOCAL GOVERNMENT REFORM PROGRAM (SLGRP), AND SERBIA ENTERPRISE DEVELOPMENT PROJECT (SEDP) Submitted to: USAID Prepared by: The Mitchell Group June 2, 2008 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 i Table of Contents EXECUTIVE SUMMARY ...................................................................................................... E-1 E.1 INTRODUCTION............................................................................................................. 1 E.2 METHODOLOGY AND SCOPE ................................................................................... 1 ECONOMIC IMPACT ................................................................................................................ 1 CITIZEN PARTICIPATION ...................................................................................................... 1 E.3 FINDINGS ......................................................................................................................... 2 E.5 CROSS PROGRAM SYNERGIES ................................................................................. 6 E.6 LESSONS LEARNED ...................................................................................................... 7 1. PURPOSE OF ASSESSMENT ............................................................................................ 1 2. PROGRAM BACKGROUND ............................................................................................. 2 2.1 CRDA Design and Implementation Strategies ............................................................... 2 2.2 SLGRP Design and Implementation Strategies .............................................................. 3 2.3 SEDP Design and Implementation Strategies................................................................. 4 3. APPROACH AND METHODOLOGY .............................................................................. 5 4. ECONOMIC IMPACT AND CITIZEN PARTICIPATION FINDINGS ....................... 7 4.1 Economic Impact ............................................................................................................ 7 4.1.1 Findings................................................................................................................... 7 4.1.2 The effect of the change from CRDA to CRDA-E ............................................... 13 4.1.3 Models for Future Economic Growth ................................................................... 14 4.2 Successes and Challenges ............................................................................................. 17 4.2.1 Economic Grants to Not-For-Profit Organizations Not Always Successful ........ 17 4.2.2 SEDP Assistance Had the Lowest Impact When It Was Intermittent .................. 17 4.2.3 Factors Impeding SME Growth and Sustainability in Serbia ............................... 18 4.3 Impact of Grant Disbursement on Employment ........................................................... 21 4.3.1 Grand Awarding Process ...................................................................................... 21 4.3.2 Leveraging Impact ................................................................................................ 22 4.3.3 Efficiency of the Grant Mechanism ...................................................................... 22 4.4 Key Obstacles to Implementing Economic Development Projects in Serbia ............... 22 4.4.1 Government and Government Regulations ........................................................... 23 4.4.2 The Judiciary ......................................................................................................... 25 4.4.3 Lack of a Business Support Network.................................................................... 25 4.4.4 Limited Availability of MSME Finance ............................................................... 26 4.4.5 Business Management Skills ................................................................................ 26 4.4.6 Inflexible Labor Markets ...................................................................................... 27 4.4.7 Non-Competitiveness of Serbian Businesses in World Markets .......................... 27 4.5 Contribution of CRDA/SLGRP to Citizen Participation .............................................. 29 4.5.1 Contributions of CRDA to Citizen Participation .................................................. 30 4.5.2 Contribution of SLGP to Citizen Participation ..................................................... 34 4.5.3 Suggested Additional Work .................................................................................. 40 4.5.4 Monitoring and Evaluation ................................................................................... 40 5. FINDINGS FOR ADDITIONAL AREAS OF INTEREST ............................................ 42 5.1 Monitoring and Evaluation ........................................................................................... 42 5.1.1 Review and Critique of Indicators and Performance Monitoring for CRDA, SEDP and SLGRP ........................................................................................................... 42 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 ii 5.1.2 Assessing the Performance of Web-PRS .............................................................. 45 5.1.3 Recommendations for future M&E efforts ........................................................... 45 5.2 Cross Program Synergies .............................................................................................. 47 5.3 Lessons Learned and Recommendations ...................................................................... 48 5.3.1 Internal restructuring for greater competitiveness ................................................ 48 5.3.2 Improved management skills ................................................................................ 49 5.3.3 Innovative financing methods ............................................................................... 49 5.3.4 Access to Export Markets ..................................................................................... 49 5.3.5 Establishing Linkages with Strategic Partners ...................................................... 49 5.3.6 Improving corporate governance at Serbian MSMEs ........................................... 50 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-1 EXECUTIVE SUMMARY E.1 Introduction In the early 2000s, USAID sought to promote participatory democracy and reinvigorate the economy of Serbia following an extended period of war, economic hardship, and international isolation. Three programs were central to this effort: • The $200 million Community Revitalization Through Democratic Action Program (CRDA) worked at the grassroots level to improve local living conditions and jump-start the economy through infrastructure and small business improvement grants vetted and approved by citizen development committees (CDCs), which were established and nurtured by the project in cooperation with local elected officials; • The $30 million Serbia Local Government Reform Program (SLGRP) worked with municipalities--particularly those that had been penalized by the central government over the years for their opposition to the policies of the Milosovic regime--to restore confidence in local government, strengthen service delivery, and increase financial transparency; and • The $14 million Serbian Enterprise Development Project (SEDP) focused on specific, targeted sub-sectors to increase employment, sales, and exports. E.2 Methodology and Scope Last year USAID decided to assess the collective impact of these three programs, which ran more or less concurrently from 2001-2007. Terms of reference were prepared in October￾November and a team identified to begin work in early 2008. Attention was focused on four major economic impact questions and one on citizen participation, as follows: Economic Impact 1. What do performance data and fieldwork say regarding the circumstances under which local economic and/or sectoral projects are likely to be most effective in achieving economic impact? 2. What are the main breakthroughs and dead ends, based on project performance data and fieldwork? 3. Which of the grant disbursement mechanisms lend themselves to employment generation with reliable attribution? 4. Based on the assessment, what are the key obstacles to implementing economic development projects in Serbia? Citizen Participation 5. To what extent has CRDA/SLGRP contributed to community revitalization in Serbia? Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-2 The terms of reference also called on the team to (a) identify lessons learned, (b) delineate cross￾program synergies, and (c) assess the extent to which program monitoring and evaluation systems had been effective in decision-making. The team was comprised of the following individuals: • Allen LeBel, Team Leader • Clinton Doggett, Democracy and Governance Specialist • Demir Yener, Small and Medium Enterprise Specialist • Tom Easterling, Agribusiness Specialist • Brian Scholl, Monitoring and Evaluation Specialist Following a pre-departure review of project documentation on a CD-ROM provided by the mission, the team assembled in Belgrade and began fieldwork on January 20, 2008. Beyond review and analysis of basic project documentation, the methodology consisted principally of face–to-face interviews with project stakeholders including CDC members, small- and medium￾sized business owners, NGO leaders, and municipal officials. To accurately and consistently gauge aspects of citizen participation, interviews were augmented through the use of written questionnaires, one for citizens and one for municipal officials. Over a four-week period the team visited 39 municipalities in the North, South, Southeast, West, and Central regions of Serbia and conducted approximately 175 interviews involving more than 500 individuals. Following a team presentation of initial findings to USAID on February 20, the team departed Serbia February 21, 2008. E.3 Findings The team found that CRDA had been extremely successful in revitalizing communities and generating immediate and tangible economic opportunity at the grassroots level. The result was a strong economic boost at the local level and renewed confidence in local government. Beneficiaries not only improved their living standards but also became more positive in their attitudes toward the United States. Success was defined as increases in exports and full-time sustainable employment generated. Employment generation was taken as a calculated value, based on exports. Similarly, assistance extended through SLGRP greatly strengthened the capacity of participating municipalities to extend services in an efficient, transparent manner. Opposition-led towns, most of which had been starved for resources under the Milosevic regime, benefited in particular from this program, as did their citizens. SEDP had some successes in increasing fruit exports and in organizing clinical research services for international pharmaceutical companies, as well as strengthening domestic tourism in Serbia. However, it may have missed opportunities for synergy with CRDA on economic growth programs. Working with CRDA to analyze and consciously fill gaps in the value chain, for example, might have generated greater impact. With regard to economic impact, the team found that: • CRDA had had a direct and immediate benefit on local communities, partly through the removal of infrastructural bottlenecks and partly through the stimulus of grants to small Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-3 businesses and organizing cooperatives and associations. Agro-focused projects seemed to have the largest impact, not only by generating jobs but also by getting farmers see the benefit of organizing themselves into cooperatives, an idea that had fallen into disrepute under socialism. CRDA was fast-disbursing, and its menu of project choices was perfectly suited to the post-conflict situation in which Serbia found itself. • SEDP worked mostly at the sub-sector level, so its impact at the individual firm level is much harder to ascertain except in specific, isolated cases. Certain firms, particularly in the agribusiness sector, created more jobs and moved into new export markets with help from the project, but the assessment team was generally not able to pinpoint SEDP’s exact contribution to growth. Oftentimes companies would be coached and made aware of how to show their products at a trade show, but little follow-up was provided to ensure that they actually exported their products. Probably the greatest benefit provided to companies within SEDP’s targeted industrial sub-sectors was the exposure and information it provided regarding markets and technology in the world beyond the boundaries of the former Yugoslavia. The actual economic impact of SEDP’s intervention depended entirely on the companies’ abilities and their desire to apply the information to their own business operations (cases 30 and 31). • SLGRP’s contribution to economic growth was neither immediate nor direct, but it was nonetheless significant. Often working in tandem with CRDA—for example, on establishing business improvement districts—the program helped enhance the economic enabling environment in the cities where it worked. SLGRP also helped municipalities increase transparency in local budgeting processes, thereby helping improve the climate for business. Regarding citizen participation, the team found that CRDA had opened many people’s eyes to the possibility of community action, and judged that this impact has been more or less permanent. Project stakeholders were positive about growth of citizen participation in Serbia, and the smaller the project and the smaller the community, the higher the degree of direct citizen participation and input observed by the assessment team. SLGRP was also a major contributor to increased citizen participation through the institution of open budget hearings and the establishment of modern, service-oriented citizen assistance centers in dozens of municipalities. Data collected by the team through face-to-face interviews with project stakeholders revealed a significant discrepancy in perception between citizens and municipal officials on the degree to which current systems are in fact transparent and fully participatory. Following are summary findings for the five principal questions posted in the terms of reference for the assessment. Question 1: What do performance data and fieldwork say regarding the circumstances under which local economic and/or sectoral projects are likely to be most effective in achieving economic impact?  For economic development to occur, a strong entrepreneurial class is required. In all regions of Serbia the team saw a need for business mentoring and support for more effective production, marketing, and management practices leading to greater competitiveness. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-4  Businesses cannot compete in today’s world without access to technology and information. Many Serb businesses, and especially agribusinesses, are not competitive because they use outmoded production equipment and practices, they have unproductive animal breeds and plant varieties, and they lack information on markets and marketing. The team concluded that, in some cases, strategic partnerships with foreign companies would be a good way to overcome this problem.  A business support network is necessary. The team found that, in Serbia, the number of business associations that actually provide services to members is extremely limited, and national-level organizations tend to be politicized and ineffective as business service providers.  Financing needs to be available, from capital markets and also from the banking sector. Repeatedly we noted that, in Serbia, capital markets are severely limited and credit is not readily available for small businesses.  Government policies, practices, and regulations must support economic development. The perception of entrepreneurs and business operators in all of the regions we visited is that government--especially central government--is the adversary. Government is considered to be part of the problem rather than the solution. There is no expectation that government entities will provide any support to economic and business development in Serbia other than through targeted subsidies. Question 2: What are the major successes and challenges, based on project performance data and fieldwork?  CRDA’s grassroots nature helped producers from associations from the ground up, by creating the sense of mutuality and solidarity on which democratic responsibilities of citizenship depend. Individuals took their own destiny into their hands.  The grants encouraged beneficiaries, especially at the micro level, to determine the needed personal and physical resources and the size of the market for their products. The application process helped people systematically assess whether they could make money at their new business. These factors played a major role in unleashing entrepreneurial potential. Economic grants to not-for-profit organizations were often less successful (case 26).  Agricultural cooperatives not only helped transform rural producers from being confined to their immediate market environment but also increased their market outreach with more value added products. Their incomes rose significantly, which also helped change their perceptions about the coop form of organization that was discredited.  Construction or renovation of economic infrastructure was by far the major contribution of the CRDA program, followed by the larger size grants that leveraged the job creation impact. Grants to agricultural cooperatives had the most lasting economic impact on all the communities they served. The impact of grants mechanism on micro-enterprises was very positive but had a relatively smaller impact on jobs creation. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-5  SEDP helped institutionalize capacity building and skills development through its assistance to the faculties of applied arts, architecture, and information technology. The project also helped establish the Association of Clinical Research Professionals.  SEDP helped some SMEs expand into export markets but was not as successful in textiles as in food packaging, for example. Policy assistance helped the Serbian Import-Export Promotion Agency (SIEPA) improve its service delivery.  Highest impact activities under SEDP included promoting higher value-added exports in the food sector, forming an association of clinical research professionals, collaboration with SIEPA, and training programs at academic institutions. Impact was often low where assistance was provided only intermittently. Question 3: Which of the grant disbursement mechanisms lend themselves to employment generation with reliable attribution?  Larger grants generated considerably more employment than smaller ones.  Screening grantees through a competitive selection process ensured better economic outcomes.  Tailoring the size of grants to the ability of recipients to create jobs, including those for minorities, was highly effective in generating employment.  Leveraging job creation efforts through the Serbian National Employment Agency increased impact (case 35). Question 4: Based on the assessment, what are the key obstacles to implementing economic development projects in Serbia?  Local governments lack legal authority to stimulate business development.  Structural characteristics of financial system inhibit borrowing by MSMEs.  Serbian businesses, including agribusiness, are generally not competitive outside former Yugoslavia. Question 5: To what extent has CRDA/SLGRP contributed to community revitalization in Serbia?  Community mobilization activities were a revolutionary and uplifting experience for most CRDA beneficiaries. Impact was most likely permanent. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-6  CRDA helped raise the physical quality of life significantly in municipalities where the project was active, both through infrastructure improvements and through higher levels of citizen participation.  The smaller the project, the higher the level of direct citizen participation.  Citizens in CRDA areas are more satisfied with the services they receive from municipal government.  SLGRP was effective both in helping municipalities improve the delivery of basic services and in advancing decentralization through policy dialogue and legal reform.  There is a significant perception gap between citizens and municipal officials on whether local government systems continue to be transparent and participatory. Secondary objectives of the assessment included an examination of monitoring and evaluation systems were effective, identifying cross-program synergies, and defining lessons learned from these programs that will support USAID’s next-generation economic development programs. Findings related to these topics are summarized below. E.4 Monitoring and Evaluation  Massive M&E efforts at the implementer and Mission level produced extraordinary amounts of information and data.  Most Indicators were well-selected, however some were broad ranging and consequently ambiguous for purposes of evaluation.  Methodologies for calculating program impact (beneficiaries, jobs created) lent themselves to distortion.  Selection, Integration and Management of Indicators may have been improved by having third party assist in their development.  Web-PRS was major step forward and served well as a tracking system but it never developed into an evaluation tool Additional in-depth surveys would have strengthened evaluation effectiveness of Web-PRS” E.5 Cross Program Synergies  CRDA, SEDP, and SLGRP all helped improve the economic enabling environment.  CRDA and SLGRP worked together to build a solid bridge between citizens and local government. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-7  Greater collaboration between CRDA and SEDP might have yielded more impact. E.6 Lessons Learned  Rapid, tangible, useful projects are an effective tool for increasing citizen participation and awareness.  Grants have the most economic impact where they are jobs-oriented, replicable across sectors, timely, equitable, and urgently needed.  Sector level programs are most effective when they are market-oriented, narrowly focused, link to export sales, are relevant to SMEs, and build sustainable systems for training, research, and education.  To maximize economic impact, grants should be directed at for-profit entities rather than non-profit organizations.  SME growth is an important component of development in a post-sanctions environment.  Matching grants to non-profits created specifically for the project are generally not sustainable but can create significant good will. In summary, the combined legacy of the CRDA, SEDP, and SEDP projects is significant and can be seen at various levels:  Basic infrastructure put into place under CRDA made a major contribution to reinvigorating economic growth at the local level and appears to have been well maintained.  CRDA also created thousands of new jobs. Firms continue to employ more workers than they did before receiving the USAID assistance.  CRDA changed attitudes by opening citizens’ eyes to the power of neighborhood mobilization, organized problem-solving and systematic discourse with local government. SLGRP left municipal officials with a much more constructive attitude toward service provision and citizen input in resource allocation decisions. And SEDP gave firms in selected sectors confidence that they could expand into new export markets.  All three projects introduced new skills, systems, and procedures that remain in place today. Basic procedures and strategies used by CDCs are still fresh in citizens’ minds, and the NGOs that came out of these continue employing these skills in applying for grants from their municipalities and other donors. The systems and procedures established for municipal management—budgeting, strategic planning, service provision improvements—are not only still in place but are being taken to higher levels in second-generation projects.  The principal institutional legacies of the three programs appear to be (a) the local economic development (LED) and economic enabling environment (EEE) committees set up in various Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 E-8 municipalities by SLGRP and successor activities and (b) the Association of Clinical Research Professionals, an advance considered critical to Serbia’s advancement as a player in the international pharmaceutical industry.  SEDP appears to have been successful in forging long-term business linkages between firms in certain sectors—processed fruit, for example—and overseas export markets.  The projects also generated considerable good will after a period of hostility toward the U.S. stemming from the 1999 NATO bombing campaign. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 1 1. PURPOSE OF ASSESSMENT The primary purpose of this assessment was to provide USAID/Serbia with an objective external economic impact assessment of three programs so that their successful results may be applied to USAID’s next generation economic growth programs. The three programs include: • Community Revitalization Through Democratic Action Program (CRDA); • Serbia Local Government Reform Program (SLGRP); and • Serbian Enterprise Development Project (SEDP). Specifically, the assessment was designed to address economic impact and impact on citizen participation. In addition, the secondary objectives were to identify lessons learned, cross￾program synergies, and assess the extent to which program monitoring and evaluation systems were effective in decision making. This report presents results of the four-month assessment, and provides feedback to USAID/Serbia on the assessment’s findings. Section 2, Program Background, includes overviews of each of the programs. Section 3, Approach and Methodology, describes the processes used by the team to conduct the assessment. Section 4, Assessment Findings, presents the outcomes of the assessment and is organized by each of the major assessment questions, including: 1. What do performance data and fieldwork say regarding the circumstances under which local economic and/or sectoral projects are likely to be most effective in achieving economic impact? 2. What are the successes and challenges, based on project performance data and fieldwork? 3. Which of the grant disbursement mechanisms lend themselves to employment generation with reliable attribution? 4. Based on the assessment, what are the key obstacles to implementing economic development projects in Serbia? 5. To what extent has CRDA/SLGRP contributed to community revitalization in Serbia? Section 5 includes findings related to the secondary objectives of the assessment, including: • Lessons learned; • Cross-program synergies; and • The extent to which program monitoring and evaluation systems were effective in decision￾making. Detailed program descriptions, case studies, survey results, bibliography, contacts, and detailed assessment data are presented in the Annexes 1 through 9. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 2 2. PROGRAM BACKGROUND USAID created CRDA and SLGRP in 2000 to address national citizen, governmental and economic problems that included: • Indecisive and fragmented Government coalition; • Economic decline due to weakening socialist production systems and elimination of as many as 35,000 jobs in individual factories subject to NATO bombing; and • Impatient expectations of a citizenry looking unrealistically for State guidance. USAID initiated SEDP in 2003 with its primary focus on linking Serbian SMEs to their markets, increasing the competitiveness of targeted sectors, and to a more limited extent, companies. In developing these programs, political program constraints needed to be considered, such as the Lautenberg Amendment1 and Certification. 2 In addition, the USAID solution had to function within the unique FRY circumstances of a sanctions environment and regional differences among the FRY’s two republics and the Serbian province of Kosovo. Other constraints included the GOS continued lack of cooperation and good faith commitment to the Dayton Accords - a prerequisite to establishing a democratic society and free market economy. As a result, design requirements for USAID’s programs had to be Immediate and inclusive, flexible, and driven by local residence. USAID implemented its “solution” as three programs. Community Revitalization through Democratic Action (CRDA) operated from July 2001 to July 2007. The Serbia Local Government Reform Project (SLGRP) operated during the period September, 2001 – 2006. The Serbia Enterprise Development Program (SEDP) operated from July 1, 2003 to June 30, 2007. CRDA and SLGRP had one mission, bridging the gap in relations between citizen and local government organizations at the village, town and municipal levels. SLGRP also helped with development of legislation such as revenue sharing between the national and local governments. SEDP had an independent economic mission – strategic stimulation of more promising economic sectors of Serbia’s economy. The following sections describe each of the programs’ design and implementation strategies. 2.1 CRDA Design and Implementation Strategies CRDA “aimed at promoting citizen participation in and between communities to identify and address the critical needs for the economic and social revitalization of community life.3 With a 1 The Lautenberg Amendment to the 1998 Foreign Assistance Act restricts bilateral assistance and the US-imposed “outer wall” sanctions that constrain Serbia’s economic access to the international community. The “outer wall” sanctions prevented traditional policy reform advice and technical assistance and potentially large transfers of policy conditioned-capital from the World Bank 2 US administrative certification that Serbia conforms to an American law on cooperation with the international war crimes tribunal is a pre-condition for US assistance to Serbia. Occasional interruption of this certification during the period 2000- 2006 interrupted UAID program efforts. 3 CRDA Cooperative Agreement, Award Number 169-A-00-01-00124-00, Attachment II, Program Description, p. 9. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 3 budget of $200M, CRDA implemented projects in 111 municipalities that represented 613 communities. Five implementing partners were assigned “Areas of Responsibility,” that is geographic parts of the country where they had sole responsibility for implementing CRDA activities. The implementing partners addressed certain “pillars” that included (a) citizen participation ($20,563,000), (b) income generation ($18,414,296), (c) civil works (44,487,099), (d) environment ($8,697,337) and ad hoc earmarks of $5,123,005 for reproductive health and refugee/Internally Displaced Persons (IDP) of approximately 42 ml. Implementing partners had discretion to use their own “grass-roots” designs to organize local communities for CRDA services. Designs evolved during 2001 – 2004, but the basic approach was to have communities prioritize project implementation using the shared resources of CRDA, the community and local government. For the most part, citizen participation was expressed through Community Development Committees (CDCs) that CRDA implementing partners helped create. CRDA grants for public infrastructure, entrepreneurship and small and medium businesses (SME) development served as a catalyst for these CDCs. Once CDCs emerged in communities, CRDA staff would teach the members how to identify, prioritize, plan and propose projects to receive grants on a cost sharing basis with citizens, local governments and sometimes the GOS. Projects included local infrastructure rehabilitation, job creation, support to micro entrepreneurs through “Kick-Start” grants of $200 – $500, junior achievement, capacity building and networking of cooperatives and associations, targeted support to internally displaced persons and refuges, reproductive health and domestic violence problem solving, and more generally leveraging resources through public private partnerships. In August, 2005, CRDA became CRDA-E and its primary purpose shifted from citizen participation to almost exclusively economic development. Following the shift, implementing partners committed between 75 and 90 percent of their resources to economic development. With CRDA-E, implementers were required to report on progress in agriculture sector development, tourism sector development, SME development, economic environment, trade and promotion and market access, and special initiatives not associated with economic development. USAID created a new pillar, “Economic”, to stimulate business and job growth. CRDE-E received $26,648,155 or 48 percent of the $55 ml. allocated to this pillar. 2.2 SLGRP Design and Implementation Strategies SLGRP employed a variety of innovative techniques to help an ever-expanding cohort of municipalities improve service delivery, efficiency, and transparency through better financial management and public procurement systems, computerization, the establishment of modern, customer-oriented Community Assistance Centers (CACs), and the initiation of public budget hearings at which ordinary citizens could voice their opinions on local funding priorities. To measure impact and track progress, SLGRP developed a Municipal Capacity Index (MCI) with concrete indicators showing progress in five key areas--financial management, municipal systems, communal enterprise, transparency, and citizen services. SLGRP also worked at the policy level to speed passage of new legislation aimed at decentralizing fiscal and functional authority to the municipal level. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 4 SLGRP was designed to revitalize community-based management from within Serbian society in order to re-establish strong local government traditions. It operated in 83 Serbian municipalities, including the City of Belgrade and 12 of its 17 constituent municipalities.4 Notwithstanding municipalities that are part of the City of Belgrade – where CRDA did not operate, it worked in 70 municipalities in tandem with CRDA. Within local governments it provided training, technical assistance and hardware. It aimed to improve municipal management and capacity, financial management, public procurement, citizen participation, communal enterprise management, and information technology. SLGRP also worked at the national level to support the institutional development of the national municipal association, that is, the Standing Conference of Towns and Municipalities (SCTM) and to support policy reform, specifically related to government decentralization. Implemented by DAI, its budget was $30 ml for its five year life. SLGRP indicators include (1) program municipalities conduct annual budget hearings which include citizen input to planning processes, (2) program municipalities have transparent financial management systems as measure by publication of annual budget review letters and (3) progress on a municipal capacity index. 2.3 SEDP Design and Implementation Strategies The SEDP Program focused exclusively on Serbia's economic development. The $14.04 million project was implemented by the US consulting company, Booz Allen Hamilton (BAH). It was originally scheduled to operate from July 1, 2003 until June 30, 2006, but was later extended for an additional year until June 30, 2007. SEDP was conceived and developed under Strategic Objective 1.3 of the USAID Mission strategy in effect at that time: "Accelerated Development and Growth of Private Enterprises in a Competitive Environment". Its overall mission was to enhance the competitiveness of Serbian SMEs. Its main goals were economic development and enterprise support. SEDP worked on improving Serbian companies' abilities to access and sell in mainly international, markets. The SEDP approach was to assist entire industries to move away from low value production to higher value markets. It focused on increasing the competitiveness in six sectors, as measured by increased employment, export and sales. The six sectors were fruit and fruit juice, apparel, pharmaceutical research, information and communications technologies, tourism and, wood and furniture products. However, wood and furniture products were dropped due to inadequate industry interest. SEDP addressed economic growth from the sector level. It assisted companies to improve product and packaging for appeal in higher value markets. Overall, its demand driven strategy focused on products, markets, communications, regulatory reform and curriculum development in university faculties. Early efforts to create a National Competitiveness Council were abandoned due to political disunity following the 2004 elections. Cluster-based enterprise development strategies were also abandoned due to long-term distrust of business associations encompassing multiple municipalities. Key performance indicators were (1) exports at the national, sector and sub-sector level, (2) employment created and (3) sales. SEDP, implemented by Booz Allen and Hamilton, had a total budget of $14,036,572 during its four years of operation. 4 DAI, SLGRP Final Report, November 2006, p. ix Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 5 3. APPROACH AND METHODOLOGY The evaluation team divided into three separate but overlapping lines of inquiry. Team I, comprised of D/G Specialist Clinton Doggett and Team Leader Allen LeBel, focused on citizen participation under CRDA and SLGRP, while Team II (SME Specialist Demir Yener and Agribusiness Specialist Tom Easterling) examined the economic impact of CRDA, SEDP, and, to a lesser extent, SLGRP. Monitoring and Evaluation (M&E) Specialist Brian Scholl remained in Belgrade to examine project data systems, review legislative achievements of the program, and provide needed information in real time to the teams in the field. The assessment team employed a combination of documentation reviews, sites visits and interviews with CRDA and SLGRP stakeholders. Information garnered from interviews on citizen participation was augmented and reinforced by the use of questionnaires developed specially for the assessment, one for individual citizens and one for municipal officials. • Individual citizens were interviewed with an eye to assessing impact in the areas of citizen empowerment, improved quality of life, relations with local government, and sustainability. Respondents included former CDC members (some of whom had gone on to establish NGOs of various kinds); managers of public facilities such as schools, health clinics, and community centers that had benefited from CRDA grants; and individual businesses owners who had received “Kick-Start” or other types of enterprise development grants from the program. How many interviewees and were they all respondents? What about non-response to individual questions? • Interviews with municipal officials focused on the extent to which systems were in place for increased transparency, improved service delivery, and active outreach to encourage citizen participation. Interviewees included mayors, deputy mayors, heads of finance and/or economic planning units, and other staff. The team met in Belgrade with USAID officials and consulting organizations conducting background studies related to this assessment on January 21 and 22 prior to beginning field interviews. Interviews in the field began on January 23rd and continued until February 18th. Field interviews were with citizens, staff of community development councils, government officials and business owners who had received assistance from one or more of the three programs. USAID provided the team with lists of contacts for CRDA, SLGRP and SEDP throughout Serbia, except for Kosovo. Interviews with contacts depended on their geographic proximity to the team’s somewhat flexible travel route and the timing of their availability. Violeta Jovanovich made virtually all the appointments for the team, almost in real time, that is, a day or two in advance, and e-mailed daily interview schedules to the team . Each team member in the field traveled more than 4,000 km to attend these interviews. The goal was to have 10 interviews per day. Each day, two team members covered five citizen participation-related interviews, and the other two team members covered five SME and agricultural-related interviews. Team feedback to direct interviews was minimal, except where scheduling misunderstandings occurred. Team members experienced non-response for some Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 6 SEDP interviews. Non-response took the form of interviewee lack of recognition of SEDP assistance. The team conducted 175 interviews in 39 municipalities of Serbia. The assessment did not include Kosovo. A breakdown of the contacts and subset of interviewees contacted is presented in Table 3-1. Program Number of Interviewers Number of Interviewees ACDI/VOCA 15 45 ADF 37 111 CHF 21 63 CHF Grantees 3 9 IRD 27 81 MC 24 72 SEDP 27 81 Belgrade\1 21 63 175 525 Source: assessment team records 1/ Interviews for CRDA, SLGRP and SEDP Table 3-1: Assessment Team Interviews by Program This table indicates the distribution of interviews among the CRDA and SEDP implementing partners. Interviews for SLGRP overlapped mainly with all but the SEDP interviews. Meetings in Belgrade are not categorized by implementing partner. They generally involved discussion of programmatic issues addressing multiple programs. Table 3-1 references a total of 525 interview attendees. This is an estimate of three interviewees per meeting. Sometimes six or more attendees would gather for interview meetings, particularly in municipal or Local Economic Development group settings. Careful records were created for the principal attendees, but often not all of them due to extended discussions after meetings ended. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 7 4. ECONOMIC IMPACT AND CITIZEN PARTICIPATION FINDINGS 4.1 Economic Impact This section of the report provides an assessment of the economic impact of CRDA, SLGRP, and SEDP. Although widely different in their approaches, all three programs contributed to Serbia’s economic development. CRDA began with its primary emphasis on community development through democratic principles, with economic development playing a secondary role. Within three years, however, its focus changed and economic development was thrust into the forefront of project involvement. SLGRP was first and foremost, an initiative to strengthen Serbia’s municipal governments that also worked to improve the enabling environment for business development within its sphere of influence. SEDP was a project dedicated exclusively to economic growth. The indicators used for monitoring the economic impact of the three projects are shown in Table 4-1. These indicators show the desired economic outcomes of the projects. SLGRP SEDP CRDA SLGRP had no economic indicators 1. Exports (US$) 1. Employment created - in person-months 2. Employment generated - full-time equivalent jobs Note – employment generation was a calculated value, based on exports 2. Additional income Generated (US$) 3. Increased agricultural sales (US$) 4. CRDA-E: Employment generated - full-time equivalent jobs Source: SLGRA, SEDP and CRDA final project reports to USAID Table 4-1: Economic Indicators for SLGRP, SEDP and CRDA Projects 4.1.1 Findings The impact of CRDA on Serbia’s business and economic development was substantial and positive over the project life. The combination of its micro-grants, business development grants, technical assistance and economic infrastructure projects had a considerable impact on the targeted municipalities in ways that include: • Economic infrastructure projects helped to solve deficient electric power, water, and access roads that constrained business activity and economic development (cases 9 and 18). • A substantial part of the portfolio of economic development projects in each AOR was composed of agricultural and agribusiness projects. Projects in this sub-sector tended to have the greatest economic impact per dollar spent (cases 6, 7m and 31). While numerical data are not available at the firm, individual, or project level to confirm this finding, it is clear from the team’s field visits and corresponding discussions with CRDA beneficiaries that this finding is entirely valid. The main reason is the secondary effect of an agribusiness project intervention on other actors throughout the value chain. For example, by providing grant funds to help milk and meat processors expand their output (cases 6 and 7), their annual Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 8 turnover increased by roughly 7 times. In addition, these agribusinesses provided market outlets leading to substantial economic benefits for hundreds of small-scale suppliers or raw milk, beef cattle and ruminants. Service industries also benefitted, such as suppliers of animal feed, packaging material, and transporters. Consequently, financial and/or technical assistance provided to an agribusiness had a follow-on benefit for the local community as well. In comparison, the secondary effect of CRDA assistance provided to a manufacturing firm such as a machine shop or a stove manufacturer was much less than for an agribusiness. For these firms, the secondary effect on other businesses and on the local community was limited. • The impact of economic development projects was direct and immediate – for example, a machine installed at a SME manufacturing facility could almost immediately quadruple production output leading to greater sales revenue (cases 1, 2, 11, 16, and 22). • CRDA’s support to cooperative development – e.g. the creation and organization of business cooperatives - was a vital part of the project intervention in rural communities. Moreover, beneficiaries noted that the technical assistance and training that CRDA provided was as important as the matching grants they received for facilities and equipment that enabled their business to start operating (cases 13, 14, 17, 19, and 21). A common theme heard during the team’s interviews with numerous CRDA beneficiaries during the evaluation was the positive economic impact of TA and training, particularly those interventions related to cooperative creation and development. Most of those cooperative businesses the team interviewed considered the benefit from technical assistance and training to be equally important as that of matching grants. While numerical data are not available5 to substantiate this belief by a substantial number of CRDA beneficiaries interviewed, it is an entirely logical conclusion. Before CRDA, given Yugoslavia’s socialist past, a “cooperative” had an entirely negative connotation since production cooperatives were perceived by small farmers as exploiters. An early task of CRDA was to convince individual small farmers of the benefits they could derive from membership in marketing cooperatives that provided valuable services to its members. This was followed by training in cooperative formation and management, and technical assistance to support cooperative development. With the successful creation and strengthening of the cooperatives, the individual members benefited from joint marketing efforts, consolidated purchases of farm inputs, and in many cases, from financial assistance from State institutions such as the Ministry of Agriculture. For these reasons the beneficiaries considered the value of technical assistance and training provided to them equal to that of the grant that they received. • The beneficiaries had simple production-marketing chains. In almost all cases, markets were readily available for the items produced, either locally, or in the FRY. In general, marketing was not a constraint (cases 6, 7 and 17). 5 Neither the project implementation reports nor the CRDA project M&E facility provide data that could be used to generate cost vs benefit comparisons as a result of project interventions with individual project befeficiaries (individuals, companies, or cooperatives); nor is data available to make a comparative analysis of the economic benefit resulting from grants vs TA and training for individual beneficiaries. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 9 • Self-help groups such as industry and producer associations can make an important contribution to the economic development of a particular subsector. These organizations provide an efficient means for channeling information, technology and technical assistance to multiple beneficiaries. After project completion these groups can provide continued support to the businesses within their subsectors to help insure their sustainability. Hence, mobilizing these groups leverages CRDA impacts (cases 12, 18, 20, 21 & 35). • The value chain approach followed by Mercy Corps was found to be the most effective approach for economic development under CRDA-E. First, Mercy Corps analyzed the economic potential of the different sub-sectors within its AOR, and selected two of the most promising sub-sectors: fruit and dairy production and processing. It then established a comprehensive program whereby all project elements including business development grants, technical assistance, and training worked together in a mutually reinforcing manner to support the development of these targeted subsectors. Next, Mercy Corps supported and strengthened agro-processors with substantial business development grants that enabled them to become a viable force in their agro-industry, and to provide a reliable market outlet for farm commodities produced by small-scale farmers. Finally, Mercy corps worked at the farm level to help small producers to improve farm productivity, output, and product quality, while simultaneously working to create farmer associations capable of providing continued assistance to their members. Mercy Corp’s approach to project implementation was unique, and its implementation strategy was comprehensive and highly effective. In comparison, the approaches followed by other implementation partners for economic development were considerably more limited in scope than that of Mercy Corps. For example, ACDI/VOCA and ADF provided business development grants only to cooperative organizations, and not to privately-owned MSMEs. CHF tended to provide small grants to small enterprises, whereas Mercy Corps supported relatively large, as well as small enterprises. Furthermore, with the exception of Mercy Corps, all IPs tended to focus on specific parts of the value chain instead of working along the entire value chain to solve problems and remove constraints to agricultural production and marketing. A comparison of the reported increase in agricultural sales for the entire CRDA project life shown in the following table supports this conclusion. CRDA-E Life of Project Economic Impact by Implementer Implementing Partner ACDI/ VOCA ADF CHF IRD Mercy Corps Total Increase in agricultural sales (US$ 000) 6,973 16,449 2,827 8,694 29,253 64,196 Source: Project Reporting System (PRS) website http://www.sada.usaid.org.yu/ Table 4-2: CRDA-E Life of Project Economic Impact by Implementer • A more intense project intervention with only a few beneficiaries has a considerably greater overall impact than limited interventions covering a large number of beneficiaries. One example that illustrates this finding is the comparison between SEDP’s approach to firm￾level technical assistance with clothing manufacturers and the approach followed by EBRD’s Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 10 Turn-Around Management (TAM) program6 . SEDP covered a large number of companies but provided limited assistance, whereas EBRD’s approach was more in-depth assistance for fewer beneficiaries. For EBRD’s approach there was a significant impact for the beneficiaries because it helped them make fundamental improvements to their businesses. • The CRDA menu of services was highly appropriate for the post-sanctions, depressed economic and political environment that existed in Serbia in 2001. In the opinion of the assessment team, SEDP’s contribution to Serbia’s economic development were considerably less than that of CRDA, for the following reasons: 6 • SEDP generally worked with service organizations within broad industrial sub-sectors (e.g. SIEPA; Ministry of Agriculture) to remove constraints, and with groups of enterprises to help link them with external markets. Since SEDP worked at a level above the individual companies, it took longer for the companies themselves to realize any benefit from the SEDP intervention – and the nation’s economic output is directly affected by companies, not by service institutions within their industrial sub-sector (case 4). • SEDP’s interventions at the firm level were relatively limited. For example, SEDP helped a several companies to obtain HACCP certification, which is a necessary, although insufficient condition to generate export sales, and may or may not result in increased business activity (cases 5, 24, 30, and 31). • The grant component of SEDP played a smaller role in promoting economic expansion for the project, in comparison with CRDA. While there was no articulated preference in SEDP’s contract to provide an immediate returns to its beneficiaries, the impact of its grants were relatively small. First, the magnitude of SEDP’s grant component was limited, compared to CRDA. Second, its matching grants were generally used for marketing and promotional materials, and for participating in events targeted on external markets that in many cases did not result in an immediate return to its beneficiaries. • SEDP’s interventions were generally limited in scope, and not continuous. For example, companies would be coached and made aware of how to show their products at a trade show, but no follow-up was provided by SEDP to ensure their ability to actually export their products. For example, case 3 shows Ethno-Foods, receiving SEDP guidance to attend the Fancy Food Show in New York, securing a proposal from Williams and Sonoma, but not 6 Based on feedback the team received upon submitting the draft evaluation report, USAID/Serbia is not in favor of providing in-depth, firm level technical assistance to Serbian SMEs. However, given the concrete benefits derived by the SMEs from TAM training and technical assistance (improved business practices, improved operating efficiency and improved competitiveness), the assessment team believes this is an effective approach to business development. Regardless of its acceptability as a development tool, we believe that it does demonstrate the finding that a more intense project intervention with only a few beneficiaries has a considerably greater overall impact than limited interventions covering a large number of beneficiaries. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 11 following though. Continued assistance to Ethno Foods may have resulted in dramatic increases in sales7 (see case 3 in the Annex). Another example is shown by SEDP’s intervention with the JASMIL Knitting Company in Arilje (case 34). JASMIL’s owner told the assessment team that SEDP provided a useful introduction to the US market, but little or no technical assistance. SEDP staff did not follow-up with JASMIL subsequent to its consultant’s brief orientation to the US market. Overall, they were not satisfied with the SEDP program. Yet another example is shown by the experience of Todor Fashion Wear Company in Vrnjacka Banja (case 27). According to Mr. Vladimir Todorovic, Executive VP, Todor had a stand at the New York Pret a Porter Fashion Fair in year 2006. However they were very disappointed to receive only a very small order of around $2,000. Frustrated, they returned home, but tried to deliver the goods. Good were shipped to the buyer but were returned because of late delivery. Whatever the outcome, for Todor SEDP was not a good experience. They tried contacting the US consultant, who did not return their calls, and SEPD did not follow up with them. A final example was described by Mr. Milos Krcevinac, General Manager and Association Member, Arinova Apple Growers Association in Arilje (case 24). in 2006, SEDP, in cooperation with SIEPA, financed a study tour of fruit growers in Bolzano, Italy for the Association members. Even though some members were not able to obtain EU visas due to political status of Serbia, three of their members eventually made the trip. IRD provided the association with 15,000 cartons of 10 kg packaging boxes along with promotional material. The boxes, designed by SEDP consultants are the containers used for shipping to export and domestic customers. The association did not receive any further technical or marketing assistance from SEDP. • Many of SEDP’s client companies were not were not at a stage of development where they could benefit from SEDP’s intervention. For example, many Serbian food and textile companies were introduced to foreign markets, but they had neither the level of sophistication nor the business capability to capitalize on the market opportunity. It seems that SEDP’s menu of services that it provided Serbian SMEs was too limited in scope or too sophisticated to substantially benefit many targeted SMEs. • Probably the greatest benefit that SEDP provided to companies within its targeted industrial sub-sectors was exposure and information about markets and technology in the wider world, beyond the boundaries of FRY. The impact of SEDP’s intervention depended entirely on the companies’ abilities and their desire to apply the information to their own business operations. Case 4 for Magelan Travel Agency shows SEDP assistance with promotional 7 Mr. Remer Lane, the previous COP of the SEDP project, disagreed with this statement. USAID feedback to the assessment team after the team submitted the draft evaluation report stated the following: “This conclusion is informed by a single view and a follow up discussion with Mr. Remer Lane, who was briefly interviewed before the fieldwork but not after, could have revealed that Ethno-Food actually provided incorrect information to SEDP about their production facilities. Even so, SEDP offered to hook them up with a co-packer, but it was actually Ethno Food who was not being responsive and did not follow-up”. Despite this disagreement over who was at fault in this particular case, the team believes that the conclusion is entirely valid. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 12 material and an owner’s continuing attempt to translate resulting publicity into a reservation booking system (case 10). • In some cases, SEDP rightly collaborated with other programs (SIEPA; Ministry of Agriculture) to link Serbian cooperatives and associations to export markets. For these shared efforts, it is not possible to determine the economic impact of only one participant, such as SEDP (cases 10 and 20). • In hindsight, it appears that SEDP’s client companies needed more practical, hands-on, in￾house technical assistance and training interventions than SEDP was able to provide. For example, the vast majority of the textile companies the team interviewed were not satisfied with SEDP’s interventions to improve production efficiency on the factory floor. They felt that SEDP’s interventions were too brief and too limited in scope to provide much benefit. In comparison, many of the same companies spoke very highly of the EBRD’s current Turn￾Around Management (TAM) program, where experienced textile executives are assigned to work within a single company for several weekly sessions over a period of eighteen months. During that time, the visiting executive directly assists company management to improve the company’s administrative procedures, production processes, marketing efforts, product sales, product design, and management practices across the board. SLGRP’s contribution to Serbia’s economic development was limited in scope and impact, but it did have positive impacts on the enabling business environment within its targeted municipalities. SLGRP’s contribution to Serbia’s economic development, as noted above, focused on: • Improving enabling business environments within its targeted municipalities by developing Citizen Assistance Centers (cases 8 and 9); • Improving municipal financial management and procurement (case 12); • Shared creation with CRDA of business improvement districts (BIDs) in four municipalities fostered public-private partnerships that supported economic development.8 (case 15); and • Creation of Local Economic Development offices, sometimes with CRDA (cases 12 and 23). Through the creation of business improvement districts (BIDs) in four municipalities, SLGRP fostered public-private partnerships to improve economic development. It also created a model for BID development that MCI used to create three additional BIDs (cases 8, 9, and 15). There was relatively little synergy noted by the assessment team between the economic development activities carried out under CRDA-E, and the SEDP project. This is not too surprising, since CRDA-E had a geographical focus and SEDP’s focus was on specific industrial sub-sectors. Also, CRDA focused on smaller businesses, including agricultural cooperatives and 8 Two of CRDA’s Implementing Partners, IRD and Mercy Corps completed additional BIDs in Loznica and Prokuplje with a CRDA predecessor, MEGA. Mercy Corps also completed a BID by itself in Prijepolje. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 13 associations, whereas SEDP’s focus was on larger exporters. However, greater emphasis on collaboration for mutual benefit by the two projects could have provided greater synergy. There were a few notable exceptions to this general finding. For example, Mercy Corps provided matching grants for cold storage facilities to two mushroom exporters, and SEDP helped these companies link with US markets. Another joint intervention occurred in Central and Western Serbia, where the CRDA-E implementing partners in these locations assisted spas and other tourist attractions by installing minor infrastructure such as road signs, hiking trails and rest areas, and SEDP provided marketing training to the organizations that managed these facilities. Overall, however, the limited collaboration between the two projects was a missed opportunity to achieve greater economic impact. Similarly, there was not a great deal of reinforcement between the economic development component of CRDA and CRDA’s community development activities. These two CRDA components were generally implemented as entirely separate activities. This is by no means a negative finding; it simply reflects the inherent differences between the two major components of CRDA. In view of these differences, two of the five CRDA implementing partners – Mercy Corps and ADF – hired specialized sub-contractors to carry out economic development activities on their behalf. 4.1.2 The effect of the change from CRDA to CRDA-E The August 2005 decision by the US Embassy in Belgrade to change the raison d’être of the CRDA project from a predominantly civil society focused-program for community development to a project whose main purpose was economic development was not only highly controversial, but also had a profound impact on project implementation. The assessment team would be remiss unless it addressed the effect of this mid-course change on the overall impact of CRDA. The following are the team’s findings and conclusions on the effect of changing from CRDA to CRDA-E. By changing the emphasis of the project in years four and five to job creation and economic development, there was undoubtedly a positive impact on economic development. However, the impact was not as much as it seems. First, by the end of the fourth year of the six-year project, most of the funds had already been spent. Second, non-economic projects that were underway had to be completed, instead of abruptly suspending this work. Third, many of the projects categorized as economic development would have been carried out anyway, under the community development component (e.g. electricity and water infrastructure projects). While the change to CRDA-E resulted in increased economic development activity, this came at a considerable cost. This unilateral change amounted to a broken contract with the grassroots community organizations who were involved in CRDA. Since the relationships between the project and the communities it served were formed within the context of “America cares”, this change undoubtedly heightened the degree of cynicism by the involved community groups. This change also interrupted the CDC efforts to build trust at the local level, a prerequisite for later establishment of associations and cooperatives. Beneficiaries particularly in Southern Serbia appear to have needed more time for trust and understanding of the benefits of collective efforts to permeate communities. Furthermore, many municipalities that had been waiting patiently for their turn to benefit from CRDA-sponsored civil works projects were told that their requests Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 14 could no longer be considered under CRDA-E. This led to the departure of some municipalities from CRDA. The interruption of project activities supporting community development and confidence￾building measures between Serbia’s different ethnic and religious groups resulting from CRDA￾E, noted above, occurred late enough in the project cycle to reveal lasting benefits. The predominantly ethnic Albanian areas surrounding Kosovo’s administrative borders remained quiet during the 2004 political unrest in Kosovo, largely the result of CRDA’s interventions to build social cohesion within these areas. A program designed to be primarily a community-based development and civil society building initiative is unlikely to foster the best possible economic development program. For example, the most economically sound approaches the team saw in CRDA were ones most removed from CRDA’s primary participatory process. Cases 6 and 7 show MCI unilateral decisions to assist a dairy and abattoir grow rapidly to support the MCI value chain strategy. These opportunities might have been lost had these decisions been made entirely at the community level. While the assessment team has concluded that economic development under CRDA-E was effective, it likely would have been even better if implemented from the outset as a separate MSME program for economic development and job creation. Such a program might have meant selection of different implementing partners – a set with focused economic development skills. By contrast, the USAID’s CRDA implementation partners that were contracted by USAID were undoubtedly chosen for their skills and experience in implementing a civil society focused￾program for community development with a minor component for grassroots economic development. To confirm this hypothesis, the assessment team noted that two of the IPs – Mercy Corps and ADF – wisely hired professional managers at the outset as sub-contractors to help implement their economic development component of CRDA. This decision served them well after the change to CRDA-E. The successful implementation of a community development program requires a different style and approach than the successful implementation of an economic development program. There is an inherent tension between the two approaches. Table 4-2 compares the approaches that are required for successful implementation of the two types of programs. Civic Engagement/Community Development Programs Economic Development Programs Process oriented Results oriented Encourages collaboration Encourages competition Seeks social cohesion Seeks accumulation of wealth Participatory approach Encourages self reliance Provides social benefits and social cohesion Generates profits Table 4-3: Approaches Required for Successful Program Implementation 4.1.3 Models for Future Economic Growth In view of the constraints to economic growth and business development in Serbia and in light of the positive results on economic development achieved by SLGRP, SEDP and CRDA, the Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 15 assessment team has devised the following recommended models for future economic growth in Serbia. These proposals are designed to enhance the competitiveness of Serbia’s MSMEs, including agribusinesses. 4.1.3.1 Model for Economic Growth –SMEs9 The growth and development of Serbian SMEs is severely limited by their lack of competiveness in international markets. Eventually, they will face increasing competition from globalization, regional trade agreements, WTO membership, and eventual EU entry. Consequently, they must become more competitive to survive a flood of foreign products reaching their local markets. In other words, international competition truly begins at home. Many Serbian SMEs must undergo firm-level restructuring to become internationally competitive. Through restructuring, a company will be able to limit its financial vulnerability and improve its profitability and cash flow. Restructuring involves a three-step process. First, the firm must undertake an internal analysis and review of its operations. It must define (or re-define) its mission, values, goals, objectives, strengths, weakness, competitive position, product line, and available markets. Second, based on the internal analysis, a firm must devise a strategy for growth and development over the intermediate term. The strategy must consider issues such as access to credit, access to export markets, and linkage with strategic partners. Third, it must formulate a plan of action to bring about fundamental improvements in its business operations. In carrying out its restructuring plan, a firm must take specific action to increase productivity, increase operating efficiency, capture new markets, and improve its financial results. SMEs must improve their business management skills, through management training and by bringing professional management into family businesses. This process will help SMEs to become more profitable and more credit-worthy, which will make it easier for them to obtain credit for working capital and business expansion. As SMEs become stronger and more competitive, for continued growth they will have to expand into international markets. Establishing linkages with strategic partners can enhance competitiveness by providing access to new markets, new technology, and financing for joint investments. These elements form a model for economic growth for SMEs in Serbia. 4.1.3.2 Model for Economic Growth – Agribusiness The development of Serbia’s agribusiness potential could be carried out through the careful identification of opportunities, targeted technical assistance and training interventions, and by 9 USAID/Serbia does not agree with this model recommended for economic growth of SMEs in Serbia. Feedback on the draft assessment report stated that “recommendations on SMEs seem to reflect the Team members’ own biases towards a specific approach (turn around management) rather than being built on what they learned about the approach taken under SEDP. Moreover the Team’s comments do not seem to be informed by the long and extensive process undertaken by the mission, in conjunction with EG/Washington, in defining the approach the mission is taking today.“ The recommended approach is based on the team’s experience on successful development practices in other locations, along with its assessment of the challenges to SME development that presently exists in Serbia. This recommendation may not be entirely consistent with the previous approach taken by SEDP, nor with USAID/Serbia’s current programs. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 16 working with the Government of Serbia to improve the climate for agribusiness development. The entire development process must be led by private businesses that supply selected agricultural products to local, regional, and international competitive markets. The ultimate beneficiaries of these interventions should be the small- and medium-scale producers who are the primary suppliers of agricultural products to these markets. A SEDP-type activity could lobby quite valuably for sector level change once private businesses identify needs. The team proposes a market-led agribusiness development model to strengthen selected agro￾industries by incorporating value chain concepts to increase the production, processing, and sale of related agricultural products. An agricultural value chain describes how producers, traders, and processors work together to add value to commodities before they reach their final market. Although the product moves physically along the value chain from farmer to producer and finally to a consumer, the starting point for determining the direction of the value chain must be the market and its demands, rather than the farmer, to ensure that demand exists for farm production. Several important principles should guide these interventions: • Focus on a certain number of the most promising commodities for diversification, export, and for import substitution. • Interventions must be market-led. Market-led means that the product is being offered in whatever form the market requires; the volume of commodity produced can be sold during a defined seasonal period, and the price received is sufficient to be profitable after deducting all real costs. All activities must be carried out in light of market requirements and must work to meet market demand. • Operate along the entire value chain to solve problems when they appear and to help remove constraints to development. • Leverage resources by working collaboratively with other development initiatives that support development objectives. • Provide support to key players, entrepreneurs, and risk-takers throughout the value chain, and focus interventions on the weak links in the chain. In other word, resources must be used to reduce the risks incurred by the risk-takers within the targeted agro-industry. For example, if small farmers are unable to produce an agricultural product that meets market standards, development resources should be used to help improve standards. • Finally, it will be necessary to create producer and industry associations with the capability to provide ongoing support services to the different actors within the targeted agro-industries. Development resources should be used to ensure the financial viability and the technical and managerial capability of these associations. These associations should continue to support the value chains after development assistance has ended to help ensure sustainability of the targeted agro-industries. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 17 4.2 Successes and Challenges CRDA’s broad dispersion of projects throughout Serbia has not only affected the livelihoods of citizens throughout the country, but it has shown the value of personal initiative in changing one’s economic conditions. SEDP was more specific, and it more often than not focused on sector level assistance, with very few firm level interventions. SEDP has generated benefits mostly for firms with adequate production capabilities, with scale and management capacity to enter into new ventures and export markets, and with ability to sustain the quality of their products. This had its pluses and minuses. Under both CRDA-E and SEDP, some approaches had little impact. In fact some approaches impeded overall program success. Each is addressed in the following sections. 4.2.1 Economic Grants to Not-For-Profit Organizations Not Always Successful Under CRDA-E, some grants were made to not for profit NGOs. Most of these NGOs disappeared, or are bound to disappear after a few years. The assessment team thinks that this is mainly due to management’s lack of entrepreneurial skills, or adequate knowledge of market skills. However, the overall impacts of these interventions on the communities were very positive, and created good will. If we consider this aspect, they can be considered as part of the successful community revitalization programs. One case is the Sokobanja Women’s cooperative named Ethnos that was given a small grant for building remodeling and sewing machinery to make ethnic costumes and hand crafts (case 32).10 The location of the remodeled building is 3 kms outside the town, and does not have that much tourist appeal. The 7-10 women who are active are not full time, and they do not always come to the Center to work and create their hand crafts. The management is poor. However, it was a good idea to support the traditional Serbian life styles through this intervention, even though it is not profitable. Another similar case is the Izvor Association of Knajeva (case 33).11 4.2.2 SEDP Assistance Had the Lowest Impact When It Was Intermittent The one size fits all does not always work. Regional realities must be taken into consideration. CRDA-E was designed with the regional realities in mind, even though it involved refocusing the existing CRDA project. For instance, Mercy Corps and CHF applied different approaches to awarding grants in their AoRs to beneficiaries. MCI implemented a policy of larger grants, whereas CHF followed a policy of medium sized grants. Both were successful in overall outcome because the characteristics of the businesses, business capabilities and outlook on life and business in their regions were quite different. MCI occasionally waived the matching grant requirement whereas CHF relied on matching grants for business development. However, in both cases, the beneficiaries were given hands on assistance and both implementing partners followed up on MSMEs progress and provided technical advice when necessary. SEDP applied an across the board approach because its focus was more sectoral and not firm level. However, the Team finds that this approach did not successfully work in the textile sector, while it did work in the food and agriculture sectors. In terms of SEDP, one example is the case 10 Case 32: Sokobanja Women’s Cooperative- Ethnos 11 Case 33: Knajeva Izvor Women’s Association Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 18 of JASMIL Knitwear Company of Arilje (case 34).12 SEDP assistance to JASMIL began with good will and high expectations, but resulted in failed company aspirations to access the US export market. Despite this negative outcome, the company executives consider their association with SEDP as a learning experience. Others such as Todor (case 27) had more harsh assessment of their experiences with the SEDP. The assessment team believes that SME’s appear to have the highest likelihood of sustainability if they gain access to regional and national markets before USAID assistance terminates. Such access depends partially, but critically on access to finance and reform of legal and regulatory frameworks before termination, as discussed subsequently. 4.2.3 Factors Impeding SME Growth and Sustainability in Serbia Based on the assessment, the key obstacles to implementing economic development projects in Serbia are quite a few. These are discussed in the following sections. 4.2.3.1 Weakness in Accounting Systems The importance of the efficiency and effectiveness of micro and SMEs in Serbia and their activities require more emphasis on aspects of accounting that include: • Curriculum development in business accounting and auditing concepts and theory, • Accelerating the accounting reform, • Accounting training and education, and • The scope for special certification (or diplomas) in accounting and auditing. Serbia has long been part of the international fora on IFRS Rules of Accounting and Reporting standards. However, having interviewed many SMEs executives, the assessment team is aware that most enterprises do not seem to maintain proper accounting records, or keep dual accounting reports—one for themselves, one for the government. Case 14 shows a cooperative believing its making a profit of $10,000, though it doesn’t include 43,000 Euros of debt servicing or depreciation in its financial statements. Some managers maintain that the duality of accounting records are necessary because of perceived excessive government interference with business practices which have at times overwhelmed their firms, and stretched their resources. Others simply wish to operate extra-legally in the gray economy to escape from public scrutiny. Few SMEs indicated that they rely on their accountants or other agents to keep accurate accounting of their operations. Yet, accurate accounting records will be very crucial if Serbian companies are to price their products and services competitively or attract capital from financial institutions and investors. Hence, they need to improve the perception of Serbian companies among creditors and investors alike, if they can improve the reliability, relevance, replicability and timeliness of financial information. This will improve transparency of businesses, resolve business risk, and allow the SMEs to gain better access to sources of finance. Most managers do not seem to have a good handle on their accounting statements. 12 Case # 34: Jasmil Extreme Intimate Apparel. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 19 The lack of management capacity to understand accounting and financial control process properly also lowers managerial efficiency. Weakness in interpreting the financial information prevents the company from calculating its risks, and determining its prices in a competitive manner in the market place. Serbian SMEs could do much better if they had proper knowledge, understanding and control of their costs. The assessment team surmised that due to poor pricing policies, SMEs lose valuable customers, or sales leading to irreconcilable losses. The team’s conclusion is that SMEs and perhaps banks and other similar financial institutions operate in an inadequate accounting and auditing framework, without the tools for efficient and effective management. The inadequacy of accounting systems limits the understanding of financial statements, while the absence of appropriate reporting and supervisory guidelines constitutes significant dangers for the financial system. 4.2.3.2 Inadequate Management Skills Management skills that will put the MSMEs on par with similar enterprises in Europe are generally lacking in Serbian SMEs. Many non-agriculture sector businesses are family owned and operated and non-competitive. CRDA assistance for instance may have helped launch hundreds of small family enterprises, but, their operations are at managed mostly in traditional ways. Many privately owned Serbian companies have reached a size that will require new management talent so that they can take on the next step into their sustainable growth as larger, more medium sized companies with institutionalized corporate cultures. These firms have typically reached over 150 -200 employees, have export sales, and could use significant amounts of financing to reach their full potential as internationally competitive entities. Management of most family owned operations have now been transferred to the second generation, but professional management skills are still lacking. CRDA and SDEP were not designed to address MSME management needs, though SLGRP helped improve municipality accounting standards. The moribund economic conditions that particularly CRDA was designed to address have improved dramatically since 2001. Now, management talent is more necessary to sustain the healthy growth trend being achieved. The assessment team’s observation is that management skills are much better developed in the North, with competitive export oriented production capabilities than the mainly family oriented firms in the South and the Southeastern parts of the country. The firms in the Southern and Southeastern regions are also relatively smaller, but still help create a vibrant economy in their communities. Increasing executive management and marketing skills could be an extremely important USAID contribution to MSMEs in the future. 4.2.3.3 Information Gap Business success depends on information and improved knowledge management. It allows them to gain access to new markets for their products, and learn about competitive pricing. In addition, it helps firms network for guidance and direction such as is necessary in developing a business plan or, applying for a bank loan. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 20 There are regional SME development agencies in Serbia that could be useful in this capacity and skills upgrade and development efforts. However, they have severely limited resources and outreach into communities in all parts of a municipality. SEDP attempted to create an environment in the Southern regions of Serbia that would allow the local firms in targeted sectors to build managerial skills in marketing, finance and management, export potential and provided relatively minor but important firm level assistance to ensure higher productivity, more efficient capacity utilization, and quality of products to improve competitiveness. SEDP collaborated with the Sandzak Enterprise Development Agency (SEDA), the regional SME development agency in the Novi Pazar Region.13 Working with SIEPA, SEDP successfully provided guidance and direction to Serbian exports in textiles, food, furniture and plastic sectors. We have mentioned earlier a SEDP initiated electronic price discovery mechanism for agricultural commodity prices, which can be accessed by dialing a certain cell phone number. Farmers interviewed appreciated this system. This was useful however not adequate. Serbian SMEs should have increased access to global prices of not only commodities, but also goods that are to be exported, or simply benchmarks so that competitive corporate policies can be devised. 4.2.3.4 Legal and Regulatory Frameworks CRDA, SEDP and particularly SLGRP worked on legal and regulatory issues for business, but more work is needed. The overly centralized government system that places the authority for regional and local governance affairs in the hands of Belgrade is a major impediment to further development of business, especially for Micro and SME businesses. Many SME executives have indicated that the Central Government is a part of the problem and not the solution. Each region has its own idiosyncratic characteristics and realities that are different from other regions. People interviewed by the Assessment Team indicated they would always be better off if they could avoid Central Government regulation. Chambers of Commerce are not seen as friends of the private business either. By law, all registered economic entities must become member of their respective regional chambers of commerce, and all registered firms are required to pay dues to the Chamber of Commerce. Business owners complain that they do not receive any services in return for the dues they are paying. These SME owners distrust and resent the Chambers. The Chambers of Commerce, on the other hand, are said to be flush with membership dues, but fail to deliver services that could help firms gain access to new export markets or provide other forms of firm level assistance. Thus there is a large amount of dissatisfaction and mistrust for CoC. The Government of Serbia implements a relatively favorable income tax policy towards MSMEs. However, since MSME executives think that there is always the likelihood that Government may begin a new campaign for aggressive tax collection policy, current leniency is not well-appreciated. The team has heard from the SME executives interviewed that on short notice, GOS may begin closer monitoring and more regulation of SMEs. Such changes are likely to cause many SMEs to switch to the gray market to escape GOS scrutiny. As a result, however, they will lose an opportunity to reach their full potential. 13 Please see Case 20: SEDA, Novi Pazar. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 21 A business friendly taxation policy to encourage SME growth may be appropriate. USAID could consider enlisting a think tank to determine the tradeoffs in various entrepreneurship policies such as lowering tax rates or rebates to exporters that may be appropriate. Tax breaks to SMEs could be linked to the introduction of the rule of law and regulations for SMEs. The introduction of a legal framework for banking and non-bank financial institutions to enable SME access to multiple sources of finance would help incentivize the rapid expansion of a variety of new financial instruments and institutions to enable SME access to both equity and debt financing. This may also include innovative financial products for municipalities such as municipal bonds—general obligation or revenue bonds USAID could also consider using its existing programs to assist the GOS or its relevant agencies to promote SME finance with USAID support. SIEPA is already working as a quasi governmental organization to improve Serbian business community access to export markets and attract FDI. SEDP’s cooperation with this agency has been quite fruitful. SEDP provided assistance to SIEPA in gaining access to international markets, increasing export potential of Serbian manufacturing companies, and attract investment. USAID may consider continuing with these support services, that may include introduction of new financial instruments specifically custom tailored for Micro and SMEs, business incubators. And business advisory services to ensure sustainability and increased capacity. 4.3 Impact of Grant Disbursement on Employment Larger CRDA grants generated considerably more employment than smaller ones. Small grants were extremely effective as a tool for reviving the economic activities in all communities. Sharing the burden of business risk with matching grants approach is an effective way for micro and small businesses to value the contribution towards themselves. This would essentially work as an incentive for MSMEs that received grants to ensure their own success. Micro and Small businesses benefited relatively more from CRDA and SEDP grant mechanism than larger companies. The micro enterprises had no access to financial resources prior to the grant, while larger enterprises already had the accumulated capital and had some access to finance. On the other hand, it is very important to note that the larger, better enterprises had a greater chance to increase their sales potential and create additional jobs than the smaller ones. This is the leveraging effect of the grant mechanism on firms with a longer history of existence. 4.3.1 Grand Awarding Process Screening CRDA grantees through a competitive selection process ensured better economic outcomes. The grant application process, including preparation of business plans, helped people determine if they can actually make money at their new business practice. These factors played a major role in unleash the entrepreneurial potential in the citizens. The grant mechanism provided a major incentive in the re-emergence of a long-dormant entrepreneurship process in Serbia. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 22 In addition, tailoring the size of grants to the ability of recipients to create jobs, including those for minorities, was highly effective in generating employment. In development, “absorption capacity” is a factor that seems to be most often neglected. CRDA-E grants, however, were carefully structured so that SMEs could utilize them efficiently, and to organize, staff, manage, plan and control the operations of an entity. CRDA final reports usually indicate the criteria for beneficiary selection. For instance, ACDI/VOCA followed a set of very stringent criteria for awarding micro grants, which ensured success. To begin with, the entity had to be in a kind of business in grey economy or had gained certain skills through work experience, but did not have sufficient funding to start an individual business.14 Award criteria further required the entity to be registered at the National Employment Service; developing a plausible business plan; and providing a matching grant of at least 25% through in￾kind contribution. An important requirement was to register the business with the GOS before the delivery of equipment. Further, the entity had to stay in business and hire at least one or two workers with the two years following the grant. This was a top necessity to ensure sustainability of the business. 4.3.2 Leveraging Impact Leveraging job creation efforts through the Serbian National Employment Agency increased impact. CRDA IPs required that all grant applicants had to be registered with the local unemployment offices to be eligible for the micro or the small business grants. Through this passive collaboration with a GOS agency, CRDA ensured that each recipient was a legitimate business entity operating intra legally. This also helped revenue generation for the local and central government authorities through taxation. This practice not only helped create jobs and generate income, it helped the nation build a record of its private sector economic activities. CRDA also helped the National Employment Agency improve job placement rates. IRD assisted the Job Club in Valjevo with its Kick-Start program to encourage entrepreneurship. Later, case 35 notes that it provided the Job Club with computer equipment so that “Active Job Seekers” could search the National Agency’s database for jobs. 4.3.3 Efficiency of the Grant Mechanism Through the grant mechanism, Micro and SME businesses, and agricultural producers were able to obtain much needed equipment to start their business or agricultural operations which they would not have been able to get otherwise, using their own means. By “priming the pump” for economic activities, CRDA improved the lives of many Serbian citizens and created many sustainable jobs. Though grants were extremely useful in the economic environment when CRDA began they need to be replaced now with effective business financing methods. 4.4 Key Obstacles to Implementing Economic Development Projects in Serbia The following is a discussion of the main obstacles to economic development in Serbia. These same factors also hamper the implementation of economic development projects. 14 Please see CRDA Activity Final Reports, July 2007 of ACDI/VOCA, IRD, CHF, MC and, ADF. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 23 4.4.1 Government and Government Regulations There is an appalling lack of trust and confidence in central government by the Serbian business community. There is no expectation of government support or assistance to business, other than through targeted subsidies in selected industries, especially agribusiness. Business people say their best hope from government is to be left alone. Many would argue that this reputation is well deserved. There is a proliferation of laws and regulations affecting business, and it appears that Serbian industry is over-regulated. Furthermore, government tends to be heavy-handed in dealing with MSMEs. Underlying this problem is a weak system for enforcement of laws and regulations. For example, the national assembly in Belgrade may pass a law regulating a certain business tax but those enforcing the law are improperly trained and poorly informed, so enforcement by tax inspectors is spotty. This is particularly problematic at the local level, where capacity is very low. Another issue is government inspections. Inspectors have the power to close a business for the slightest infraction. During the assessment team’s visits to food processors, they complained that frequent food safety inspections were highly disruptive because the inspectors would close their processing lines while inspections were being carried out. In case of an infraction, the factory was temporarily closed. Customs procedures for imported goods, as well as exported products, are bureaucratic, cumbersome, and time consuming. As is the case for many developing countries, Serbia depends a great deal on import duties to supplement its revenues so heavy emphasis is placed on administrative procedures and customs valuation for imported items. This can lead to lengthy delays in receiving imported items. Exemption from customs duties is an investment incentive in Serbia, but the administrative cost to recover the exemption is high. Export procedures are also burdensome, since a considerable number of permits, inspections, and certificates are required. Government corruption is a hidden obstacle to economic development. Corruption is perceived as widespread. Serbia ranks 90th out of 163 countries in Transparency International's Corruption Perceptions Index for 2006. While corruption is rarely discussed openly, an environment of excessive regulations with spotty enforcement encourages corruption by government officials. A poorly paid food safety inspector with the power to close a factory for an alleged violation also has the power to extort a payment for letting the factory remain open. Customs is another potential area for corruption. Substantial documentation requirements at border crossings can be an incentive for an importer to provide an expediting payment to a customs officer. Recently, senior customs inspectors, including customs administration anti-smuggling coordinators were accused of being involved in schemes to falsely declare goods in transit through Serbia when, in real fact, they were unloaded in illegal warehouses and smuggled to companies and individuals without paying duties.15 15 Boris Begović and Boško Mijatović (editors) CORRUPTION IN SERBIA FIVE YEARS LATER Center for Liberal￾Democratic Studies, Belgrade, 2007 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 24 A separate, although related issue is the disproportionate power held by central government, in comparison with the municipalities. Local governments are weak, politically and economically, and they have very limited ability to develop programs to attract and encourage business investment and expansion. For example, the authority of the municipalities to tax local citizens and business entities is considerably limited, as is their ability to provide tax relief as a means for attracting business to the local communities. Municipalities only have the power to tax a small percentage of personal income, and to impose a business tax on registered businesses within the municipality. Municipalities have no power to obtain debt financing, without the prior approval of the Ministry of Finance. Approval is difficult to obtain, in view of National Government’s concern with debt management. Consequently, their ability to borrow money for the construction of facilities to attract business such as an industrial zone is considerably limited. Municipalities are also limited in their authority to lend money – or, at least, to collect interest. What some municipalities do to encourage investment is to facilitate bank loans for investors, and then pay the interest on behalf of the borrower. Another difficulty is land ownership: Municipalities cannot own land, since all municipal land is the property of National Government. However, national land that is not used by a national institution (e.g. military) can be leased by the municipalities for a period up to 100 years, for subsequent use by private business. These limitations on the authority of local municipalities to tax, obtain financing, and control land ownership considerably limits their abilities to provide financial and fiscal incentives for economic development. What municipalities can do most effectively is to expedite the issuance of licenses and permits related to business construction. This serves to reduce the legal and administrative difficulty of establishing a business facility. Limited sources of municipal income present a second obstacle to economic development arising from the skewed power structure between National and local governments. Not until the 2006 new law on local government finance did the National Government initiate routine transfers to local authorities. This law specifies that local governments will receive annual revenues amounting to 1.7% of gross domestic product. This change provides some degree of financial stability to the municipalities. Before this change, the amount of annual funding for local governments was uncertain, and often politically motivated. As a result, local governments had to develop their own sources of revenue, which in some cases, has become a heavy burden on local business. For example, fees on the development of urban land are a huge cost to developers. Construction permits and licenses are extremely costly, and account for 10 percent of municipal revenue. The connection fees for public utilities – water, electricity, and sewer – are also high. These fees and assessments substantially increase the cost of urban development and construction in Serbia, and are a disincentive to economic development. One bright spot in this picture is the progress the Serbian government has made in facilitating new business registration. A Business Registration Law was passed in 2004, and an Agency for Business Registers began operating in 2005. Registering a business has become an Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 25 administrative function rather than a court procedure, and has become easier and faster. As a result, there has been a tangible increase in the growth of business registrations. Equally, the time required for tax authorities to allocate tax registration numbers to new businesses has been cut from seven to three days. 4.4.2 The Judiciary In a survey carried out in 2005 in Serbia by the EBRD and the World Bank16, 52% of respondents named the judiciary as the greatest impediment to doing business in Serbia.20 The Serbian judiciary is viewed as being unduly influenced by the executive branch of power and is incapable implementing laws and resolving disputes impartially. Government officials and state employees exert pressure on the courts, public prosecutors, and try to influence the results of important (political) judicial trials. The judiciary is also perceived as being inefficient and corrupt. Serbian courts have large backlogs of cases. In some courts, each judge handles an average of one thousand cases per year. With 128 new cases arriving for each judge’s consideration in a year, substantial backlogs are inevitable. Serious delays in trial or resolution (more than a quarter of cases are extended into the next year) represent a strong incentive for agents to use extra-legal means in enforcing their rights. 4.4.3 Lack of a Business Support Network To its credit, in recent years the GOS has made a concerted effort to stimulate SME development. In 2001 the government established the Council for SMEs and Entrepreneurship, which has now grown to a network that includes 14 regional centers in Serbia’s major population centers. These centers provide business information, training and advice to SMEs and entrepreneurs, and support the local municipalities in their effort to implement local economic development policy. A second institution supporting economic development in Serbia is the Serbian Chamber of Commerce, whose history began when the Prince of Serbia founded the National Trade Board in 1857. However exemplary are government’s attempts at SME development, unfortunately its efforts have fallen short. The SME Council is poorly funded, poorly staffed, and its effectiveness is limited. The Chamber of Commerce is politicized, bureaucratic, and reflects government’s views more than those of private business. There is a compulsory requirement for all private businesses to become member of the Chamber. However, there is significant dissatisfaction within the business community of the level of services the Chamber provides its constituents. This dissatisfaction severely limits its effectiveness as a voice for the business community. Presently, there is no effective support network available to assist Serbian MSMEs. Entrepreneurs must work entirely on their own for business creation and expansion. What is needed are industry-level, private, member-based service organizations that serve SMEs within each industry. These could provide access to information and technology, provide a forum for the exchange of ideas and experiences, and link their members to external markets. This limitation provides a considerable opportunity for donor intervention to strengthen the business support network in Serbia. 16 EBRD-World Bank Business and Enterprise Environment Survey (BEEPS) http://siteresources.worldbank.org/INTECAREGTOPANTCOR/Resources/BAAGREV20060208SAM.pdf Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 26 4.4.4 Limited Availability of MSME Finance The lack of access to finance is a major impediment to business activity for MSMEs in Serbia. Most micro and SMEs are chronically under capitalized. Rural credit, particularly in southern Serbia is constrained by structural limitations within the banking sector. While government￾provided credit is available to farmers and processors, the amount of funds actually available is insufficient to meet the needs of the agricultural sector. The private banking system in Serbia is highly liquid, and banks are swollen with funds. At the same time, hundreds of thousands of MSMEs are unable to obtain credit. Distortions in the banking sector and a poor credit environment are responsible for the mis-match between lenders and borrowers. Real interest rates on credit are high, making the cost of borrowing prohibitively expensive for many MSME. There is a three-tiered schedule of interest rates, with individual borrowers paying the highest rate, large companies paying the lowest, and MSMEs in between. The annual interest rate for MSMEs is around 15%. The cost of MSME capital seems excessive, considering that debt is evaluated in Euros. Cases 11 and 14 indicate the frequent denomination of loans in Euros and the distrust this practice causes when loans are “marked to the market.” However, high interest rates are an indicator of high risk. In Serbia, much of the risk is external to the banking sector, and can be explained as high political risk. Many believe that high political risk is related to Serbia’s difficulties such with political unrest, unsolved problems with neighboring states, and the need for collaboration with the Hague Tribunal. High risk is also related to internal problems, such as the inefficient court and justice system. In addition, Serbia is identified as a country with weak financial systems, such as a poorly developed public and private credit reporting systems. An efficient reporting system would help creditors diminish risk and allocate credit more efficiently by assessing potential borrowers and providing information on their financial viability. Another factor is the difficulty for creditors to ensure the return of debt collateral in case of default. The legal system is weak, the land registry is incomplete, and bankruptcy procedures typically last several years. Finally, regulations on secured lending, such as collateral and bankruptcy laws, are inadequately implemented and applied. Within the financial sector, structural problems limit MSME’s access to credit. Bank lending is often collateral-based, with required coverage amounting to several times the loan value. There also appears to be a cultural aversion to credit by MSMEs, undoubtedly the result of their deep distrust of the banking system. Borrowers need training and TA in formulating business plans and project concepts, while bank officials need to learn more about MSME financing, particularly for agricultural enterprises, and its risks. Ideally, bank lending officers would expand use of cash flows as the principal lending criterion. 4.4.5 Business Management Skills Management skills are generally deficient within Serbian MSMEs. Neither CRDA nor SEDP was designed to meet these needs. In the northern parts of Serbia, management skills are much better developed for a competitive, export oriented production and growth potential than the south and south-eastern regions of the country. Strengthening executive management and Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 27 marketing skills would be an extremely important contribution if MSMEs are to grow and prosper. In family businesses, top management positions in companies are often assigned to relatives, as opposed to professional managers selected on the merits of their qualifications. Consequently, growth is typically limited to the number of sons or daughters who are available to mange the business. Furthermore, there is a general reluctance to delegate duties to subordinates, which means that top management controls most decision-making. People-management skills are limited, with very little invested by MSMEs in employee training and advancement. This perception of human resources, when practiced by companies in need of specifically skilled or highly qualified staff, represents an obstacle to retaining valuable employees. Completing and publishing financial reports is considered to be a legal obligation, and not a fundamental source of information for managing a firm. MSMEs believe that Serbian business administration and management schools do not meet quality standards, and that management education is inadequate for business needs. 4.4.6 Inflexible Labor Markets Serbia's labor laws have made its labor force less flexible than in neighboring countries. This is undoubtedly the result of government's efforts to provide a safety net to workers made redundant when State firms are privatized. The new Labor Law, enacted in February 2005 imposes administrative barriers to worker layoffs and separations in order to slow the growth in unemployment. Employers must give a formal warning with precise reasons for being dissatisfied; inform the labor union about the employer's intentions, and offer the employee alternative employment. Also, the law establishes an Emergency Fund for workers who lose their jobs due to bankruptcy. Employers must pay supplementary taxes to finance the Fund, which makes the workforce more expensive. This also slows restructuring, competitiveness of firms, and economic development. 4.4.7 Non-Competitiveness of Serbian Businesses in World Markets In 2005, the World Economic Forum rated Serbia in 80th place of 121 countries included in its Global Competitiveness Index,17 up from 89th place in 2004. While there is no direct relationship between Serbia’s competitiveness rating and the competitiveness of Serbian MSMEs, this global rating does reflect Serbia’s lack of international competitiveness. The real indicator of competitiveness is the ability of domestic firms to compete on international markets. Given Serbia’s high, negative balance of trade, Serbian firms overall are performing very poorly in this respect. For Serbia businesses, a fundamental obstacle to establishing global competitiveness over the long term is that its main export sectors are dominated by primary products with low value added. Few Serbian MSMEs are able to compete in world markets, outside former Yugoslavia. There are several reasons for this weakness. Marketing in Serbia is underdeveloped and inadequate, and not sufficiently sophisticated. Even the very concept of marketing is very poorly understood, and focuses on mere promotion of goods. Selling primarily into local markets encourages 17 According to the World Economic Forum, the Global Competitiveness Index “measures the set of institutions, policies, and factors that set the sustainable current and medium-term levels of economic prosperity.” Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 28 enterprises to compete based primarily on price, whereas competition in export markets is generally based first on quality, and then price. Productivity of Serbian enterprises is low compared to global competitors, mainly due to the limited use of technology and weak management skills. Business processes - administrative procedures, financial controls, accounting, and financial reporting- are weak, primarily as a result of weak management. There is a strong need for improved business skills and business restructuring. In the agricultural sector, with fruit and corn being exceptions, Serbia’s farmers are generally non-competitive in world markets. Due to high levels of protection and subsidies, farmers producing agricultural commodities for domestic markets are significantly more profitable than they would be if they produced for export markets. While this situation is sustainable as long as the domestic market is protected from international competition, Serbia has set a goal of entering the World Trade organization (WTO) in 2008. When Serbia enters this international organization, borders will eventually be opened and inefficient farmers will be forced to compete with cheap imported agricultural commodities. Serbia's farmers have a "subsidy mentality" and respond to government incentives rather than market signals. For example, Serbian farmers receive government subsidies for fertilizer, milk production, below-market credit for working capital, and grants for on-farm storage. 18 Farmers are currently shielded from external competition by high import tariffs (e.g., 30% tariff on corn and wheat; even higher for dairy and pork) as well as by non-tariff barriers such as burdensome licensing procedures and high charges for border inspections. Although government is slowly phasing out most agricultural subsidies, their application has created a mind-set of dependence on government. Farm productivity is low, primarily due to the limited use of technology and small-scale production. Many Serbian farms, especially those outside the relatively productive and wealthy Vojvodina region, practice "low input-low output" farming with a large share of production consumed on-farm, and excess production sold into local markets. They make limited use of inputs such as fertilizers, farm chemicals and premixed feed, and typically do not have the scale of production required to adopt agricultural technologies such as refrigerated milk storage, grain storage, and mechanized tilling. Nor can they afford technological innovations such as hybrid seed, improved animal genetics, and better animal nutrition. Grain marketing costs are high due to limited availability of drying and storage facilities. Many corn producers have limited access to affordable grain drying facilities, and consequently are forced to sell their corn at harvest with high moisture content, often to regional trading monopolies, who assume the drying costs. This translates into lower farm gate prices. High drying costs could be reduced through the use of earlier maturing varieties and investing in on￾farm storage. The marketability of wheat is also reduced due to limited drying facilities. Because it is not dried, Serbian wheat has a relatively low baking quality and is exported mainly as animal feed. 18 Author’s field notes Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 29 Only a small percentage of Serbia’s 700,000 farms are operated as commercial enterprises. Farm area covers approximately 88 percent of the available agricultural land. Of this total, some 600,000 are small family-owned farms less than 5 hectares in size that produce mainly for self￾consumption, with only excess production sold into local markets. The remaining 100,000 farms, or 15% of the total, are commercially-oriented private farms. These tend to be family farms of approximately 10 hectares in size, engaged in mixed farming of high value crops such as berries, orchards, vineyards and animal production, located largely in central Serbia. In Vojvodina, there is a higher concentration of larger farms. These include large, socially owned agro-kombinats and cooperatives as well as an emerging class of larger private farms that have emerged from privatization. The large, private farms cultivate grain crops such as corn, wheat, and industrial crops such as sugar beet for export markets. Some smallholders have started to move into commercial farming while others are downsizing and moving into subsistence farming or entirely out of agriculture. However, the consolidation process is exceedingly slow. Speedy land consolidation is held back by under-developed agricultural land markets and land lease markets, which are constrained by an inadequate legal framework for land titling and land ownership. Mr. Tomic discusses some of the weaknesses n land ownership policies in relation to his desire to expand fruit production in case 10. To develop functioning markets for land and land leases, improvements must first be made in immovable property registries, land titles, and the adoption of mechanisms for land marketing. Substantial farm consolidation leading to economies of scale cannot occur until these improvements are made, Although the CRDA implementing partners made good progress in establishing and strengthening farmers' associations and cooperative, much still needs to be done. These organizations are emerging as a market force and have the potential to greatly improve farm income through their ability to negotiate for better prices, by reduce input costs through bulk purchasing and VAT recovery, and by product aggregation for efficient marketing. In other words, producer associations have the same effect as land consolidation. Greater efforts are needed to organize small-scale farmers into producer associations and cooperatives. There are many shortcomings in Serbia's agricultural extension services. Possibly the greatest shortcoming has been the exclusive focus on crop and livestock production. Extension services should also assist farmers and agro-processors adjust to a less protected economic environment with fewer subsidies. They should help to improve farm, as well as factory efficiency and to adopt the international standards for food safety and quality required to become competitive in both domestic and export markets. 4.5 Contribution of CRDA/SLGRP to Citizen Participation The assessment team found that citizen participation is a question of attitudes and perceptions, and it should not be surprising that these aspects vary depending on who is being interviewed and what is happening politically in the country as the interview or focus group discussion occurs. Research conducted by the assessment team confirmed that CRDA and SLGRP provided the tools and systems needed to increase citizen participation, improve municipal effectiveness, and bolster local economies, and that the impact was substantial across all three objectives. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 30 USAID helped to lay the foundation, but permanent improvements in the quality and extent of citizen participation in Serbia will require a national commitment to build upon it. The contributions of each of the programs to citizen participation are discussed in the following sections. 4.5.1 Contributions of CRDA to Citizen Participation 4.5.1.1 Impact on Citizen Empowerment Interviews with 61 individual CRDA beneficiaries confirmed that community mobilization activities carried out across all five AORs had been a revolutionary and uplifting experience for most. Almost universally, participants said they had not realized their power as ordinary citizens to effect change locally and influence decision-making in their municipalities. This was particularly true of citizens interviewed in the smaller towns, where modest but vital improvements in water systems, schools, roads, and other basic infrastructure had been selected and overseen by local councils working in a highly participatory fashion. The projects were augmented with local matching contributions, often exceeding the size of the USAID grant, and implemented with the assistance of CRDA implementing partners. As shown in the tables beginning on the page 36 of the annex, fully 51 of those interviewed19 felt they had influenced municipal decisions in ways that improved their communities tangibly, and 47 percent said they continued to have such influence. The vast majority of stakeholders interviewed—58—said they were more involved in community affairs than before CRDA began. The team observed that the citizen committees had for the most part ceased to function, through some had refashioned themselves into not-for-profit interest groups. Building on skills they had acquired through participation in CRDA, some former CDC members had established nascent consulting firms and were providing fee-based services to municipalities and external donors such as the United National Development Program and the European Union. Many CDC members voiced disappointment that CRDA had ended before citizen participation systems and practices could fully take root in their municipalities. Others complained, sometimes bitterly, that a changeover of political party in their municipality had effectively frozen them out of the discussion with municipal authorities. For these individuals, at least, meaningful participation in municipal affairs seems to have ended. 4.5.1.2 Impact on Quality of Life CRDA was a major contributor to economic growth in Serbian towns and villages at a time when such assistance was vitally needed. The assessment team saw many examples of such projects, including refurbished schools, tarred roads, and enterprises that had either been established or strengthened as a result of new equipment grants and other types of assistance. Fully 85 percent of respondents to the survey of individual stakeholders felt the business environment had improved as a result of the project. Moreover, by involving local people in the vetting of these 19 Percentages were calculated using the sum of the values in the first two columns as the numerator, and the total in the row as the denominator. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 31 projects, 90 percent said they thought living and working conditions had improved in their town as a direct result of increased citizen participation. The primary impetus for citizen participation in this case was the presence of funding and technical assistance provided by an external donor, a situation that no longer obtains. However, individual interviewees almost uniformly affirmed that the CRDA experience had been a galvanizing one for local communities—one that had not only opened their eyes to the possibilities of mobilization but also gave them the skills needed to approach other funders in the future, including their own municipalities. 4.5.1.3 Impact on Citizen Relations with Local Government The assessment team found that program beneficiaries were more satisfied with the services they receive from municipal government than before CRDA began, largely as a result of (1) the citizen assistance centers (CACs) and one-stop permitting centers (OSPCs) established under SLGRP and (2) the initiation of more transparent budget systems and holding of open budget hearings. Three-quarters of citizens responding to the questionnaire said they were aware of or had personally benefited from a citizen assistance center or one-stop permitting center set up under the CRDA project, and 69 percent said that municipal service delivery had improved in their town in the past 12-24 months. The team found a rather large discrepancy in perception between citizen beneficiaries and municipal officials about the degree to which the budget process was open and transparent, however. Although 85 percent of municipal officials interviewed said their town conducted annual budget hearings, only 52 percent of citizen stakeholders filling out our questionnaire could confirm this. Most likely this is only a difference in perception, as the municipal officials were in a strong position to know—and were certifying it in writing—while citizen respondents were only expressing what they felt to be the case. It is telling, however, that fewer than half of the individuals interviewed thought citizen initiatives were being included in their municipality’s annual capital investment plan versus 90 percent of municipal officials who said they could provide documentation showing they were. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 32 Questionnaire C-1 CRDA Questionnaire for Individuals Summary of Results Strongly Agree Agree Don't Know/No Opinion Disagree Strongly Disagree N/A 1. I have influenced municipal decisions in ways that improved our community tangibly. 22 30 5 3 1 0 2. I am more involved in community affairs than I was before the CRDA project began. 33 24 1 2 1 0 3. I continue to influence municipal priorities in my town. 23 23 6 5 2 1 4. Systems are in place to ensure that citizens can participate meaningfully and regularly in municipal affairs. 8 27 4 11 11 0 5. Living and working conditions in my town have improved as a result of improved citizen participation. 16 39 2 4 0 0 6. Systems and procedures for including ordinary citizens in municipal priority-setting still need strengthening. 44 15 1 1 0 0 7. My municipality conducts annual public budget hearings at which input from ordinary citizens is solicited. 12 20 12 6 9 2 8. I think my municipality does a good job encouraging citizens and community groups to attend community-wide events such as budget hearings. 13 18 6 11 12 1 9. Municipal service delivery has improved in my town in the past 12-24 months. 10 32 9 6 4 0 10. My town is more independent of the central government than it was before the CRDA project. 9 22 9 16 5 0 11. The business environment in my municipality has improved as a result of the CRDA project. 13 39 8 1 0 0 12. Citizen initiatives are included in my municipality's annual capital investment plan. 5 25 19 4 3 5 13. I am aware of or have benefited from a Citizen Assistance Center (CAC) or One-Stop Permitting Center (OSPC) set up under the CRDA project. 26 20 7 0 0 8 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 33 There were other, more serious discrepancies in perception. For example, only 57 percent of CRDA beneficiaries interviewed felt that systems were in place to ensure that ordinary citizens could participate meaningfully and regularly in municipal affairs. And, although every single official interviewed said their municipality did a good job reaching out to both urban and rural MZs and encouraging them to attend community-wide public meetings such as budget hearings, only 51 percent of ordinary citizens agreed. Indeed, an overwhelming proportion of citizen respondents—97 percent—said they thought systems and procedures for including ordinary citizens in municipal priority-setting still needed strengthening. Thus, there remains a fairly strong gap in perception about whether and how much additional work is needed to ensure effective citizen participation in the municipalities where CRDA operated. 4.5.1.4 Impact on Decentralization The team sought to gauge how local citizens felt about the degree to which CRDA and SLGRP had increased the power of their municipality relative to that of the central government in Belgrade. This is a complex issue that is addressed more fully in the section below on local governance strengthening, but it bears mentioning here that only slightly more than half of citizen stakeholders interviewed by the team (51 percent) felt their town was more independent of the central government than it was before CRDA. 4.5.1.5 Sustainability As noted previously, citizen participation activities and successes that were the hallmark of CRDA would not have happened without the presence and largess of an external donor. In town after town the team heard from project stakeholders that citizens had been leery of the process until they had begun to see tangible benefits. Once the process was underway, CDCs performed brilliantly as a grassroots mechanism for securing action on local priorities. Some of the CDCs reestablished themselves as public interest groups after project funding ended, but most seem to have simply faded away (case 25). USAID staff confirmed to the assessment team that it never intended that the CDCs would become permanent structures. Nonetheless, it is fair to ask whether the process of citizen participation set into motion through the CDCs is sustainable. The answer to this question is two￾fold: • First, at an institutional level, many of the NGOs that were spun off from this process, though not community committees in the pure sense are carrying on the tradition serving as watchdogs and pressure groups to lobby municipal governments for special interests such as the environment, or on behalf of handicapped individuals with special needs. This is an important and very tangible legacy of the CRDA program. • Second, at the attitudinal level, the team repeatedly heard program beneficiaries say that the very concept of citizen participation was so revolutionary (and distinct from the experience under socialism), that the impact has been lasting, at least in areas where the project was active. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 34 4.5.2 Contribution of SLGP to Citizen Participation SLGRP sought to strengthen both the institutional and operational capacities of municipalities while fostering changes in the policy environment to speed devolution of decision-making authority to the local level. An underlying objective—and the focus of this part of the assessment—was to build and strengthen two-way communications between municipalities and the citizens they serve. Over the course of approximately three weeks in the field the team met with dozens of local government officials, including mayors, finance directors, and heads of strategic planning departments to review project achievements and assess the degree to which systems had been out in place to foster citizen participation. Information was collected both through open-ended discussions and through use of a 16-part questionnaire. Municipal officers universally praised the assistance received, though some instances there were minor complaints about training and software procurement.20 In general, the team concluded that the assistance had been highly effective. USAID focused initially on opposition municipalities that had systematically been starved for resources by the central government, so marginal impact was high. Assistance included training for municipal officials, computerization of information systems, improvements in public procurement procedures, the establishment of modern and efficient citizen assistance centers (CACs) and one￾stop permitting centers (OSPCs), and systems to increase transparency and public participation in municipal budget-setting. SLGRP also collaborated with CRDA to establish business improvement districts (BIDs) in the municipalities of Zrenjanin, Valjevo, Nis, and Krusevac. 4.5.2.1 Impact on Municipal Effectiveness The impact of USAID’s assistance on municipal effectiveness was substantial, and carefully documented by DAI through use of a municipal capacity index (MCI). After first establishing a baseline for each city, the index traced growth in five areas: financial management, municipal systems, communal enterprise, transparency, and citizen services. Figures 4-1 through 4-3 from DAI’s final report show the combined growth in capabilities for these five areas across three groups of municipalities.21 As observed in the report, the longer the individual group of municipalities (cohort) was in the program, the greater the discrepancy between it and a control group comprised of non-participating municipalities (compare the charts for Cohort 1 against those for Cohorts 2 and 3 for example, which joined the program later). Cohorts and controls need to be explained. The assessment team saw and heard of many examples of tangible improvements to municipal operations in interviews with local officials, and physically reviewed a number of municipal 20 In Loznica and Novi Pazar, municipal officials complained that the U.S. implementing partner had insisted on the procurement of a particular software package known as “FINOVA” that ended up not being usable. Later these two municipalities purchased appropriate software using their own financial resources. A former mayor in Novi Pazar said she felt that staff recruited by the U.S. implementing partner to conduct training were ill-prepared and under￾qualified for the job; the great majority of municipal officials we spoke to lauded the training provided under SLGRP, however. 21 Serbia Local Government Reform Program Final Report, Development Alternatives, Inc. October 2006. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 35 planning documents. The team also visited at least five community assistance centers to witness their use by ordinary citizens. SLGRP Cohort 1 MCI Scores 12.9 24.0 33.6 51.3 56.8 57.5 57.9 11.3 13.8 15.8 20.0 23.3 23.7 24.7 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 Sep-02 (Baseline) Apr-03 Sep-03 Mar-04 Sep-04 Mar-05 Sep-05 SLGRP Cohort 1 Control Municipalities Figure 4-1: SLGRP Cohort 1 MCI Scores SLGRP Cohort 2 MCI Scores 15.1 21.5 28.5 40.1 42.9 45.5 12.0 14.0 19.5 19.5 21.0 24.3 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Apr-03 (Baseline) Sep-03 Mar-04 Sep-04 Mar-05 Sep-05 SLGRP Cohort 2 Control Municipalities Figure 4-2: SLGRP Cohort 1 MCI Scores Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 36 SLGRP Cohort 3 MCI Scores 15.3 21.9 28.6 36.7 17.5 20.8 23.3 25.3 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 Apr-04 (Baseline) Sep-04 Mar-05 Sep-05 SLGRP Cohort 3 Control Municipalities Figure 4-3: SLGRP Cohort 3 MCI Scores SLGRP also made major contributions at the policy level, helping to shape and speed passage of laws to foster decentralization, starting with the 2006 Law on Tax Sharing and Municipal Transfers, which helped improve basic fairness in the revenue sharing system. This was followed by the Intergovernmental Finance Law, which not only brought increased revenue sharing but also enabled municipalities to borrow from commercial banks for the first time. 4.5.2.2 Impact on Citizen Participation SLGRP worked not only to build capacity and enhance the professionalism of municipal staff but also to enable increased citizen participation through mechanisms such as open budget hearings and the establishment of citizen assistance centers.22 The assessment team did not systematically collect and analyze primary budget data but did ask municipal officers a range of very specific questions about the budget process and their efforts generally to include citizens in municipal decision-making. As shown in the summary tables for Questionnaire C-2 on page 43, the overwhelming majority number affirmed that they not only continue to conduct annual budget hearings but also make special efforts to encourage attendance by ordinary citizens. Again, the team found discrepancies between the perceptions of municipal officials participating in SLGRP and those of individual citizens benefiting from various kinds of CRDA grants. While not doubting the written assertions of municipal staff, it is noteworthy that a significant proportion of respondents to the questionnaire for individuals (see previous discussion of Questionnaire C-1) were unable to confirm that open hearings were a feature of their municipality’s budget process, or that their city did a good job in reaching out to the public to foster publication. The assessment team concludes this is most likely a function of lack of familiarity and/or interest by individual citizens with the mechanics of municipal management, possibly combined with a 22 There is no law requiring the holding of public municipal budget hearings in Serbia, but all of the municipalities participating in SLGRP agreed to do. Some even passed local ordinances requiring them. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 37 failure of municipalities to advertise the process more broadly. Indeed, in Tutin and some other towns, mayors said they wished ordinary citizens would show more interest in the municipal budget process. The team saw a number of different approaches to budget hearings. In the Municipality of Paracin, for example, open budget hearings had become so much the norm that the mayor felt it necessary from a political standpoint to attend each one personally, especially in the smaller, rural MZs making up the municipality. In larger cities like Krusevac, by contrast, it appeared citizen input was being solicited principally through mass media, for example through questionnaires inserted in local newspapers, where citizens can vote either for or against individual projects. In the dozens of interviews conducted by the team conducted across Serbia, and as documented in the citizen and municipal official questionnaires, it was abundantly clear that CRDA helped citizens mobilize their energies, identify priorities, and engage their municipalities in a joint effort to upgrade local facilities and boost incomes. Likewise, by increasing transparency, improving service delivery, and including ordinary citizens in budget formulation and review processes, SLGRP helped cement the notion that municipalities can respond to citizen needs effectively. For project beneficiaries, therefore, it can be said that a solid “bridge” was built through the mutually reinforcing and sometimes joint activities of CRDP and SLGRP. Unfortunately it is not possible to extrapolate these findings on citizen participation to the Serbian population more broadly. The Serbian context has been a dynamic one since CRDA and SLGRP began in the early 2000s, beginning with humiliation and resentment of the population over the NATO invention, moving into hope and encouragement with the fall of Milosovic and the ramping up with assistance from the international donor community. Recovery, however, was followed by disillusion with the Serbian Democratic Party, and, finally, the loss of Kosovo just days before the assessment team completed its fieldwork in Serbia. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 38 Questionnaire C-2 CRDA Questionnaire for Municipal Officials Summary of Responses Strongly Agree Agree Don't Know/No Opinion Disagree Strongly Disagree N/A 1. This municipality conducts annual budget hearings to which the public is invited and given ample opportunity to provide input. 15 3 0 2 0 1 2. This municipality reaches out to all mesna zajednicas, both urban and rural, and encourages them to attend community-wide public meetings. 16 5 0 0 0 0 3. This municipality reaches out to community and special interest groups and encourages them to attend community-wide public meetings. 12 6 2 1 0 0 4. Voluntary attendance records are kept at annual budget hearings held by this municipality. 7 8 3 2 0 1 5. Citizen views expressed at the hearings are recorded and factored into the annual budget deliberations of my municipality. 10 5 6 0 0 0 6. The mayor of this municipality prepares an annual budget review letter and includes it when the budget is distributed through public media outlets. 11 3 2 4 0 1 7. This municipality continues to conduct community-wide public meetings to gather citizen input for the capital improvement plan. This includes outreach to all of the mesna zadjenicas throughout the municipalities, both urban and rural, as well as community and special interest groups. 12 6 1 2 0 0 8. This municipality includes citizen initiatives in the annual capital improvement plan. 10 8 1 2 0 0 9. The annual capital improvement plans shows clearly which citizen initiatives have been included and which have been excluded. 4 9 3 5 0 0 10. This municipality organizes one or more public events to present the annual budget to the public. 14 3 0 3 0 1 11. My municipality has a policy or ordinance requiring that annual public hearings be held in advance of final adoption of the budget by the municipality. 12 4 1 3 0 1 12. My municipality has a policy or ordinance requiring that community-wide public meetings be held before the annual capital investment plan is prepared for inclusion in the draft annual budget. 7 6 2 2 1 2 13. I can provide you with documents showing that citizens, mesna 8 9 1 2 0 0 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 39 Questionnaire C-2 CRDA Questionnaire for Municipal Officials Summary of Responses Strongly Agree Agree Don't Know/No Opinion Disagree Strongly Disagree N/A zajednicas, community groups, and other special interest groups are being provided an opportunity to be heard at public hearings and other such meetings. 14. I can provide you with documents from the 2007 and (if available) 2008 current budget verifying that citizens, mesna zajednicas, community groups, and other special interest groups are being provided an opportunity to be heard at public hearings and other such public information meetings. 11 6 1 1 1 1 15. I can provide you with documentation showing clearly how much this municipality included in citizen initiative-requested funding in the 2007 budget. 11 8 0 0 2 0 16. I can provide you with documentation showing clearly how much funding this municipality has been requested for citizen initiatives for 2008 and has included in the budget. 10 8 0 1 2 0 Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 40 While the assessment team was conducting its interviews in the field, a Belgrade-based marketing research firm, Strategic Marketing, conducted focus group discussions on behalf of USAID in cities across Serbia. Strategic Marketing’s task was to assess the general outlooks of ordinary citizens, including many who had benefited directly from CRDA and SLGRP.23 The firm’s results supported the assessment team’s findings in some areas, but not all. On the positive side, participants commented favorably on improvements in water systems, schools roads, sewerage, transformer stations, telephone lines, municipal offices, and other local infrastructure and conveyed a sense that municipal institutions had improved in the quality and speed of local service delivery. With regard to citizen participation, however, the focus groups painted a very mixed mixture, including: • Ordinary citizens have little influenced on the decision-making and activities of local authorities; • Citizens general lack awareness that public budget hearings are taking place (tempered by the belief that information can be obtained if sought); and • Citizen impression that public discussion of the budget is conducted just for form’s sake, and that citizens have little possibility of affecting those decisions. 4.5.3 Suggested Additional Work CRDA and SLGRP built a solid foundation for two-way communication between municipalities and ordinary citizens, but additional efforts are needed to strengthen the process. Nascent public interest groups arising from the community groups established under CRDA, for example, need nurturing to survive and thrive. Technical assistance and training in the proper roles, organization, and functioning of these young NGOs would greatly improve the chances of continued success in lobbying their municipalities effectively to bring change from the bottom up. Assistance to local public media would also bring more responsible journalism and effective coverage of local issues, thereby further reinforcing the pressure on municipal governments to maintain transparency and accountability to their constituents. From interviews with municipal officials, the team observed genuine a commitment to new systems and procedures set up under SLGRP, though this was tempered in some cases with a feeling that they “know best” when it comes to deciding how to provide basic services to citizens and choose among competing spending priorities. Municipal officials also evinced a level of frustration over the perceived dominance and heavy-handedness of the central government. The basic systems for democratic participation seem to be in place, however, and it remains only for Serbia’s mayors to continue implementing and reinforcing new systems and procedures set up under SLGRP. These steps are envisioned as part of the Municipal Economic Growth Activity (MEGA), a second-generation program inaugurated by USAID/Serbia in 2006. 4.5.4 Monitoring and Evaluation The assessment team took a close look at CRDA’s monitoring evaluation system Web-PRS and concluded that it served it purpose of tracking implementation progress. Indeed, the program was 23 Strategic Marketing Research, Evaluation of CRDA Projects, February 2008. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 41 deemed successful enough in these two areas that USAID has since replicated it in other countries. The team could not find instances in which Web-PRS served as a decision-making tool or led to any significant mid-course corrections on implementation, however. The monitoring and evaluation systems for all three programs are discussed in detail below in the Section 5 of this report. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 42 5. FINDINGS FOR ADDITIONAL AREAS OF INTEREST 5.1 Monitoring and Evaluation Impact measurement is a challenge in all development activities. When an M&E approach is configured for post-conflict conditions, the barriers are even more pronounced. During the period evaluated USAID/Belgrade undertook large, complex projects with thousands of sub-activities and multiple indicators, both quantitative and qualitative. Capturing program impact appears to have been a constant struggle for all three programs due to the dynamic nature of operating in Serbia during the performance period. Meeting reporting requirements for such significant measurement meant each program implementer spent considerable time, effort and money collecting data to demonstrate causality. Prior to deployment, the Team reviewed the Mission provided data on CD-Rom and the available secondary sources such as CEVES. Once in Serbia the Team began validation of the M&E methodology, the indicators, the data collected for each program, and the usefulness of these data for evaluation. It is important to note that due to the dynamic post-conflict operating environment in which development programs are implemented, it is often only with hindsight that we can identify and reflect upon issues that could have been approached more effectively. 5.1.1 Review and Critique of Indicators and Performance Monitoring for CRDA, SEDP and SLGRP This section provides a review and critique of the monitoring performance indicators for each of the three programs. CRDA Within the complex operational context that USAID/Belgrade and the five IPs were working, tremendous effort was devoted to M&E activities. In general, the Assessment Team findings show that: • Implementers appear to have operated within the guidelines established by USAID/Belgrade’s defined framework and the parameters established by the Mission in consultation with other implementers. • The Mission’s effort devoted to monitoring activities was comprehensive and employed an extensive network of field offices. Each of the 5,000 projects was reported to have been visited multiple times to confirm as much as possible, faithful delivery and use within specified terms to beneficiary communities, organizations, and individuals. This effort perhaps exceeds the standards established within USAID’s operations worldwide. • USAID and some implementers corroborated progress though independent market research/survey data firms. Such efforts might be the most useful data for evaluation of a program of this size and scope. Initial review of CRDA’s indicators show that they: • Were designed to track the broad development progress of the multiple communities that CRDA served; Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 43 • Were designed to capture performance of the program during real-time implementation; and • Appear to have been designed to be logistically easy to implement in order to conserve program resources devoted to M&E activities. In overall terms, both the mission and implementers deserve recognition for the effort and seriousness in which they approached program monitoring. Nevertheless, the Team identified several problems with indicator measurement, attribution of benefits, and other issues that impeded the usefulness of indicators in program evaluation. SEDP The implementer appears to have made a serious effort to capture program impact, cautiously approaching overall results and repeatedly keeping in contact with mission staff to verify, agree upon, or revisit assumptions made in the calculation of their indicators. This was important in light of numerous shifts in programmatic approach. Specifically, the CTO, SEDP staff and leadership deserve recognition for: • Creativity and analytical rigor in attempting to derive attribution and program impact; • Making special effort to focus on the level of attribution and having a clear desire to be conservative in the impact assumptions being made; • Making additional efforts to directly track performance of constituent firms within their database system, rather than purely assuming jobs or income as a result of the type of project; • Attempting to link their program and results to external statistical data in order to have externally sourced data with which to estimate impact; and • Developing a database that makes a strong effort to collect and organize information valuable for conducting an assessment of project performance. However, as with most programs, issues arise in an ex-post evaluation that challenge the usefulness of indicators, the way they were calculated, their reasonability and the level of impact and attribution. . Nevertheless they do tend to highlight general problems in monitoring and evaluating development programs all over the world. For SEDP indicator performance appears to be at least somewhat overstated. Possible reasons for the overstatements include: • In agreeing upon indicators, SEDP and the Mission staff struggled with calculating job creation. However, as is evidenced by the differing methodologies used to calculate job creation between SEDP and CRDA (and, even between the 5 CRDA partners) there is no universally accepted methodology for assessing job creation or income generation within USAID. This is not simply an issue for the USAID Serbia Mission. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 44 • SEDP attribution levels are an attempt to limit SEDP’s claim over impact in terms of direct export and job creation. However, the SEDP firm-level attribution factor was primarily subjective and was typically the responsibility of the sector leader – the individual that would presumably have the most incentive to push attribution levels to their highest. Ideally, an independent observer could calculate/oversee levels of attribution. • The export and job creation numbers attributed under the program appear overly large in relation to SEDP’s size. SEDP was a relatively modest program in size, scope and budget of about USD 14 million. However, the Final Report presents sector-level results of over $605 million in exports and 88,297 new jobs. Hence, these results suggest that (a) every million dollars of SEDP’s budget created over $43 million of exports and (b) every $158 of SEDP’s budget created a new job. Neither figure appears realistic or defensible. • Much effort seems to have been made within SEDP and the mission to distinguish SEDP from firm-level assistance programs. Yet, for SEDP, the distinction between firm-level assistance and sector-level assistance is at least partly overstated. In some cases such as tourism it is clear that SEDP undertook major sector-level activities, such as working with the Tourism Organization of Serbia, but in many other cases SEDP provided grants, market linkages, training and other inputs directly to a subset of constituent firms (for example, efficiency improvements in the Apparel sector). In fact, many of the sector-level activities undertaken by SEDP are often normally undertaken to various degrees in programs that fall under the firm-level assistance rubric after business associations are established (for example, under CRDA, ADF/FI agricultural development operations created large-scale producer marketing organizations that were supported by the program and had the ability to both market goods and advocate on behalf of members). As such, the Assessment Team largely agrees with the recommendations of the CEVES report that a more thorough quantitative assessment of SEDP activities should take place at the firm level using e.g. Firm Registry data, or perhaps separate firm-level surveys (See CEVES Report, Sections III.4.2.1-III.4.2.2). SLGRP As a program, SLGRP appears significantly better positioned for the conduct of ex-post evaluation activities than either SEDP or CRDA. Tasks and objectives implemented under SLGRP were far less complex, wide-reaching or ambitious than those implemented under CRDA, and the project deliverables are more readily verified and therefore measurable. The final report includes clear and easily identified municipality cohort phase-ins and the years of activity. SLGRP also appears to have made the most serious attempt of any implementer to track program impact through objective measures such as whether a municipality budget letter was published or not. Hence, the program deserves credit for attempting to make serious evaluation possible and more easily conducted, developing potentially verifiable indicators of program performance and for tracking the performance of at least some non-treatment beneficiaries, that is, non-program municipalities. This is not to say that SLGRP attempts to monitor and evaluate were without flaw. Only a limited number of control municipalities were tracked, the selection process for treatment-control municipalities was not random and perhaps resulting in an overstatement of the treatment effect, many of the indicators used were perhaps too simplistic to be informative, and the overall Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 45 process of development is sometimes ambiguously related to the indicators (for example, the production of an annual budget letter does not translate into greater transparency of municipal budgeting unless the average citizen can access such a letter and the process for publishing the letters has been institutionalized). The detailed assessment of CRDA, SEDP and SLGRP M&E is contained in Annex 5. 5.1.2 Assessing the Performance of Web-PRS Web-PRS was an ambitious effort to consolidate and unify tracking for five separate CRDA implementing partners. The system managed to evolve and adapt to meet changing needs and ensure conformity of tracking widely different goals while implementers used different vocabulary. For example, CHF’s KSP was intended to encourage risky entrepreneurial activity, while a similar program by ACDI/VOCA was intended to bring grey market activity into the formal sector. It is a complement to the original CHF designers that the system is used today to track M&E in many USAID programs worldwide. Web-PRS represented a major conceptual and operational leap forward in terms of performance monitoring and evaluation. That said, the Team’s view is that the Web-PRS did not reach its full potential as a management tool. Perhaps the most important overall issue with Web-PRS as a system is that it does not appear to have evolved or developed in sufficient synchronization with visions for program management, implementation, logistics or evaluation. By concentrating on the collection of information and data relating to monitoring, opportunities were missed to take the next step on the evaluation side, that is, to draw conclusions, make decisions and effect program changes. The detailed assessment of Web-PRS is contained in Annex 5. 5.1.3 Recommendations for future M&E efforts This section of the report provides a short review of several observations and recommendations that are intended to inform USAID/Belgrade’s M&E efforts going forward. Naturally, comments in this section are selective; it is beyond the Assessment Team’s SOW to deal with the specifics of how the evaluation of current programs should proceed and it is important to note that many of these recommendations could only have been made with hindsight. General Observations • Monitoring Challenges. There is always a tension between ideal monitoring and the smooth logistics of implementation. In the case of CRDA, extensive effort was devoted to monitoring “delivery.” Yet, indicators were largely constructed with an eye to logistical tractability, a convenience that may have been appropriate given the complexities of the CRDA approach, but certainly one that may limit their value as a true measure of performance; in such circumstances it is only through a more formal quantitative evaluation that can assess the efficacy of the program. • External data. This resource is a vital and important tool for performance evaluation. In the case of CRDA, the Public Opinion poll conducted by Strategic Marketing in 2007 uses a Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 46 well-designed questionnaire developed in partnership with Mission M&E staff. The data collected by this survey can be used to answer perhaps the most important evaluator question: how did CRDA contribute to the achievement of the USAID Strategic Objectives under which the program was contracted? Polling/survey data of this type is often more meaningful than indicator values in evaluating progress against strategic objectives • Beneficiary Selection. The mechanisms used to identify program beneficiaries provide insights useful to evaluation. Such records were not available for our assessment. • Evaluation Function. Doing this effectively should place program activities into an appropriate practical context and should not be confused by conceptual framings of such activities. For example, SEDP was framed as a sector development program - notoriously difficult to evaluate - rather than a firm-level assistance program (generally easier to evaluate). But in reality, much of SEDP activities, excluding perhaps those in the tourism sector, appear to have been similar to what would have been conducted under a firm-level assistance initiative. • Monitoring vs. Evaluation. We have already drawn attention to the amounts of monitoring data collected within Web-PRS. What is missing is data and data collection practices that are useful for evaluation. Yet, such data are important to help insure that CTO’s and COP’s have the tools to effectively and efficiently manage their common program activities. • Indicator Selection is vital. Implementers and USAID must identify and stick to a direct target beneficiary or ultimate end-beneficiaries for programs or program sub-components to be evaluated convincingly. Not doing this will create persistent problems and distractions for program implementation and ultimate evaluation. For example, in CRDA the most obvious direct target beneficiary was the constituent community and the ultimate beneficiaries were the inhabitants of that community. If this seems reasonable, then the indicators seem to lose sight of this in devising calculations that were more relevant to measuring individual rather than community impact • Simplification of Indicators. This helps with measurement, but oversimplification can make it difficult to understand a program’s use of resources. SLGRP’s monitoring and results tracking were relatively easy to verify, understand and interpret. However, many of the indicators tracked delivery of the contracted deliverables instead of measuring the developmental impact in terms of the strategic objectives. That is, SLGRP’s indicators emphasized whether a budget letter was produced rather than the extent of citizen influence and level of awareness in relation to the local budgeting process. SLGRP’s development objectives might have been better measured by time and costs required to: (a) start a business; (b) obtain a construction permit; (c) register property; and (d) enforce contracts. It may have also been more relevant to determine whether or not it is possible for the average citizen to obtain information about the local municipality budget; or whether the average citizen has confidence in their own ability to influence local government. • Indicator Calculations. Just as important as the Indicator is the calculations to be employed in their measurement. The lack of independence in devising and reporting indicators, Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 47 monitoring and evaluating impact (i.e. a reliance on implementers to perform this function) tends to lead to overstatement of indicator values, and also tends to greatly reduce their comparability across time, implementers, and geographies. Recommendations This section of the report offers a model for USAID/Belgrade’s consideration when designing M&E efforts in the future. Key features of this model include investment in Mission-wide MIS infrastructure, shifting most M&E responsibilities from the implementers to an independent evaluation organization (or organizations), and stronger integration of evaluation with other crucial program components such as program design. A schematic and explanation of how such a restructured approach might function is provided in Annex 5. This concept that may be helpful to orient future program design and lend additional credibility to those M&E processes. However, this section simply provides a concept and obviously requires far greater development in order to be tailored to the programmatic needs of the Mission/Belgrade. In closing, it is noteworthy that a great deal of effort was devoted to M&E activities by all three programs under review; much of this effort was in order, timely, appropriate and commendable. The Assessment Team has been highly impressed with both the implementation of the programs and the extensive Mission’s effort to carefully track every dollar spent. 5.2 Cross Program Synergies CRDA and SLGRP had one mission, bridging the gap in relations between citizen and local government organizations at the village, town and municipal levels. SLGRP also helped with development of legislation such as revenue sharing between the national and local governments. SEDP had an independent economic mission – strategic stimulation of more promising economic sectors of Serbia’s economy. CRDA “aimed at promoting citizen participation in and between communities to identify and address the critical needs for the economic and social revitalization of community life.24 Implemented more or less concurrently, CRDA and SLGRP can be viewed as two sides of the same coin, or, as USAID preferred, two banks of a river connected by a bridge. On the CRDA side, community development committees (CDCs) were organized to identify local priorities, prepare funding proposals, and identify local matching contributions - all with hands-on assistance from five U.S. implementing partners operating in different geographic areas of responsibility (AoRs). Projects were organized into the four “pillars” of citizen participation, income generation, civil works, and environment, with the remainder going to cover administrative costs of the implementing partners. Data drawn from the CRDA-wide monitoring and evaluation system known as the Web-PRS indicate that the program implemented 4,725 projects benefiting a cumulative total of 27.5 million people over the life of the project.25 24 CRDA Cooperative Agreement, Award Number 169-A-00-01-00124-00, Attachment II, Program Description, p. 9. 25 Individual beneficiaries were considered to have often benefited from multiple projects. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 48 On the other side of the “river,” U.S. implementing partner Development Alternatives, Inc. (DAI) worked to help local governments, “once again become more effective, responsive, and accountable to citizens” following a sharp decline in living conditions and ability to provide services after years of active neglect by the central government.26 SLGRP employed a variety of innovative techniques to help an ever-expanding cohort of municipalities improve service delivery, efficiency, and transparency through better financial management and public procurement systems, computerization, the establishment of modern, customer-oriented Community Assistance Centers (CACs), and the initiation of public budget hearings at which ordinary citizens could voice their opinions on local funding priorities. Both the CACs and the public budget hearings complemented CRDA efforts with CDCs to increase entrepreneurship and citizen participation in local government. CACs made it easier for new entrepreneurs and MSMEs receiving grants to register their businesses and begin operations more quickly. Public budget hearings permitted citizens who learned how to prioritize projects in CRDA’s CDCs to exercise the same talents in municipal budget hearings. • CRDA and SLGRP also cooperated in the creation of Business Improvement Districts. These efforts improved prospects for SMEs in the BID zones, reduced crime and improved the aesthetics of the areas improved. The process of developing BIDs again allowed citizens to exercise skills garnered in the CDCs. It also allowed municipalities to exercise new management skills learned through SLGRP efforts. To measure impact and track progress, SLGRP developed a Municipal Capacity Index (MCI) with concrete indicators showing progress in five key areas--financial management, municipal systems, communal enterprise, transparency, and citizen services. SLGRP also worked at the policy level to speed passage of new legislation aimed at decentralizing fiscal and functional authority to the municipal level. All of these SLGRP efforts were attempts to improve the business environment for SME’s, an important CRDA and CRDA-E target. 5.3 Lessons Learned and Recommendations This Section presents a number of recommendations for reliable approaches to overcome obstacles for CRDA and SEDP projects. 5.3.1 Internal restructuring for greater competitiveness First, Serbian SMEs should restructure to become more profitable and credit-worthy, thereby making it easier to obtain most appropriate form of finance for working capital and business expansion. Serbian SMEs are at a juncture that they need restructuring in order to meet the increasingly demanding market conditions and become more competitive. By “restructuring” we actually mean a process that all companies need to go through in order to adapt to changing market. Restructuring would address the formulation of a strategy to improve the internal and external performance; three common weaknesses of most Serbian SMEs: improve and streamline the operations of the business; and, gain better access to finance that will optimize their capital structure (debt and equity) and improve cash flow and liquidity. 26 SLGRP Final Report, Development Alternatives, Inc., October 2006. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 49 5.3.2 Improved management skills Second, they must improve their business management skills, through training and by bringing professional management into family businesses. Most non-agriculture sector businesses are family owned and operated and on average they are about 20-25 years old. These SMEs have reached a size that will require new management talent so that they can become better able to meet or surpass their competition and attain sustainable growth. Management of most family owned operations have now been transferred to the second generation. However, professional management skills are still lacking. Strengthening management skills are necessary to ensure achieving success in competitiveness of Serbian SMEs. 5.3.3 Innovative financing methods Third, better access to innovative financing sources must be provided in the niche area for MSMEs. Access to finance for most MSMEs is a major problem in Serbia. This situation may be addressed in the future by activities that emphasize building Non-Bank Financial Institutions, and working with banks to create innovative financing approaches for micro and small business financing. USAID’s DCA program can be tapped to support new programs such as business development and management in partnership with some financial institutions. We think that the pervasive financing gap for Micro and SMEs in Serbia can be resolved through innovative financial sector development programs. We take a brief look at this in our recommendations segment. 5.3.4 Access to Export Markets Fourth, SMEs need to improve their international competitiveness. International competitiveness of Serbian MSMEs is associated with increasing export sales. Increasing exports are a quantifiable goal. However, there are factors that do not lend themselves readily to quantification of accessing international export markets. Capacity for technological innovation, degree of product specialization, the quality of the products involved, or the value of after-sales service are all factors that were helped by USAID programs that may influence Serbia’s trade performance favorably. Policy level interventions are needed to define the competitive position of Serbian MSMEs relating to international cost or price differentials or to changes in such relative measures. USAID could affect changes in this regard. 5.3.5 Establishing Linkages with Strategic Partners Establishing linkages with strategic partners would enhance competitiveness by providing access to new markets, new technology, and financing for joint investments, networking and innovation. SEDP has already provided assistance for associations to establish strategic partnerships through various initiatives, but also through SIEPA. The managers of many sizable SMEs the team interviewed wanted to establish long lasting partnerships. They are ready and willing to collaborate with similar companies in EU or US to achieve their desire to access world markets for their products, by sharing risks, take on responsibilities, create resources or develop new competencies and take advantage of the benefits to share knowledge. These would be meaningful and long lasting partnerships that would allow the Serbian SMEs, ready to compete in international market place the foundation for success. Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 50 These partnerships would enable the Serbian firms to make continuous improvements in their management, deployment of their assets to their highest and best uses and increase employment opportunities, as well as generate new revenues and income for all stakeholders. The Serbian SMEs need to maximize their synergies by leveraging their assets (resources, capabilities, expertise, client base etc.) for the mutual benefit of both parties. The basic elements of business strategy in the new knowledge economy are being transformed. Serbian SMEs that are operating in the export markets have repeatedly indicated their willingness to make the move if the enabling environment was to be created. Instead of focusing on physical assets and economies of scale, the drivers of success reside in connectivity with partners both domestic and international. There are also intangibles that affect this predicament. Businesses increasingly need to develop and manage complex ecologies or organizations around themselves so as to succeed. The selection of strategic partners with whom to collaborate is now becoming a life or death issue for most firms in Serbia—they are hungry to learn more and take the next steps if the right partner (or mentor) can be found. Finally, and without doubt, successful partnerships would be good grounds for attracting more foreign direct investments into the country. Serbian companies are badly in need of an opportunity to make this happen. USAID is best positioned to ensure the success of this through its existing and future projects. 5.3.6 Improving corporate governance at Serbian MSMEs We have mentioned that Serbian MSMEs suffer from weakness in a number areas including, accounting, transparency and lack of information, managerial skills, and general lack of access to sources of finance. One of the key elements in improving economic efficiency for Serbian MSMEs is to improve their corporate governance. This would involve a holistic set of relationships between a company's management, its board (depending on its structure and legal status), its shareholders and other stakeholders. Since corporate governance is concerned with the efficient structuring of business entities, it is of fundamental importance to the performance of economies, particularly for a transition economy as Serbia. Weaknesses in the business sector are a major them in this report. Improving corporate governance will mean better access to finance for most Serbian MSMEs. To do this, would require significant efforts to develop the proper environment for business and efficient functioning of the financial and legal and regulatory frameworks. Safety and stability in the financial sector are of paramount importance for the foundation of any viable financial system, in order to build investor trust and confidence in the market. Transparency, accountability, responsibility and fair and ethical conduct in the management of private enterprises characterize “disclosure.” Disclosure is the product of good accounting practices. We have mentioned that due to weakness in accounting practices, Serbian companies had difficulty in accessing the sources of finance. Lack of reliable accounting practices, create higher Impact Evaluation of CRDA, SLGRP, and SEDP The Mitchell Group June 2, 2008 51 than necessary levels of risk perception by banks or other financial institutions. Adequate disclosure will promote the maximum participation in the organized financial markets as well as the most efficient market behavior. An important aspect of a transparent enterprise system is to have a well established internationally accepted accounting standards and independent auditing practice in place. In the Serbian case, improving corporate governance would need to go through a holistic focus on holding the balance between economic and social goals and between individual and communal goals. The improved governance framework could encourage the efficient use of resources and equally to require accountability for the stewardship of these resources. The aim of good corporate governance is to align the interests of individuals, corporations, and society. The incentive to corporations is to achieve their corporate aims and to attract investment. The incentive for states is to strengthen their economies and discourage fraud and mismanagement of scarce resources. USAID is best positioned in Serbia to consider offering programs through systemic and firm level interventions, such as legal and regulatory reforms and restructurings to improve competitiveness of Serbian firms at all sectors. At the end, improving corporate governance will mean better access to finance for many firms.