FINAL EVALUATION OF ETHIOPIAN NGO SECTOR ENHANCEMENT INITIATIVE (ENSEI) Contract No: PO# 663-O-00-02-00343-00 Submitted To: USAID/Ethiopia Prepared By: The Mitchell Group, Inc. 1816 11th Street, NW Washington, DC 20001 EVALUATION TEAM: DEREK SINGER, TEAM LEADER GETACHEW DEMEKE, TEAM MEMBER 22 August, 2002 ENSEI PROJECT FINAL EVALUATION TABLE OF CONTENTS Executive Summary I. INTRODUCTION 1 II. BACKGROUND AND CONTEXT A. Political and Socioeconomic Context 3 B. Evolution and Current State of Ethiopian NGOs 5 III. FINDINGS 9 A. Enabling Environment 9 B. Capacity Building 12 C. Networks 15 D. USAID-PACT Relations 17 IV. CONCLUSIONS 20 V. RECOMMENDATIONS 25 Annexes 1. Scope of Work 2. List of People Interviewed 3. Bibliography 4. NGO's, Core Training and Grants 5. Pact Comments on USAID ENSEI Evaluation 6. Evaluation Team's Comments on Pact Feedback EXECUTIVE SUMMARY ENSEI, the Ethiopian Sector Enhancement Initiative, has been a seven-year program governed by a Cooperative Agreement between USAID/Ethiopia and Pact, an American NGO based in Washington, DC. Project # 663-0020 was signed in July 1995, and funded for a total of $12.3 million. The project is due to end when it completes its second phase on September 30, 2002. However, the Mission is considering Pact's request for a no-cost extension for ENSEI until May 30, 2003. After reviewing numerous Pact reports and other ENSEI documentation in Washington, meeting the Pact country director (now on leave there), interviewing senior Pact headquarters staff, and talking with knowledgeable AID/W staff and others, the team leader traveled to Ethiopia and joined with the other evaluator to undertake this four-week, USAID-sponsored evaluation of ENSEI on behalf of a U.S. contractor, The Mitchell Group, Inc. (TMG). The evaluators reviewed additional documents; interviewed nearly 40 NGO's in the capital and in other regions (including two network focus groups); took two field trips to diverse NGO project sites; and discussed the project with Ethiopian central and provincial government officials, USAID staff, UN officials, and members of the private sector. Based on these reviews, meetings, observations, site visits and briefings, the team prepared this Final Report, which was also based on the Scope of Work and LOE agreed on by the Mission and TMG. Briefly, the key conclusions of the evaluation are as follows: · Pact has documented, and its partner NGO's have confirmed, a record of impressive accomplishments in strengthening over 80 Ethiopian NGOs and at least ten NGO networks. · There is solid support for ENSEI among most of the NGOs and government officials contacted, and a strong hope that Pact will find a way to continue its work in Ethiopia. · Pact's project has also helped create and extend a more favorable environment for NGOs in Ethiopia. Examples include the impetus ENSEI gave to the development of a new NGO Code of Conduct (now adhered to by over 150 organizations); its work to help draft and explain to its partners new legislation favorable to the interests of NGOs; "educating" officials in Addis and throughout Ethiopia about the many ways NGOs complement government development efforts; and co￾sponsoring the highly-regarded "NGO Observance Days," now celebrated annually both in the capital and in many regions. · Many say that strong sectoral NGO networks will greatly affect the future influence of Ethiopia's NGOs. To its credit, Pact has supported the creation and growth of at least 10 such networks in several important fields. Even with such a favorable track record, Pact has had its share of problems. Among the most prominent noted by the evaluators are: 1. A lack of transparent and objective standards and procedures in selecting ENSEI's partners. 2. Some difficulties in maintaining strong project leadership. 3. Delays in adapting to changes in Mission program funding and sectoral priorities. 4. Some embarrassing reversals of decisions re local NGO partners, and in working agreements with other organizations to help manage the project. 5. Inadequate preparations of an "exit strategy" to help prepare and support NGO partners to survive after ENSEI. Such problems notwithstanding, Pact's overall record has been a positive one. Thus, if USAID does decide to support further Ethiopia's growing non-profit sector (and there are good reasons to do so), ENSEI has laid a good base on which to build a new NGO support program - one that begins where Pact leaves off. In brief, following are the evaluators' key recommendations: a. If Pact supplies a reasonable rationale, the Mission should agree to a no-cost extension for ENSEI through next May. During this last phase of the project, Pact should develop and provide practical and comprehensive training and advice on program sustainability to its partners. b. Later, if a good NGO law is passed and if funds are available, USAID should again undertake to support national NGO’s in Ethiopia. Such an initiative should be synchronized with USAID’s sectoral objectives and program priorities. c. Changes recommended for a new project would include fees for services (on a sliding scale); a more aggressive public relations campaign about the many benefits provided by local NGO's; and a heavier emphasis on better ways to seek sustainability through income generation, donations, and other fund-raising techniques. d. As and when it materializes, the new project should be coordinated with the new initiative currently under discussion between the GOE and the World Bank to strengthen civil society. e. The menu of services offered by the new project to its paid subscribers would be market based. An enhanced program to create an ever more favorable operating environment for Ethiopian NGO's would also be implemented, financed (at least in part) from dues or fees paid in by clients. 1 I. INTRODUCTION Purpose of the Evaluation This Final Evaluation of the Ethiopian NGO Sector Enhancement Initiative (ENSEI) was conducted by The Mitchell Group, Inc. for USAID/Ethiopia from June 25 to July 21, 2002. According to the Scope of Work (Annex 1), its purpose was to carry out the following tasks: · Assess ENSEI's results in relation to the goals and objectives set forth in the USAID-Pact Cooperative Agreement, CA No. 663-0020-A-00- 5501-00 (July, 1995), as amended. · Evaluate the services which were provided, and the impact achieved, for each of the three principle ENSEI objectives, i.e.: ß Improving the enabling environment in Ethiopia for NGO's ß Building the capacity of local NGO's, and ß Promoting and strengthening NGO networks in Ethiopia · Provide conclusions and document lessons learned from ENSEI · Formulate recommendations to USAID re: possible future support to NGO's in Ethiopia This evaluation report seeks to assess the extent to which results were achieved in the key planned activities, and to document impacts achieved within each of the three major programmatic components of ENSEI. In addition the evaluation has reviewed the implementation process (project management), and has sought answers to such questions as: 1. Was Pact's management structure adequate and effective? 2 2. Was communication optimal among USAID, Pact, and the Ethiopian government? 3. Did the reporting, monitoring and evaluation systems of ENSEI worked well? 4. Was there evidence of significant beneficiary participation in the formulation and implementation of project activities? 5. Was the premise correct that NGO's can be effectively supported in their quest to become significant contributors to, and useful players in, Ethiopia's national development and progress? The evaluation was conducted by two consultants, an American and an Ethiopian under contract to TMG. Both were experienced in development programs, and the African NGO sector. In reaching their findings, the consultants interviewed a wide array of representatives from USAID, Pact, the NGO's, national and local government officials, as well as knowledgeable academics, researchers and private sector people. Besides many interviews in Addis Ababa, field visits were made to Pact-assisted activities in four other regions of the country (see Annex 2). The consultants also reviewed a significant amount of literature emphasizing previous assessments, studies of and by indigenous NGO's, the new draft NGO legislation), many pertinent Pact and AID reports, and other documents. 3 II. BACKGROUND AND CONTEXT A. The Political and Socioeconomic Context In 1991, Ethiopia emerged from three decades of protracted civil war, recurrent drought and famine, and a harsh military dictatorship. The Ethiopian People's Revolutionary Democratic Front, a coalition of ethnic-based resistance (rebel) movements, emerged victorious in a bitter civil war. The EPRDF then took over the reigns of power by establishing the Transitional Government of Ethiopia. However, in the early 1970's a combination of natural calamities, political turmoil, and the protracted civil war had combined to inflict immense suffering and devastation throughout the country. Large numbers of people fled their homes and became either refugees in neighboring countries or were internally displaced. Forced military recruitment, ill-advised resettlement schemes, and other abuses triggered a multitude of ills. Prominent among them were the fast-spreading impoverishment of many jobless rural and urban dwellers; the disintegration of families and communities; and a virtual breakdown of the rule of law, particularly in the countryside. Pervasive state control of the economy, coupled with a disproportionate allocation of scarce public resources to the "war effort," resulted in the neglect and undermining of physical infrastructure and social services alike. Public investment in education, health, and other state services, scarce in the best of times, was totally wiped out in the conflict areas, and minimal in the rest of Ethiopia. The cumulative effect of war, natural disasters, economic decline, social disintegration, and unchecked population growth accelerated the impoverishment. By 1991, Ethiopia was classified as the poorest country in the world by the UNDP Human Development Index. Confronted with such daunting problems, the EPRDEF set in motion a number of social, economic, and political reforms that fundamentally changed the polity and economy of the country. By granting full independence to the 4 breakaway province of Eritrea, the government brought an end to decades of brutal and devastating civil war. For the rest of the country, the new regime adopted a decentralized, ethnic-based federal form of government by creating fourteen "autonomous regions." The regional authorities were given broad political and economic powers, and shared significant decision-making authority with Addis Ababa. In short, the EPRDEF turned Ethiopia's form of government from a unitary centralized state to a federal decentralized one, thereby neutralizing most long-standing ethnic-based yearnings for autonomy and localized power-sharing. These steps brought a large measure of peace to the war-ravaged nation. Another major political reform was the largely-successful effort to transform the national government from a Marxist military dictatorship to a constitutional democracy. Several elections for local and regional assemblies were held from 1991 to 1994. The National Constitutional Assembly was was elected at the end of 1994, and a new Constitution of the Federal Republic of Ethiopia was prepared and ratified. National and regional elections for members of the federal and regional assemblies were conducted in 1995, and a second round of elections for the same bodies was held in May, 2002. The governing party emerged as the winner in Addis, and in some (but not all) of the regional contests. Economically, the nation also made a major transition from a command to a free market system. Largely adhering to the framework laid down by the IMF structural adjustment program, the new economic policy launched in 1991 introduced key changes that reduced the role of the state while increasing and strengthening private sector activities. Market-based reforms, including extensive privatization of public enterprise, financial and trade liberalization, tax reforms, investment incentives, etc. were enacted. Also, several public corporations were dismantled; state farms were closed; and the role of government in the economy was significantly diminished. 5 Regarding national development policy, the new government began to notably shift from a focus on relief towards rehabilitation and development. The problems of drought and famine were addressed, along with the extensive human suffering and dislocation caused by the long wars and unrest. At the same time, priority measures were adopted to get rapid humanitarian aid to citizens in the worst-hit regions; a structure for disaster prevention and management was established; and an official commitment was made to work towards sustainable development and the alleviation of rural poverty. Food self-sufficiency was adopted as Ethiopia's top economic goal, and for the past decade the EPRDEF has stuck by a "food first" policy. While it is still too early to declare victory, many observers believe the government's food security program in the rural areas has made definite progress. B. The Evolution and Current State of Ethiopia's NGO Sector In spite of its social problems, Ethiopia has a rich tradition of associational life. From time immemorial, voluntary associations (often called community￾based organizations) have been encouraging community solidarity, addressing their members' needs for help and encouraging community solidarity. Edir (community associations for burial), Equib (community savings and credit associations), and Debo (voluntary community labor groups) are some of the many traditional community based associations in Ethiopia. These groups have managed to survive the many political, social and economic changes of the past three decades. Particularly in the rural areas, they still remain credible institutions to which people turn in times of need, hardship, and affliction. They were also breeding grounds for the country's modern NGO's. More precisely, the start of the modern NGO sector in Ethiopia can be traced to the 1950's, when a number of privately chartered bodies such as the National Boy Scout Association, the Ethiopian Red Cross, Ethiopian women's associations, and the Family Guidance Association of Ethiopia. Professional groups such as the National Bar Association and the Ethiopian Teachers Association were also created during this period. The Ethiopian droughts of 6 the mid-1970's and mid-1980's attracted a major influx of international NGO's, and the same emergencies also triggered the formation of many modern Ethiopian NGO's. At first, these groups understandably focused heavily on relief and rehabilitation, and many of them forged links with the international NGO's that had come to help in the food crises. Especially in the '80's, both international and local NGO's fighting the famine gained prominent roles as some foreign donors, suspicious of aid diversion by the Marxist military government to the war fronts, shifted their assistance to the private groups. As a result, after the famine subsided a jealous and suspicious government sharply cracked down on the nascent E/NGO's by curtailing the scope of their programs, restricting their finances, and harassing (even jailing) their leaders. Hence, by 1991 virtually all local NGO's had become tools of the state, and were discouraged from any serious development efforts. It was not until the 1991 transition that an independent Ethiopian NGO sector began to reappear and began to operate in spheres such as community development, environmental protection, basic education, D/G, and human rights. This reemergence was, in fact, one of the visible signs that the 1991 transition had taken hold in the country. Today, there are some 400 relief and development-oriented NGO's registered with the GOE Ministry of Justice. An additional 50 or so may be operating unregistered. Considering a population of 60 million-plus, and compared to other African countries such as Kenya, South Africa, and Senegal, the number of Ethiopian NGO's is relatively small. About 30 percent of the NGO's in Ethiopia are international, and 70 percent national. The great majority were formed after 1991, so they are relatively new and inexperienced. Roughly 70% to 80% are urban based, and some 90% of their activities are carried out in the five largest regions of the country: Southern Nations People, Amhara, Addis Ababa (region 14), Oromia and Tigray. Some of the newer administrative regions along the borders with Sudan and Somalia are particularly under￾served and in need of help from the NGO community. 7 According to estimates from the UNDP and DPPC (a government NGO regulator), roughly five percent of external assistance is being brought into the country via NGO's. CRDA, the major national association of NGO's, as well as the Ministry of Foreign Affairs, estimate the real figure is closer to 20%. In either case, it is clear that NGO's in Ethiopia generate significant foreign support here - perhaps $40 to $50 million p/a. With little or no government help, and few domestic donors or other assistance locally available, most E/NGO's must seek most of their funding abroad. Lacking much experience, and with few trained staff, the majority of these groups are still weak and have limited programmatic impact. There are exceptions, however. The humanitarian and development branches of some political movements, such as the Relief Society of Tigray (REST), and the Organization for the Rehabilitation and Development of Amhara (ORDA), have built up a capacity to design and implement a number of significant projects in their regions. Several of the larger Ethiopian church affiliated groups (e.g., Medkan Yesus) are also strong leaders, with the institutional base, the resources and the popular support they need to make a significant impact in relief and development work in many parts of the country. Collectively, Ethiopian NGO's try to fight poverty and support development in many ways. These include: · SERVICE DELIVERY: Most of Ethiopia's nascent NGO's can supply some goods and services at the community level (rural and urban). Increasingly, they try to deal with the major problems of youth: HIV/AIDS, unemployment, and education. · SOCIAL MOBILIZATION: Local NGO's serve as catalysts to mobilize communities and help form community based organizations (CBO's). The CBO's are assisted in attacking their distinctive local concerns: social, economic, environmental, etc. Groups like REST and ORDA, 8 referred to above, have particularly emphasized such local empowerment techniques. · HUMAN RIGHTS AND GOOD GOVERNANCE: While many believe all NGO's are intrinsically concerned with strengthening good governance and popular participation, after 1991 a new crop of D/G and HR NGO's emerged in Ethiopia that focused specifically on these concerns. Such groups undertake a number of related activities, such as monitoring human rights, civic education, election observation, and advocacy on behalf of disadvantaged groups. · NON-PROFIT INTEREST GROUPS: While some were started over three decades ago, many professional associations and issues-based interest groups have become nationally active only recently. Groups like the Ethiopian Teachers Association, the National Chamber of Commerce, the Ethiopian Journalists Association, and the Ethiopian Economics Association now play significant roles in policy areas affecting their members and professions. On several occasions, they have successfully challenged the government on national strategies and policies that have an impact on the future of their affiliates. Their influence on, and sometimes their cooperation with and support of, the E/NGO's, is often important to the success of many development groups. 9 III. FINDINGS A. Enabling Environment The ENSEI project carried out a range of activities designed to create an atmosphere to encourage national NGO's (E/NGO's) to increase and prosper. Among these activities were supporting a coalition of NGO's that successfully promulgated a new Code of Conduct for these groups which has now been adopted by over 200 national and international NGO's. Pact also helped promote a set of guidelines and principles for NGO-government cooperation at both the federal and local levels. A typical result of this cooperation was the establishment of a new curriculum for Alternative Education now in use in the Oromyia region (and elsewhere). Another example was Pact's successful effort to bring together representatives from the Ministry of Labor and Social Affairs and the E/NGO's in Addis Ababa, leading to an ad hoc committee that now promotes cooperation between the two parties in a variety of social service fields. Similar groups, the evaluators learned, have been formed in Southern and Amhara regions. Through this project, Pact has also encouraged better communications in a number of ways to change public understanding and perceptions of NGO's in Ethiopia. Specific accomplishments include a media handbook to help these groups learn how to work better with the press; an NGO Press Kit to better publicize NGO contributions to development and poverty alleviation; videos that air the work of local NGO's on TV and at special events; and significant work to organize and implement the increasingly popular "NGO Observance Days" where these groups now publicize and put on display their accomplishments for the general public. ENSEI also featured training and "exposure tours" that were partly targeted at both journalists and government officials (local and national). These helped highlight many valuable lessons from other countries in Asia, Africa and the Americas where NGO's play a more extensive and prominent role in 10 development and advocacy than they do at present in Ethiopia. Finally, Pact has helped organize Ethiopian NGO's to review and support important new legislation now eagerly awaited by the NGO community. Passage of the new law is likely this year, and the E/NGO's believe it should open long-awaited new avenues to seek voluntary donations, conduct fund raising freely, and explore income generation options, something the government has not permitted many non-profits to do before. Discussion Nearly everyone interviewed by the evaluators agreed that the NGO enabling environment notably improved in Ethiopia over the past six or seven years. Understanding and good relations between the GOE and the NGO's have improved at all levels; the public perception of NGO's has changed for the better (e.g., a successful pilot NGO TV Telethon show recently took place); and the adoption and enforcement of the NGO Code of Conduct has won praise from many quarters. Respondents cited a number of other examples to illustrate how NGO's have gained increased understanding and appreciation in the policy field. Among these were: · Both the recent poverty reduction strategy paper and the national HIV/AIDS control and prevention strategy, received substantial input from the E/NGO's. The GOE has also now agreed to provide several million dollars to a number of these groups active in these two fields. The fact that resources of this magnitude are being availed by the GOE to local NGO's is both unprecedented and emblematic of the growing influence of these groups, as well as of the value both the public and the government now place on their efforts. 11 Some benefits anticipated from the new NGO law have already been mentioned. When it is passed, the evaluators were told, a simplified government registration process is also likely; regulation of NGO lobbying and advocacy activities should be eased; NGO's will be able to freely form sectoral networks and umbrella groups which will (for the first time) be allowed to raise funds to support themselves; and tax breaks are anticipated that will encourage giving to non-profits (as in the U.S.) by individuals and businesses in Ethiopia. · As noted, there is a growing tendency for government and NGO's to collaborate on local projects, particularly in social services, education, and HIV/AIDS prevention and control. Additional such partnerships are also growing in such fields as assistance to street children, youth employment, non-formal education, and micro-finance. Respondents stressed that this increasing cooperation, particularly at the local working level, mirrors the growing positive public image of NGO's in Ethiopia, and a better appreciation of their diverse contribution's to Ethiopia's national development. While it is difficult to empirically establish Pact’s precise contribution to these advances, no one contacted by the evaluators suggested that Pact’s efforts had not played a key role in improving and enriching the environment in which NGO's operate in Ethiopia. It is less clear whether the same NGO's, without Pact's support, can and will continue to sustain and move forward with cooperative campaigns to sustain a truly enabling environment for these groups. In fact, the evaluators believe that even more collaboration, and more proactive steps, will need to be taken after ENSEI ends in order for the NGO community in Ethiopia to sustain (and certainly to increase) the gains it has made so far. 12 B. Capacity Building Under Objective II, ENSEI project activities focused on strengthening NGO's to the point where they can carry out effective program activities, and become eligible for, and capable of, raising financial support from other sources. To meet this need, the project concentrated on strengthening NGO capacities by effective organizational capacity assessments; training in strategic planning and management; participatory project design, monitoring, evaluation, and reporting; and financial management. A broad range of technical training was provided, including institutional strategizing; board development for non￾profits; conflict prevention and management; fundraising strategies; leadership and team building; gender planning; natural resources management; HIV/AIDS prevention; and many others. Under this heading, the project also upgraded the technical and mentoring skills of over 25 Ethiopian consultants, who were called on periodically by Pact (and others) to help train NGO staff. A number of Ethiopian NGO leaders (and related GOE NGO liaison officers) have broadened their horizons and obtained new ideas, skills, and appreciation of NGO impact through exposure tours to other countries in Africa, Asia, North and South America. Others have been sent abroad to attend specialized seminars and short courses. Pact has also operated a Development Resource Center, where NGO practitioners (and others) access up-to-date technical information electronically, consult reference books, and learn research skills. In addition to technical training, most of the 83 E/NGO members benefited from Pact’s Strategic Action Grants. These grants provide broad opportunities for nascent (new) NGO's to rapidly gain experience in both project management and grant administration. Depending on the particular needs of the group, they have provided help in the form of personnel secondments, office equipment, and start-up funds for projects in the field. 13 Discussion The evaluators found widespread satisfaction with the institutional strengthening/capacity building side of the Pact program. An Annex to this report provides a comprehensive list of the diverse Core Training programs and Grants provided to all ENSEI project partners throughout Ethiopia up to the present. A few organizations were de-registered by the GOE. Some others were dropped when USAID financing shifted away from the Democracy and Governance (D/G) account, and Pact’s sectoral and geographic focus was redirected. Apart from such interruptions, this aspect of the ENSEI project was widely appreciated. The evaluators found no disagreement that it has made a solid contribution to creating a core group of stronger, better managed and more capable Ethiopian NGO's. However impressive such results may be, follow-up and "sustainability" for AID are key concerns that all projects must address. In the capacity building segment of ENSEI, following an extensive self-assessment exercise NGO's become accepted as partners. During an initial phase, they participate in extensive training and "mentoring" programs; receive technical assistance (directly or through grants) from Pact and its consultants; and can also apply for personnel and commodity grants. Pact follows up on these benefits and interventions by "graduating" partner NGO's, when deemed ready, to "alumni" status. Introductions to other donors are provided; advanced specialized training may be arranged; networking with groups that share similar interests (in Ethiopia and abroad) is facilitated; and alumni groups are encouraged to provide organizational development support to newer E/NGO's. However, in a broader sense Pact's work on sustainability has had a mixed record. At the macro level, some important work has been done. Pact has assisted in developing and implementing an "NGO Code of Conduct", which is starting to take hold in the local NGO community and has enhanced the credibility and public acceptance of the E/NGO's as a group. Pact also worked on the new draft NGO legislation which, if passed, should open the door for 14 the local groups to obtain more contributions; to raise fees for services; and to expand their currently-modest income- generating activities. That said, Pact's efforts to systematically prepare most of its partners for the day when ENSEI will end do not seem adequate. The evaluators were advised by some E/NGO's they did not know that Pact was about to end. Some were told, and given ideas and leads for possible new sources of funds. Also mentioned by a few NGO's were innovative and promising approaches for obtaining new support by building new alliances with local communities, government, and even the private sector. Apparently Pact was planning to hold a workshop and offer additional help in promoting sustainability, but this effort was sidetracked as a result of ENSEI's financing problems in 2000 and 2001. Finally, as the ENSEI project draws to a close, the evaluators determined that Pact has not come forth with a specific proposal for indigenizing the project; for proposing a plan by the formidable cadre of staff, the institutional infrastructure, and the wealth of experience the project has amassed might best be protected; nor for how fine resources Pact helped put together for the last seven years should be refocused to help fight the disease, ignorance, poverty and underdevelopment that still afflict this promising country. 15 C. Networking Pact has been quite active in helping sectoral NGO's set up and manage networks. Such federations provide groups working in specific sectoral fields with forums to discuss common issues, allow them to share experiences and best practices, and encourage them to promote their common interests. So far, ten or more such groups have been created, including NGO networks for orphans and vulnerable children, basic education, micro-finance, micro-credit development, street children support, community based rehabilitation, and youth training and employment, HIV/AIDS, civil society, and women's affairs. Assistance provided to these groups by Pact has included training in how to form networks; small grants; the loan of technical staff members; support for meetings and exposure tours; and facilitating good relations with cognizant government agencies. Collaboration between Pact and CRDA, a well￾established membership group of national and international NGO's that also gives diverse assistance to its NGO members, has been particularly good in promoting networking among sectoral groups. For example, senior management and technical committees were jointly created by Pact and CRDA to encourage joint programming (and prevent duplication of services) in their network assistance efforts. Other examples of Pact's work in this area include help to the Forum on Street Children to establish a resource center for its members, and sending staff from Pact's partners abroad to introduce them to successful networking techniques elsewhere. Discussion Most respondents gave credit to Pact for the growing number of Ethiopian Networks and consortia. Pact played a key role in establishing virtually all the networks mentioned above; they received financial and advisory support from Pact early on; and they continue to operate today basically with Pact's assistance. Pact was also credited with for its work to allay suspicions and jealousies among potential network members, and for emphasizing their shared interests and objectives as the new networks were painstakingly built. 16 Although most are still in embryonic stage, some of the networks have begun to expand; to share their experiences freely; to join together in fund-raising efforts and advocacy campaigns; and to liaise more effectively with local agencies. The non-formal and micro-credit networks for their coordination and member support work. As soon as it is passed, the new NGO legislation will be of special importance to the sectoral network groups, since it should confer formal government recognition on them for the first time. While generally encouraged by these developments, some respondents told the evaluators that many of the NGO networks were too concerned with urban matters and were concentrated too heavily on Addis Ababa. To the degree such criticisms are on target, the networks would be wise to expand their reach to the more isolated rural areas; look for recruits among all subgroups within their sector; and establish linkages with others in each sector outside of their own networks. Finally, a crucial point was made to the evaluators by several thoughtful leaders of NGO networks. It appears that neither the networks nor Pact itself have given much thought about how to sustain such groups after the ENSEI project. With few if any other sources of funding, all the networks that were interviewed appear to depend heavily on Pact's support and initiative. If they are to survive after ENSEI ends, it is imperative that the issue of sustainability and alternative funding sources be dealt with as a top priority matter by Pact and its network partners. (Note: this issue is discussed below at greater length. In the evaluators' view, the failure of Pact to seriously come to grips with the sustainability of ENSEI's key project components, NGO capacity building and networks, is the salient weakness of the project). 17 D. USAID - PACT Relations Ever since a Cooperative Agreement was signed between USAID and Pact on June 7, 1995, a number of irritants, misunderstandings and disagreements have occurred in the relationship. Both sides have suffered somewhat, and both have protested about the other's obduracy, errors, and misconceptions. There is blame enough to go around on both sides. Fortunately, such problems did not prevent the general success of the project. Nevertheless, since these concerns came to the evaluators' attention, they should be mentioned: 1. JURISDICTION (“Who’s in charge here?”) The Pact project began under GDO office was changed to HID-Education. Last year, some technical oversight was also assumed by other S.O.’s which reluctantly invested a portion of their program funds in Pact after full D/G funding was not forthcoming. Pact was unhappy when this shift forced ENSEI to curtail some activities, and move others, as a result of this funding switch. 2. SUBSTANTIAL INVOLVEMENT (“We’re just too busy”) With Mission management of the Pact grant changing and uncertain, Pact believes it did not always receive the full attention and support it needed from Mission personnel to do the best possible job. USAID technical staff, in particular, responds that after the D/G office contracted, their own time was often too short to spend much time with Pact and its local partners, particularly on field visits. 3. FUNDING DELAYS (“You promised us our money!”) Pact believes that on several occasions delays in funding were longer than reasonable, and, hence disruptive and damaging to the ENSEI program. On at least one occasion, the evaluators were told, the situation became so serious that Pact had to suspend its support to its Ethiopian partners for several months in 2000 and 2001. USAID staff believes Pact understood and agreed to these changes. 18 4. REPORTING ("Why didn't you tell us?") Complaints were heard that Pact's reporting on the use of Mission funds, especially the new sectoral funds, was inadequate; that Pact failed for months to submit a revised work plan (as requested), and that other Pact reporting was sometimes weak, late, and outside the standard reporting guidelines requested from all grantees. Pact denies this, and asserts that few if any complaints were ever received about its reporting. 5. PERSPECTIVES ON SCOPE ("We're partners, aren't we?") In the ENSEI project, Pact wanted (and thought AID had agreed to), an NGO-centered project that was neither secondary, nor subservient to, other Mission projects or objectives. USAID program documents and staff statements suggest a more limited and tightly organized effort was expected and had to be compatible with and supportive of the changing sectoral, geographic, and funding priorities of the Mission. 6. PERSPECTIVE ON SIZE ("Tail wagging the dog?") Over the seven years of the project, some at USAID say it was counterproductive to divert ever-scarcer management resources on such a small project as Pact, considering that the staff was already stretched dangerously thin trying to manage a portfolio amounting to half a billion dollars over the same period. Some sectoral grantees were also required to make the same painful choices. Basically, Pact responds that "a deal is a deal," and it was not really fair to shift the goal posts midway through the game. 7. TO EACH HIS OWN ("Let's keep it in the family.") To different degrees, the technical offices often felt more comfortable and confident working with (and relying on) familiar, sector - dedicated NGO's and contractors. Some also believe that the sectorally funded "Pact Tax" unfairly depleted their core program budgets - budgets that could and should have been used for "higher priority purposes" more effectively and efficiently. For PACT's response, see number 6, above. 19 8. FASTER IS BETTER ("When do want it? Now!") Among Mission staff, most agree that Pact has indeed done good work in strengthening local NGO's - but some think their multi-sector program of institutional capacity building is now too far removed from USAID/E's accelerated, more tightly - targeted program priorities. Program directions have changed, and the demand for more measurable and quantifiable indicators of success has become increasingly urgent (e.g., HIV/AIDS). They say prescriptive indicators are now required in the Annual Report, and AID/W demands speedier action and more clear-cut results than projects like ENSEI can produce. Pact's perspective is that even if this analysis is right, stronger, more skilled and better led Ethiopian NGO's should still be seen as the best opportunity to mobilize Ethiopia's citizenry in actively and effectively contributing to the nation's development and prosperity - something everyone should get behind! The evaluators believe that certain problems and irritations such as these will probably always be inevitable between AID and it's grantees and contractors in the field. However, if and when USAID/E does embark on a new program to support and strengthen NGO's in Ethiopia, based on such problems with ENSEI as those mentioned above, the evaluators believe that better communications and a somewhat more responsive management mechanism, would improve the final result. 20 IV. CONCLUSIONS Despite some setbacks, delays and misunderstandings, Pact's program in Ethiopia has been largely successful. With support and encouragement from AID, Pact can point to significant accomplishments in fulfilling its basic goals of fostering a more positive environment for Ethiopian NGO's; strengthening over 80 national NGO's and related groups; and encouraging the formation and growth of nearly a dozen NGO networks and umbrella groups. For example, since ENSEI became operational in 1996, Pact has: 1. Administered OCAT, their organizational self-assessment exam, to more than 105 E/NGO's (the OCA diagnostic tool was universally praised by all NGO's contacted); 2. Trained over 60 NGO officers in financial management, and more than 100 individuals in a popular computerized accounting program (Peachtree program); 3. Enhanced the project proposal development capacity for nearly 300 NGO staff and government officials; 4. Supported 18,500 enrollees at 135 Alternative Basic Education Centers run by 10 Pact partners; 5. Provided 44 education scholarships to girls at Unity College, most of whom enrolled in accounting and MIS; 6. Established and/or supported development of 10 sectoral NGO networks, including new ones for HIV/AIDS groups, gender matters, and voter education; 7. Provided rural NGO partners with significant subgrants for grain banks, food security enhancement, and small-scale irrigation projects. 21 Based on evidence gathered from interviews with NGO's and related groups that have received ENSEI support, and also with others that have not, the general positive impact and favorable results from project benefits like these are clear. To illustrate, here are some typical quotations about Pact that the evaluators gleaned from NGO staff and GOE officials: · Pact has helped the entire NGO community in trying to facilitate the passage of new, liberalized NGO legislation. Most importantly, they have worked successfully to positively change the attitudes of development officials in the GOE, so that a real partnership can develop. - Mengiste Ayele, Orphans and Vulnerable Children. · Pact should go on. Sectoral specialization is important, but it shouldn't freeze out the crucial task of capacity building among all our NGO's. - Habtami Mammo, AGOS · Pact's work, although we are not a partner, is really helpful to environmental groups like ours. In particular, strategic planning and "hands on" mentoring support is something of great value that stays with us forever. - Tefera Wegderesegn, CHE · Pact's chief value is in enhancing the enabling environment, and raising public consciousness about the value of NGO's and the need to support and encourage them. They're also very good in helping NGO's build bridges with government agencies; developing local funding strategies; and strengthening NGO staff. Finally, Pact and CRDA complement each other well in what they do, so both are needed. - Asfaw Mekuria, DPPC As indicated earlier, while ENSEI has scored many successes, the project was not without its problems. Several of Pact's difficulties with AID, particularly during the last two years or so, are described above (see "Findings"). The lack of a full understanding and good communications between USAID/E and Pact caused problems that probably slowed progress in reaching ENSEI's goals and those of its partner NGO's. While none of these appear too serious, and some 22 should probably be written off, Pact also had significant problems with its own partners during the project. Among these were: 1. A year or so after Pact teamed up with IIRR, another international NGO, to manage ENSEI, the evaluators were told that "divorce proceedings" were initiated that resulted in some unfortunate delays; caused Pact to restructure its timetable and approach to food security and rural development activities; and raised questions about Pact's judgment in selecting IIRR as an initial implementing partner for the grant; 2. Pact staff had misunderstandings with several E/NGO's whose directors believed their groups had been offered grants, consulting arrangements, or other benefits from ENSEI, and were upset when such arrangements fell through; 3. At least one prominent local NGO head said he and others were disappointed with the quality of Pact/Ethiopia's leadership after the death of ENSEI's first American director. Other informants questioned the wisdom of trying to manage Pact/Ethiopia (or any such organization) with two co-directors. One or two are still certain Pact took undue advantage of them, lied to them, or falsely claimed credit for good work done by the NGO's themselves; 4. The evaluators were told by several government and NGO representatives, knowledgeable about Pact’s partner selection process, that this important process was neither as transparent nor as fair and objective as it should have been. Notwithstanding such criticisms, however, interviews, observation, and research indicated that, for the most part, ENSEI did what it set out to do. Much was accomplished in practical terms to encourage a positive "enabling environment" for Ethiopia's growing number of NGO's and related groups in what was an historically negative, even hostile, environment in which they 23 previously had to operate. A significant number of relevant new skills were imparted to partner NGOs' boards and staff that made them more useful, effective, and (importantly) more self-confident in breaking new ground and in delivering more and better services to their clients. In another "plus," Pact was often praised for applying innovative techniques in encouraging ENSEI's partners to work together building new relationships and strengthening old ones. In looking back on these positive, diverse, and even unique accomplishments, Pact and USAID/Ethiopia should both be commended for the overall success of ENSEI - an innovative, creative, and predominantly effective development tool that came along at the right time to mobilize and inspire many Ethiopian NGO’s to combat more proactively the poverty, ignorance, disease and hunger that still afflict so many in both the rural and urban areas of this country. That said, in light of changing needs and priorities of the country and of USAID, as ENSEI comes to an end, it is time to take a careful new look at ways in which the Mission might best continue to assist national NGO's in Ethiopia. With an increased Agency stress on rapid results, a new project portfolio, new funding imperatives, and new AID/Washington priorities, all older projects and programs must inevitably be scrutinized, adapt to the times, or be replaced by others when they no longer fit these changes. At the same time, the evaluators believe it is prudent to capitalize on ENSEI's successes and accomplishments; to learn from its mistakes; and to build on the significant "lessons learned" from this trail-blazing project. And, while taking into account the Mission’s new directions, resources, and priorities, the evaluators believe the cause of development in Ethiopia would be well served by some form of continued USG attention to, and support for, the unique and valuable resources offered by Ethiopian NGO’s in their relief, development and civil society work alike. It bears repeating that the path pioneered in the Pact ENSEI model has valuable lessons to offer. These should not be discarded lightly. The final section of this report presents the evaluators' ideas on how best to capitalize on them. 24 Neither Pact nor any other AID project, of course, can be self-managing or self￾monitoring. For every activity, the Mission must be able to supply adequate project management to help insure all activities stay on track; continues to be compatible with USAID’s legislative and regulatory framework; and remain in line with the Mission’s changing program priorities. Although AID and the U.S. NGO community do have a long-standing agreement, based on trust and mutual respect, that is meant to encourage flexible and innovative field partnerships, at the field level things are somewhat different, particularly in coordinating with local NGO's. Hence, based on past experiences, present realities, and future program directions, the challenge of building a new relationship with the NGO community in Ethiopia will not be easy, and there will certainly be more problems and headaches in a new NGO program (if one occurs). But a good base has now been laid: Ethiopian NGO’s are growing in numbers, capacity and popularity, and the government is encouraging new ways by which they can become more useful and independent. As described below under “Recommendations,” the evaluators believe there are useful ways the Mission can work with its other current U.S. NGO grantees in Ethiopia to continue modest support to at least some of Pact's local partner groups. If more robust partnerships of these local groups can put together ways to get the needed management and technical skills, the Mission might also consider future direct grants to such groups. Moreover, the evaluators have concluded that when the time comes an updated, more indigenized (Ethiopianized) follow-on activity to the Pact ENSEI project should receive serious consideration from USAID/Ethiopia. In short, even with the current cut-off of D/G funding at USAID/Ethiopia, the evaluators believe that the Mission can and should still seek ways to encourage stronger national NGO's - both civil society and development￾oriented. Not to do so now would needlessly deprive the agency of an excellent tool in the war against poverty, ignorance and underdevelopment in this country. 25 V. Recommendations The predicament of African NGO's is that empowerment means access to regular funding that will cover their permanent or organizational costs, and help them grow from being an operator into being a secure and independent partner. But current conditions in Africa make it almost impossible for indigenous organizations to access such funds locally. - Strategy for Institutional Development, AHA A. Background As noted elsewhere, within the framework of Pact's original proposal, most evidence suggests that ENSEI's successes have clearly outweighed the problems which have affected the project. Over the past seven years, more than 80 Ethiopian NGO's have been strengthened and assisted in a variety of ways by ENSEI (somewhat over 20% of the current estimated number of E/NGO's). During the life of the project, there is also every indication that Pact's contribution has been substantial to the improved environment in which their partners (and other E/NGO's) have matured and developed. Pact has acted both as a catalyst and a midwife in cementing closer relations among many such groups with common concerns and agendas, and in encouraging them to work together in more effective and productive partnerships. While other factors also contributed to these improvements, the evaluators were told by virtually everyone they asked that Pact itself played a significant part in accelerating them, as well as in helping to overcome public, governmental and media suspicions and misunderstandings about NGO's. Many of these problems have deep-seated political and historical roots. Progress in improving the climate was made by Pact through such means as introducing "best practices" to Ethiopia from successful NGO's abroad; cultivating and informing government staff influential in NGO concerns; supplying training and exposure about NGO's to journalists who often shape 26 public opinion; encouraging and educating NGO leaders to learn and apply the skills and techniques of lobbying and advocacy; providing fund-raising know￾how to selected E/NGO's; and supporting popular annual "NGO Days" in various regions of the country. These programs and accomplishments were clearly in line with the Agency's policies and priorities when the Pact Cooperative Agreement was signed seven years ago. Moreover, there is a "waiting list" of nascent Ethiopian NGO's that seek to become new project partners, and many current partners told the evaluators they wanted to continue their Pact association to take full advantage of the many valuable services the project offers. Does all this mean that ENSEI should now be extended or renewed for another period of time, with additional AID funding? Even with ENSEI's significant successes, budget constraints and changed program priorities would make this, at best, an unlikely option in the foreseeable future – although it still should be considered when possible. Both by design and philosophy, AID projects are not meant to last forever, and ENSEI is no exception. After six or seven years, even the positive results of good programs like this normally must yield to a new approach that redefines and updates its goals and purposes, as well as to a new organization, management and methodology. Thus, the following recommendations are based on lessons learned from this evaluation; from previous experience with Pact and similar NGO's; from the principle that all aid projects should end at a time certain; and from the increasingly clear need for all Missions to sponsor only a limited number of projects and activities in a reduced number of fields. Today, every such initiative must also show demonstrable accomplishments and results related to USAID's changing Strategic Objectives, and do so within an increasingly short time frame. Hence, while the evaluators believe there are excellent reasons for USAID/E (and the USG) to lend additional help to strengthen the non-governmental sector as a good development partner in Ethiopia, at the 27 same time they have also concluded that a "new deal" in the Mission's future relationships with the local NGO community is inevitable. As this Pact project draws to a close, however, the Mission is urged to continue its support to E/NGO's as valuable development allies, but also to apply significant modifications in its future support, both in the near and the long-term. B. Recommendations If Pact satisfies the Mission about why ENSEI funds are still available, and presents a reasonable plan of what they propose to accomplish with this money by the end of May 2003, their progress to date suggests that a no-cost extension for the project may be justified. Some suggestions appear below about ways that Pact should focus ENSEI's final months if a no-cost extension is approved by the Mission. Local NGO’s have made a positive difference in the development and prosperity of Ethiopia, and they can continue to do so. Hence, when new budget and program circumstances permit, the evaluators urge USAID/E draft a fresh RFP (or the equivalent) to cover a new two to three year project to build on ENSEI's past accomplishments and help to move E/NGO's to the next level of useful contributions they can make to Ethiopia. Such a new project would realign the work, particularly of development NGO's, to the realities of the new millennium in Ethiopia - realities that should encourage a more realistic and equitable set of relationships among the Mission, the NGO's, the GOE, and perhaps others (e.g., the World Bank). Based on its track record so far in Ethiopia, Pact and other qualified grantees should be included among those considered to manage such a new effort. Barring any local impediments (or Agency regulations), the evaluators suggest that all groups considered be required to submit a proposal jointly with a registered Ethiopian NGO (or a suitable local consortium). The quality and practicality of each applicant's plan for indigenizing the new project should constitute a significant element in the award of the new grant. 28 Over the past half dozen years, many Ethiopian NGO's have become more viable and sophisticated, after being nurtured with significant institutional strengthening support from groups like Pact, the Ethiopian CRDA, and others. Many of these NGO's (including some already working in USAID's priority technical fields and regions) may have reached the point when they are ready for a new kind of relationship with USAID (if they are able to meet its need for relatively rapid and measurable results); the GOE (under the liberalized terms of a pending new NGO law); and the Ethiopian business community (which will hopefully soon be motivated to donate funds, property, and goods to non￾profit groups). Hence, at a suitable point AID would do well to actively seek a collaborative partnership that brings together these elements in a new thrust to encourage the community of non-governmental groups to enrich and improve Ethiopia's development in a growing number of fields and areas. Such a new project could continue to offer updated advanced institutional strengthening services to NGO's (which Pact previously did through ENSEI). Included would be courses cited as the most needed, such as board training, strategic planning, and bookkeeping/accounting. All bidders would also be required to address several additional elements in their proposals, i.e.: 1. Perform an inventory and review of the current universe of registered Ethiopian NGO's in each priority field of the Mission's operations; present plans for identifying and objectively selecting the target groups which seem the most viable; and define an approach to attract dues-paying members to the new project; 2. Grantee applicants who seek to manage the new effort would also include a plan to further enhance public understanding of, and support for, the work of Ethiopian NGO's. Specifically, ENSEI's promotion of an "enabling environment" would be carried several steps further, as the new project embarks on a new public relations activity that would more aggressively publicize the noteworthy, but little known and often misunderstood, accomplishments of the Ethiopian non-profits (both 29 past and present). This campaign would emphasize information for the public about the many important ways that NGO's are contributing to a better life for them personally, as well as generally helping the nation's development and its fight on poverty. If the long-anticipated new NGO law is soon passed (as most believe it will be), and the new project succeeds in getting established after a few months, the next step would be to help Ethiopian NGO's to boldly embark on a path towards greater financial sustainability by adding still another element. The new law should permit NGO's to actively seek income sources for their own support, and trained advisors from the new project would be there to help a selected group of them launch a serious trial fund-raising campaign at the national and provincial levels. 3. In a noteworthy departure from the original project design, would-be grantees for the new project would also address the following. While registered as a non-profit organization, the new project would be run on more business-like principles. Its Ethiopian NGO clients would pay a fee-for-service for their initial organizational diagnostics (cf. Pact's widely-admired OCAT); for popular support services like ENSEI's Development Resource Center; as well as for training, mentoring, and other Pact-spawned NGO strengthening activities that research then indicates are in greatest demand. A sliding fee scale might be applied, probably based on factors such as the longevity and financial status of each qualified NGO "customer" seeking help. In addition, a continuing educational, public relations and information campaign would be carried out by the new project management on behalf of the whole E/NGO community. This more targeted campaign would be financed by fees or dues paid into the project by the NGO affiliates, and perhaps with funds supplied by other donors interested in Ethiopian NGO's. If this is not immediately feasible, funds from the new project's start-up budget might be used at first. 30 Some final words about a new project. If funding can be found to implement this kind of activity, many questions (legal, financial, administrative, political/governmental, etc.) undoubtedly should be further explored. However, if the will is there, the evaluators believe an excellent chance exists to find ways around local obstacles; USAID would be able to protect its formidable past investment in strengthening Ethiopian NGO's; and the most fruitful results from the ground￾breaking Pact ENSEI project could be extended and multiplied, as a new generation of development-oriented NGO's in Ethiopia is assisted to move closer to the "holy grail" of sustainability and self-support. Whether or not a new NGO support program becomes part of the Mission's future portfolio, the evaluators propose that USAID and Pact to take two additional steps as soon as possible: · Most of the current NGO partners working with the Mission's sectoral specialists (e.g., FHI and Pathfinder in HPN) know Pact and its E/NGO partners, and some cooperation is already taking place between them. However, the evaluators believe that these sectoral groups should be asked to start immediate discussions with Pact about how they might provide additional support to worthy partner groups previously helped by Pact, at the same time as the U.S. NGO's carry out their work on behalf of the Mission S.O.'s (e.g., subcontracts, small grants, technical assistance, etc.). · During ENSEI's likely no-cost extension period, Pact should be asked to mount a more serious and comprehensive effort than it has so far done to provide individual consultation for all its partner groups that are urgently seeking effective ways to raise additional funds and other resources for their survival and sustainability. The evaluators spoke with several impressive E/NGO leaders about successful initiatives they have 31 undertaken that resulted in donations of land, buildings, labor and commodities, as well as money. Some were inspired by Pact in these efforts, while others were not. As ENSEI now winds down, the evaluators urge that these experiences be spotlighted as Pact now makes every effort to provide a final, comprehensive consultation and training effort to help insure the future survival and prosperity of its E/NGO partners. Fund raising know-how is crucial to the success of all Ethiopian NGO's, and to the very survival of many. ENSEI EVALUATION MAARD Page 1 of 17 Annex I Scope of Work Final evaluation of Ethiopian NGO Sector Enhancement Initiative (Cooperative Agreement No: 663 - 0020 – A – 00 - 5501 - 00) I Background The Ethiopian NOG sector Enhancement Initiative (ENSEI) was designed as a direct contribution to USAID/Ethiopia’s Strategic Objective No. 4, “ Increased Access to and Participation in a Democratic System.” Under USAID/Ethiopia’s Integrated Strategic Plan for FY 2001 _ 2006, a new democracy and Governance Strategic Objective, “More Effective Governance and Civil Society Developed,” subsumed ENSEI under the Intermediate result “ Civil Society Participation Increased.” This program design was provided in an unsolicited proposal by Pact which met the Strategic Objectives of the Mission. ENSEI is being implemented by Pact/Ethiopia under Cooperative Agreement (CA) No: 663_0020_A_00_5501_00. USAID/Ethiopia signed the CA with Pact in July 1995. After registration delays, Pact began its activities in May 1996. The CA was subsequently amended to reflect changing USAID priorities and funding realities. The ultimate completion date of the CA, as amended, is September 30, 2002, with authorized Length of Project funding of $12, 316,347. ENSEI seeks to strengthen the Ethiopian community of non - governmental organizations (NGOs). The purpose of the project is to strengthen and improve the institutional capacity of NGO and umbrella/ support organizations in Ethiopia to address the needs of targeted sectors: rural development and food security; democracy and governance/ human rights; education; street children and orphans; and health. The key objectives (numbered) with anticipated results(bulleted) from Pact’s unsolicited proposal of ENSEI are: 1. Improved enabling environment for NGOs, with the following Sub - IRs: ß Development and implementation of a transparent legal and regulatory framework supportive of NGOs creation and operation; ß Routine consultation between government and NGOs on legal and regulatory issues affecting NGOs; ß Increased civil society participation in public policy formulation and review; and ß Communication and building consensus among the different sectors of society regarding the contributions of the NGO sector to development and on the need for supportive legislative and regulatory frameworks. ENSEI EVALUATION MAARD Page 2 of 17 2. Improved NGO capacity for service delivery with the following Sub-IRs: · improved NGO planning and management capacity; · civil society organization networks increasingly effective; · overall improved financial management capacity of NGOs; and · strengthened capacity of NGOs in strategic planning and management. 3. Increased civil society participation in public policy formulation and review. 4. Formal and informal umbrellas/networks established, with the following Sub-IRs: · NGOs receive support services through umbrellas and networks; and · NGOs understand the role and value of umbrellas and networks. Indicators designed by ENSEI to measure results include: 1. Number of NGOs registered; 2. Number of strategic plans in place; and 3. Number of financial systems/procedures put in place. Description of support which ENSEI provides for Ethiopian NGOs includes: · training and mentoring in strategic planning and management, participatory project design, monitoring and evaluation, and financial management; · technical assistance; · exposure tours to other countries to share experiences; · access to a resource center; and · strategic action grants in the form of personnel secondment (e.g. project officers and financial officers), commodities and small activity grants up to US $50,000 per project. This support is principally oriented to improve the enabling environment for NGOs operating in the country and to enhance service delivery skills and advocacy capacities of the NGOs. Pact started by supporting 55 local NGOs using its Organizational Capacity Assessment Tool (OCAT) to determine the type and level of organizational needs and support. Pact has continued the development of NGO capacities using training and mentoring strategies with partners, and the identification of opportunities for partners to practice knowledge and skills through the use of strategic action grants. Pact links its capacity building interventions with three types of strategic action grants: commodity, personnel and activity. Pact has assisted more than 85 partner NGOs, of which 30 have "graduated" from Pact services. At the same time Pact has worked with other NGOs, most notably the Christian Relief and Development Association network of some 171 Ethiopian and international organizations, and various government offices to improve the legal and policy environment within which NGOs operate in Ethiopia. Pact was, for example, a key player in the formulation and adoption of an NGO Code of Conduct by 160 Ethiopian NGOs in 1999. ENSEI EVALUATION MAARD Page 3 of 17 II. TITLE Activity Title: Final Evaluation of Ethiopian NGO Sector Enhancement Initiative (ENSEI) III. OBJECTIVES The objectives of the final evaluation are to: 1. Assess ENSEI results in relation to the goals and objectives specified in CA No. 663-0020- A-00-5501-00, as amended. 2. Assess the services which have been provided, and impact achieved for each of the three project objectives, i.e.: · improving the enabling environment; · building the capacity of local NGOs; and · promoting networking in the country and NGO network potential for sustainability. 3. Document lessons learned from ENSEI. 4. Formulate recommendations to USAID/Ethiopia concerning possible future support to civil society in Ethiopia. IV. STATEMENT OF WORK This is an independent evaluation of ENSEI for USAID/Ethiopia. The purpose of the evaluation is to assess impact on the institutional capacity of local NGOs and the specific results of the program. This will be done by reviewing actual versus planned progress toward achieving key expected results; identifying problems, delays and issues and the reasons for them; and making recommendations for future civil society support. The ultimate questions will then be: Is the current approach appropriate? Should individual Strategic Objective Teams which also work with NGOs take the lead in strengthening NGOs relevant to their specific sector or alternatively, should the program continue in its current manner as implemented by Pact? What are the advantages and disadvantages of each of these? The Mission is interested to gain insights on how best to stimulate the indigenous civil society sector to help achieve the Mission’s objectives. Have some sectors been more amenable than others to strengthening? Specifically, the evaluation should answer the following questions. ENSEI EVALUATION MAARD Page 4 of 17 A. Specific Assessment of ENSEI Objectives and Activities 1. Enhancing the enabling environment in Ethiopia: What have been the positive/negative results from ENSEI strategies to enhance the environment in which NGOs operate in the country? Were these strategies effective? What was Pact’s role in developing and negotiating the NGO Code of Conduct? What has been the impact of the NGO Code of Conduct? Has the environment changed as a result of Pact's involvement? 2. Building the capacity of local NGOs: How effective has the Organizational Capacity Assessment Tool (OCAT) been in analyzing capacity needs of local NGOs? Has the tool been useful for partners, and if so, in what ways? Has it had any negative effects? 3. Promoting networking: Have ENSEI strategies to promote networking been effective? Have networks been established as a result of Pact efforts? What are the roles of these networks and what impact have they had in policy change/development? 4. Strategic Action Grants (SAGs): Assess the utility of providing grants and their potential for sustainability. Was Pact’s Grants Administration Manual for local NGOs helpful, effective and efficient? Was the review process for awarding grants adequate? What was the impact of SAGs in building local NGO capacity? What lessons have been learned? 5. Exposure tours: Pact has used “exposure tours” as a means of promoting ENSEI objectives. What has been the impact of “exposure tours”? Have they been a useful and cost-effective technique? 6. Development Resource Center (DRC): What has been the impact of the DRC? How has it been helpful to the NGO community in Ethiopia? Are the services making a difference in NGO capacity? Is this activity critical to the achievement of the program's objectives? Is it sustainable? B. Indicative Issues to be Addressed and Questions to be Answered Overall Design and Implementation · Was the project proposal realistic or over-ambitious in setting and achieving objectives? · What has been the overall impact of ENSEI? · What are the most significant accomplishments? Failures? Obstacles? · Have the strategies used for each objective promoted the overall purpose of ENSEI? Pact/Ethiopia Project Management and USAID/Ethiopia Project Management · Was Pact management structure adequate and effective? · Has the substantial involvement clause as stated in the CA been effective in the management of the program and has it promoted effective implementation of the program? · Have communications and relations with the Mission been effective in terms of achieving results? · With regards to Pact’s Monitoring, Evaluation, Reporting, Information and Tracking (MERIT) system, was it functional in terms of reporting appropriate information adequately ENSEI EVALUATION MAARD Page 5 of 17 to assess ENSEI progress and impact? Have reporting requirements been met to both USAID/Ethiopia and the Government of Ethiopia? Broader Issues · To what extent have expected results been achieved? How have the accomplishments contributed to the achievement of the intermediate result? · Given the change of funding status from D/G funds to sectoral office funds within USAID/Ethiopia Mission in 2001, what impact, if any, did the change of program structure have on the overall program? Partners? Anticipated results? · Building on what has been accomplished and identifying lessons learned, what recommendations can be drawn for future USAID civil society support? It will be important for the contractor to discuss ENSEI accomplishments and continuing challenges within the context of overall civil society development in Ethiopia. Underlying issues, including the following, will need to be assessed and answered: · How has civil society developed since the signing of the CA? · What major obstacles have existed and which have been overcome? · What are the principal advances that have been made by CSOs that do advocacy focusing on citizen participation? · How has ENSEI contributed to the relationship between civil society organizations and the Government? · In general, to what extent has ENSEI contributed to NGO enabling environment? · What are the perceived needs and priorities of civil society in the future? Within the general context of civil society development in Ethiopia, the contractor will consider the following issues specifically related to ENSEI: · How has ENSEI contributed to improvement of civil society's service delivery capacity? · How have the Strategic Action Grants enhanced the capacity of NGOs? · What has been the impact of institutional strengthening for NGOs? To what extent has the ENSEI helped NGOs ensure their financial sustainability? · How effective and responsive have the technical assistance (TA) and training and mentoring programs been to NGOs' needs? Specifically, what has been the quality and impact of technical assistance and training provided to the NGOs? Was appropriate TA provided to sub-grantees? Will the TA and training program have a lasting effect? Has there been effective and adequate follow-up to ensure this? · Did ENSEI achieve the expected results? What remains to be done to promote the sustainability of local NGOs? What has been achieved in formation and functions of NGO networks? · What other results, expected or unexpected, were achieved under ENSEI? For example, What was the impact, if any, in changing the views of the Ethiopian Government, their beneficiaries and the general public toward the roles of civil society organizations? · What were the key problems that affected ENSEI? · What are lessons learned? ENSEI EVALUATION MAARD Page 6 of 17 Cost-effectiveness and Sustainability · How cost-effective is ENSEI? How, for example, do NGO capacity building activities carried out by Pact under ENSEI compare with similar activities focused on one sector by Pathfinder? · How sustainable are ENSEI results? Recommendations · What actions should USAID consider regarding future efforts to support strengthening and development of civil society in Ethiopia? · What factors should USAID take into consideration in planning future activities, if any? V. LEVEL OF EFFORT AND TIMING The evaluation requires two consultants, preferably one international and one local. (See criteria for evaluating proposed consultants' qualifications in Attachment A). It is anticipated that the contractor will conduct the evaluation in four weeks in Ethiopia, with the equivalent of a total of up to one additional week (which is expected to include meeting officials in Pact Head quarters); pre-field preparation and post-field finalizing the report. A six-day workweek is authorized in Ethiopia. The evaluation is to begin o/a March 18, 2002 and end four weeks after commencement. It is expected that the time will be allocated as follows: · Week One: initial meeting with USAID/Ethiopia Front Office and Democracy and Governance Office; initial meeting with Pact/Ethiopia; submission and approval by USAID of detailed work plan and methodology before the end of the week; review relevant documents (see Section VII below); commence interviews with USAID Strategic Objective Teams, Pact/Ethiopia, relevant USAID/Ethiopia partners, government officers, individuals and NGOs in Addis Ababa; oral status report to USAID Democracy and Governance Office at end of the week. · Weeks Two and Three: Complete interviews in Addis Ababa and the regions; commence drafting report; oral status reports to USAID Democracy and Governance Office at end of each week. · Week Four: By COB Wednesday of week four present a preliminary draft report and debrief USAID Mission staff. Receive comments and incorporate into revised draft before departure. VI. METHODOLOGY It is expected that the evaluation will be firmly based on relevant documentation and interviews with relevant individuals. A list of documents and publications is provided in Section VII below. These will be made available to the contractors at the beginning of their assignment by the USAID Democracy and Governance Office. ENSEI EVALUATION MAARD Page 7 of 17 Individuals who should be interviewed include: USAID/Ethiopia Front Office Democracy and Governance Office Contracts Office Health, Education and Agriculture and Natural Resources Strategic Objective Team leaders Pact/Ethiopia Co-Directors Leslie Mitchell and Tsegaye Chernet Other Pact staff and consultants to be identified in consultation with Pact Pact partner NGOs At least 25 representatives of Pact partner NGOs, selected to ensure coverage of all sectoral areas of focus, to include a substantial number of NGOs from at least two regions outside of Addis Ababa, and to include three NGO umbrella organizations. The contractor will work closely with the USAID Civil Society Coordinator in choosing Pact partner NGOs for interview to ensure balanced coverage, including exposure to organizations with negative as well as those with positive experiences of Pact. The USAID Civil Society Coordinator, who serves as CTO for ENSEI, may accompany the contractor to various organizations in Addis Ababa and in the field. Where appropriate, NGO interviews may be combined in focus groups. Non-Pact partner NGOs Government officials Registrar of Associations in the Ministry of Justice Vice Minister in the Ministry of Justice Disaster Prevention and Preparedness Commission (DPPC) Regional officials to be identified in consultation with Pact and NGOs Other donors The USAID Democracy and Governance Office will assist with the scheduling of interviews. VII. AVAILABLE INFORMATION SOURCES The following is an illustrative list of documents to be made available to the contractor: 1. Pact Proposals 2. ENSEI Cooperative Agreement and amendments 3. Pact Annual Work Plans 4. Pact Progress Reports ENSEI EVALUATION MAARD Page 8 of 17 5. Pact Enabling Environment Assessment reports 6. Pact Organizational Capacity Assessment Tool (OCAT) 7. Other Pact publications, including quarterly Impact 8. Pact's past evaluation reports - mid term evaluation, internal evaluation and most recent mpact evaluation of 2000 9. USAID/Ethiopia Strategy documents 10. USAID R2 documents 11. USAID CTO files 12. Recent reports on NGO Sector in Ethiopia, for example: Jeffrey Clark, Civil Society, NGOs, and Development in Ethiopia: A Snap Shot View, World Bank, 2000. VIII. DELIVERABLES Work Plan: The contractor will submit a detailed written work plan which is to be approved by USAID by the end of Week One of work. Preliminary Draft Report: By the close of business on Wednesday of Week Four of work the contractor will submit ten (10) copies of a preliminary draft written report to USAID. Following submission of the draft report, meetings should be held with USAID/Ethiopia, Pact staff and other relevant persons/organizations to review evaluation findings and recommendations. The evaluation team shall take into consideration comments, issues and questions raised at the initial review meeting in revising the draft report. Final Draft Report: The contractor shall prepare and send to USAID/Ethiopia fifteen (15) copies of a final draft report within two weeks after the evaluation ends. USAID and Pact comments on the final draft report will be sent to the contractor within three weeks of its receipt by USAID. Final Report: Two weeks after receipt of USAID's comments, the contractor shall deliver the final report to USAID/Ethiopia. The report becomes the property of USAID/Ethiopia. Fifteen (15) hard copies and an electronic file version in Microsoft Word 97 must be submitted to USAID/Ethiopia. The contractor will not circulate or distribute any report until formal approval is received from USAID. The final report shall be organized in the following manner: An Executive Summary not more than three pages in length. It should include a statement of conclusions and recommendations on the objectives listed under Section III above. The body of the report shall describe the background, objectives and methodology of the assessment, findings under the objectives listed in Section III above, and recommendations on how USAID/Ethiopia might enhance its contributions to CSO strengthening and promoting community participation and interests. In sum, the body of the report should comprehensively and separately address the issues listed in Section IV above. If the written review of the final draft report by Pact and/or USAID raises issues which the contractor is unable or unwilling to address in the final report, such issues shall be contained in a written addendum reproducing the language of the written review, together with a brief statement from the contractor explaining why the issue was not addressed in the final report. ENSEI EVALUATION MAARD Page 9 of 17 Annexes: The report will have annexes including a list of relevant individuals and organizations consulted and documents reviewed. IX. RELATIONSHIPS AND RESPONSIBILITIES The contractor shall perform the specific tasks described above under the general guidance of the USAID/Ethiopia Democracy and Governance Advisor, James Polhemus, and USAID/Ethiopia ENSEI CTO, Getahun Dendir. X. DUTY POST/LOCATION The Duty Post for this task is Ethiopia/Addis Ababa and pertinent Ethiopian regions where USAID/Ethiopia and Pact partner organizations and NGOs operate. XI. LOGISTIC SUPPORT In-country transportation and accommodation will be the contractors’ responsibility. The USAID/Ethiopia Democracy and Governance Office will assist with setting up appointments. The contractors will be responsible for their own secretarial, information technology, and printing requirements. XII. METHOD OF PAYMENT USAID/Ethiopia Regional Contracting Officer (RCO) will determine the type of contract to be negotiated. The method of payment will depend on the type of contract negotiated between the RCO and the contractor. ENSEI EVALUATION MAARD Page 10 of 17 Attachment C CRITERIA FOR EVALUATING PROPOSED INTERNATIONAL CONSULTANT Education Appropriate higher degrees, for example Masters or Doctoral qualifications in social science. A doctoral degree is preferred for the international consultant and a Masters degree for the local Consultant. Required Experience · Ten to fifteen years project management experience related to NGO institutional development. · Program design. · Project evaluation. · Good knowledge of the NGO sector management and issues around the sector. · Ten to fifteen years of field experience in working with and for donor organizations. · Knowledge of and working experience in Africa, especially in Ethiopia. · Demonstrated track record of timely delivery of work to standards acceptable to USAID. Desired Skills Knowledge of some level of Amharic language is highly advantageous. Knowledge of Ethiopian NGO sector. Work experience with USAID and knowledge of its systems is highly advantageous. Annex 2 Pact Evaluation List of People Interviewed NGOs 1 Ato Jemal Abdela, Executive Director Birhan Integrated Community Development Organization (BICDO) Addis Ababa, Ethiopia. 6 Ato Kebede Asrat Executive Director Christian Relief and Development Association(CRDA) Addis Ababa, Ethiopia 2 Ato Sahleselassie Abebe, Executive Director Center for Local Capacity Building and Studies (CLCBS) Addis Ababa, Ethiopia 7 7. Ato Mengiste Ayele, Network Coordinator Orphans and Vulnerable Children Network (OVC) Addis Ababa, Ethiopia 3 Dr. Wolday Ameha, Executive Director Association of Ethiopian Micro finance Institution (AEMFI) Addis Ababa, Ethiopia 8 8. Ato Tadesse Aynalem President Dawn of Hope Ethiopia Addis Ababa, Ethiopia 4 Mr. Tarekegn Arrarso NGO Consultant Addis Ababa, Ethiopia 9 9. Ato Tessema Bekele, Executive Director Emmanuel Development association(EDA) Addis Ababa, Ethiopia 5 Ato Zegeye Asfaw, Executive Director HUNDEE Addis Ababa, Ethiopia 10 10. Dr. Costantinous Berhe Vice president Africa Humanitarian Action Addis Ababa, Ethiopia 11 Ato Kassahun Chekole Program Officer Micro Enterprise Coordinator Addis Ababa, Ethiopia 16 Dr. Bogalech Gebre, Executvie Director Kambatta Women’s self - Help Center Ethiopia (KMG) Addis Ababa, Ethiopia 12 Ato Berhanu Geleto Executive Director Rift Valley Children & Women Development Association Arsi Negele, Ethiopia 17 Dr. Tilahun Dafurso Executive Director Ethiopian Evangelical Church Mekane Yesus – South Synod (EECMY) Awassa, Ethiopia 13 Ato Mogus Gorfe Executive Director RATSON Debereziet, Ethiopia 18 Ato Achalu Beyene Manager Forum on Street Children Ethiopia (FSCE) project Awassa, Ethiopia 14 Ato Sendeku Guadie General Mange Bahir Dar Medhaialem Orphans and Destitute family support Organization Bahir Dar, Ethiopia 19 Ato Genetu Anteneh, Deputy Manager Amhara Development Association Bahir Dar, Ethiopia 15 Ato Mekonnen Heshe Project Manager Jerusalem Association Children’s Home Bahir Dar, Ethiopia 20 Ato Shawel Debebe Project Manager Mary Joy Aid Through Development Awassa, Ethiopia 21 Ato Bekele Hambissa Executive Director Ethiopia Environment NGO Addis Ababa, Ethiopia 26 Ato Alemayehu Fulas Project Officer Dawn of Hope Ethiopia (DHE) Addis Ababa, Ethiopia 22 Ato Kebede Kejela, Executive Director African Initiative for a Democratic World Order (AIDWO) Addis Ababa, Ethiopia 27 Ato Mulugeta Gebru, Managing Director Jerusalem Association Children’s Home(JACH) Addis Ababa, Ethiopia 23 Wzo. Berhane Kelkaye Coordinator Dawn of Hope Ethiopia (DHE) Awassa 28 Wzo. Seblewengel Lema Brach Coordinator Organization for Social Services for AIDS (OSSA) Awassa 24 Ato Fassil W/Mariam, Executive Director Forum on Street Children Ethiopia (FSCE) Addis Ababa, Ethiopia 29 Ato Solomon Legesse, Project officer Addis Development Vision Addis Ababa, Ethiopia 25 Ato Ayalew W/Semaite, Executive Director African Network for the prevention of and protection Against Child Abuse and Neglect (ANPPCAN) Addis Ababa 30 Wzo. Beletu Mengistu Executive Director Integrated Service for AIDS Prevention and Support Organization (ISAPSO) Addis Ababa, Ethiopia 31 Ato Mamo Kebede Shenkut, Executive Director Basic Education Network(BEN) Addis Ababa, Ethiopia 35 Haji Adane Mamuye, General Secretary Ethiopian Muslim’s Relief and Development Association (EMRDA) Addis Ababa, Ethiopia 32 Wt. Konjit Worku Project Officer Integrated Service for AIDS prevention and support Organization(ISAPSO) Awassa, Ethiopia 36 Ato Habtamu Mamo Program officer Abebech Gobena Orphanage and School _ AGOS Addis Ababa 33 Mr. Helawi Yoseph Chirman of the Board Amhara Development Association(ADA) Bahrdar, Ethiopia 37 Ato Yoseph Negassa, Executive Director Action for Development (AFD) Addis Ababa, Ethiopia 34 Dr. Dawit Zawde President Africa Humanitarian Acton Addis Ababa, Ethiopia 38 Dr. Teffera Wogderesse Executive Director LEM Ethiopia Addis Ababa, Ethiopia Pact/Ethiopia 39 Ms. Leslie Mitchell Director Pact/ |Ethiopia Addis Ababa, Ethiopia Mr. Muluneh Sugebo Grant Manager Pact/ |Ethiopia Addis Ababa, Ethiopia 40 Ms. Sarina Prabsi Interim country Representative Pact/ |Ethiopia Addis Ababa, Ethiopia 42 Ms. Mulumebet Jembere Finance and Manger Pact/ |Ethiopia Addis Ababa, Ethiopia 41 Ms. Aster Birke Program Manager Pact/ |Ethiopia Addis Ababa, Ethiopia 43 Mr. Kederallah Idris Program Manager Pact/ |Ethiopia Addis Ababa, Ethiopia USAID/Ethiopia 44 Ms. Mary Lewellen Director USAID/Ethiopia Addis Ababa, Ethiopia 48 Mr. Jim Polhemus Governance/ Democracy Advisor USAID/ Ethiopia Addis Ababa, Ethiopia 45 Ms. Anne Nolan HIV/AIDS prevention & control USAID/Ethiopia Addis Ababa, Ethiopia 49 Mr. James Meyer D.C.M USAID/Ethiopia Addis Ababa, Ethiopia 46 Mr. Kurt Rokema Chief Agriculture sector USAID/Ethiopia Addis Ababa, Ethiopia 50 Mr. Kevin Mullaley Chief Human and Institutional Development USAID/Ethiopia Addis Ababa, Ethiopia 47 Mr. Getahun Dendir Chief Technical Officer Human and Institutional Development Sector USAID/Ethiopia 51 Mr. Mekonen Ashenafi Evaluation Officer USAID/Ethiopia Addis Ababa, Ethiopia Government of Ethiopia 52 Mr. Taddese Kassa Minister Ministry of Capacity Building Amhara Regional Government BahrDar, Ethiopia 54 Mr. Ali Suleiman Vice Minster Ministry of Justice Addis Ababa, Ethiopia 53 Mr. Getachew Gonfa Chief of Registration Office Ministry of Justice Addis Ababa, Ethiopia 55 Ato Asfaw Mekuria,, Head, NGOs Co_ ordination office Disaster Prevention and Preparedness Commission (DPPC) Addis Ababa, Ethiopia Pact Evaluation ANNEX 3: BIBLIOGRAPHY Pact Ethiopia Publications Newhall, Sarah; Clark, Jeffrey; and Abdel - Latif, Jalal. 1998. Report on the Enabling Environment for the Ethiopian NGO Sector. Clark, Jeffrey; Loughran, Lizbeth; and, Bekele, Daniel. 2000. Report on the Enabling Environment for the Ethiopian NGO Sector. 2000. Clark, Jeffrey. Civil Society. NGOs, and Development in Ethiopia: A Snapshot View. The World Bank, Washington, D.C. 1999. Code of Conduct for NGOs. Addis Ababa, Ethiopia. Source unknown. 1999. NGOs Legal Framework Consultative Working Group. Position Paper on the Legal Framework for NGOs in Ethiopia (Third Draft). Addis Ababa. Source unknown. August 2001. Ministry of Economic Development and Cooperation. Ethiopia: Plan of Action for the Formation of Poverty Reduction Strategy paper (PRSP). Final Draft. PRSP Secretariat. Addis Ababa. February 2000. Norton, Steve. Mid - Term Impact Evaluation of the Ethiopian NGO Sector Enhancement Initiative Project. Williamson, John. March 2000. Assessment of the Street Children and Orphans Component of the PACT NGO Sector Enhancement Initiative in Ethiopia. Professional Resource Group Intl., Washington D.C. December 2001. The Ethiopian NGO Sector Enhancement Initiative Semi Annual progress Report (July - Dec. 2001). January 2001. The Ethiopian NGO Sector Enhancement Initiative, Technical Proposal, Oct 1, 2002 - September 30, 2003. January - June 2001. The Ethiopian NGO Sector Enhancement Initiative. Semi - Annual Progress Report. Feb 1998. NGO Sector Enhancement Initiative Internal Evaluation. May, 1998. Report on the Enabling Environment for the Ethiopian NGO Sector. November 2000. A Consolidated Report on the Reassessment of 29 Ethiopian NGO’s. February 2002. Report on the Enabling Environment for the Ethiopian NGO Sector . June 2000. Introduction to Participatory Project Design, Monitoring, Evaluation and Reporting. March 2002. Assessment to Review the Impact of NGO Supported Credit Programs on the Standard of Living of Children. July 1997. Assessing Organizational Society Though Participatory Monitoring and Evaluation Handbook. March 2002. Development Resource Center Impact Evaluation Report. February, 2000. NGO Sector Enhancement Initiative - Impact Evaluation Report on Exposure Visits. June 2000. Introduction to Participatory Project Design, Monitoring, Evaluation and Reporting - Part I. April 2002. Report on Impact Assessment of the Board Development Training Program. February 2002. Impact Evaluation report on PACT Ethiopia: Training in Conflict Prevention and Management. June 2002. Evaluation Report on Impact of OCA/REOCA Process. June 2002. Comprehensive Briefing Document Package for Final Evaluation. August 1998. A Manual on Strategic Planning and Management. June 2002. Assessment of SPPPM Training and Mentoring Impact. OTHER DOCUMENTS 1. Annual Report - Forum on Street Children - Ethiopia (FSCE) – 2001. 2. Ethiopian Micro Finance Institutions Study on the Impact of Micro - Finance Services Among Poor Women (Association of) - April 2002. 3. USAID/ Ethiopia Cooperative Agreement ENSEI, USAID/Ethiopia, July 1995. 4. USAID/Ethiopia, “ Integrated Strategic Plan”, FY 2001- 2006. 5. International Forum on Capacity Building : Strengthening Southern NGOs - the Donor Perspective, Vol. 1, May 1998, Washington D.C. 6. International Forum on Capacity Building: Future Capacity Building of Southern NGOs, 1998, Brussels, Belgium. 7. 2000. Clark Jeffery: Civil Society NGOs and Development in Ethiopia. A snapshop view, The World Bank, Washington, DC. Annex 4 NGOs Core Training and Grants Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark A- BU-GI-DA- Ethiopian Congress for Democracy a a a a a a a a De – registered Aba Wolde Tensae Gizaw’s Mohters and Children Welfare association a a a a a a a a a Abebech Gobena Orphanage and School a a a a a a a a a Action for Development a a a a a a a Action professionals’ association for the People a a a a a - DG, no fund, no active projects Addis Development Vision(VCH) a a a a a a a a a Adult and Non – formal Education Association in Ethiopia a a a a a a a a a African Development Aid Association a a a a a a a a a African Initiative for a Democratic World Order a a a a a a a a Africa Network for the prevention of & protection Against Child Abuse & Neglect a a a a a a a Agri Service Ethiopia a a a a a Agriculture NGO, no active project Alem Children Support Organization a a a a a a a a Almaz Children & Family Support Organization a a a a a a Amhara Credit & Saving Institution a a a a a a a a a Amhara Development Association a a a a a a a a a Amhara Women’s development Association a a a a a a a a a Association of Ethiopian Micro – Finance Institution a a a a a Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark Bahir Dar Medhanialem Orphans and Destitute Family Support Organization a a a a a a a Discontinued partnership Berhane Hiywot Children's Village & Family Service Organization a a a a a Birhan Integrated Community Development Organization a a a a a a a a a Center for Development Initiative a a a Agriculture NGO, no active project Center for Local Capacity Building and Studies a a a a a a a DG, no fund, no active project Charity Development Organization a a a a a a a a Agriculture NGO, no active project Children Aid Ethiopia a a a a a a a a Civic Education Ethiopia a a a DG, no fund, no project Community Road Development Organization a a De-registered Dawn of Hope Ethiopia a a a a a a a a Development Action for Oromiya a a a a a a Development Initiative Promotion and Sustainability Assuring Program of Ethiopia a a a a a Emmanuel Development Association a a a a a a a a a Environmental Protection and Assistance Organization a a a a Agriculture NGO, no active project Ethiopian Aid a a a a a a a Ethiopian Environmental NGO a a a a a a a a a Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark Ethiopian Evangelical Church Mekane Yesus-South Synod a a a a a a a Ethiopian Evangelical Church of Mekane Yesus-Western Synod a a Ethiopian Muslim's Relief & Development Association a a a a a a a a a Ethiopian Orthodox Church/Development Inter Church Aid Commission a a a a a a a a Ethiopian Pastoralist Research & Development Association a a Ethiopian Rural Self Help Association a a a a a a a a Agriculture NGO, no active project Ethiopian Women's Lawyers Association a a a a a a Family and Children Integrated Development Foundation a a a a a a a Forum on Street Children Ethio-pia a a a a a a a a a Gadissa Association a a De-registered Gasha Micro Finance S.C. a a Godanaw Rehabilitation Integrated Project a a a a a a a a a Guraghe People Self-Help Development Organization a a a a a a a a a Harari Relief and Development Association a a a a a a a a a Hope for the Horn a a a a a a a a Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark HUNDEE-Oromo Grassroots Development Initative a a a a a a a Integrated Family Service Organization a a a a a a a a a Integrated Holistic Approach Urban Development Project a a a a a a Integrated Service for AIDS Prevention and Support Organization a a a a a a a a a Jerusalem Association Children's Home a a a a a a a a a Kambatta Women's Self-Help Center Ethiopia a a a a a a a a a Kangaroo Child and Youth Development Society a a a a a a a a Kind Hearts Children Aid Development Organization a a a a a a a a a Land Mine Survivors Network a LEM-Ethiopia a a a a a a a a a Mary Joy Aid Through Development a a a a a a a a a Meklit Micro Finance Institution a a Meserete Kristos Church Relief and Development Association a a a a a a a a a Multi Purpose Community Development Project a a a a a a a National Committee on Traditional Practices of Ethiopia a a a a a a a Nazareth Children's Center and Integrated Development a a a Nile Children and Family Support Organization a a a a Ogaden Welfare Society a a a a a a a a a Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark Terepeza Development Association a a a a a a a Unity College a a Venus United Humanitarian Organization a a a a a a a De-registered Wollo Development and Rehabilitation Association a a a a a a Women Aid Ethiopia a a a a a a a Women and Children Development Organization a a a a a a a a a Women Educationalist Association a a a Women in Development Ethiopia a a a a a Women's Association of Tigray a a a a a Women, Youth and Children Development Program a a a a a a a a a Yeteem Children's and Destitute Mothers Fund a a a a a a a a Core Training Grant Name of NGOs PPDMER Financial Management & Peachtree Accounting Software SPM Others -Conflict Resolution -Domestic Resource Mobilization -Leadership -Board Development -Gender & Development -Health -TOT, facilitators training Project funding Computer & Furniture Personnel Project Finance & Internship Technical assistance -Baseline Survey - in house workshop Remark Organization for Rehabilitation & Development in Amhara a a a a a a a a Organization for Social Services for AIDs a a a a a a a a a Oromiya Development Association a a a a Agriculture NGO, no active project Oromo Self-Help Organization a a Agriculture NGO, no active project Oromo Self-Reliance Association a a a a a a a Agriculture NGO, no active project Pastoralist Concern Association Ethiopia a a a a a a a a Peace and Development Committee a a a a a a a Pro Pride a a a a a a a a PROGYNIST a a a a a a a a Rehabilitation and Development Organization a a a a Research Center for Civic and Human Rights Education a a a a a a a DG, no funds, no active project Rift Valley Children & Women Development Association a a a a a a a a Selam Environmental Development Association a a a a a a a Agriculture NGO, no active project Society for the Advancement of Human Rights Education a a a a a a a a South Ethiopia Peasant Development Association a De-registered South Ethiopia People's Development Association a a a a a a Specialized Financial and Promotional Institution a a a a a a a Annex 5 Pact comments on USAID ENSEI Evaluation Overall there are several points raised in the report that could benefit from further elaboration and could be clearly related to the impact on program implementation and results achieved. For example, concern has been raised over lack of ‘strong leadership’, how did this impact the program? What could have been done differently? How pervasive was this view? Throughout the report, it would be helpful to the reader to understand and differentiate between the types of information points are based on, i.e, document review and confirmed by interviews, widespread perceptions, limited perceptions? 1. Sector activities · The evaluation report does not cover Pact’s work in the major sector areas of health, education, agriculture/microfinance, orphans and vulnerable children. We believe substantial impacts have been achieved in this area that are not reflected in the report. This is especially important, as even prior to the restructuring of the program and sector-based funding, Pact’s program from the start had sector areas of focus. 2. Partner selection · The report states there has been “lack of transparency” in partner selection. While we believe this is incorrect, it would be helpful to get an understanding of the basis of this finding, and how prevalent this view was in the interviews conducted. Is there a percentage of respondents that reflected this view, what specifically was not transparent? Pact has provided the consultants with the criteria used in the selection of NGOs as its local partners. These criteria were developed by Pact with significant partner involvement, and were widely disseminated to all NGOs interested to apply as partners in Pact’s ENSEI Project. The criteria reflected program and sector objectives. 3. Adapting to changes in the Mission · The opinion of evaluators that Pact was delayed in adapting to changes in the Mission program funding is not correct. In fact, Pact was able to respond in a two-week time frame and submit a proposal that realigned its program to reflect the mission’s sectoral priorities. In addition, prior to receiving sector office funding, Pact was already involved in priority sector areas. Pact would like to clarify that the funding from sectoral offices has made the program more focused and that it fully appreciates the need to respond to changing mission priorities. 4. Exit strategy Most of Pact’s 86 NGO partners are not dependent solely on Pact for funding, rather it is Pact’s intensive capacity building training and mentoring that is the unique contribution of the ENSEI project. As a result of intensive capacity building inputs from Pact, most partner NGOs have a diverse array of donors. As part of the ENSEI program, Pact has already been undertaking a range of activities that strengthen NGO skills and facilitate the transition for NGOs at the end of the ENSEI program. These include: · Local resource mobilization · Participatory project design, implementation, evaluation and reporting · Project budget preparation · Leadership, boards and governance · Linking partners with other donors, for example, with the World Bank/National AIDS Council, where Pact partners who have completed the capacity building program were linked with the National AIDS Council to receive grant funds. Progress on NGO sustainability has been challenging due to the legal environment governing NGO operations and the lack of clarity in legislation regarding NGO use of mechanisms such as fee for service and income generation. However, Pact believes there has been considerable impact in terms of sustainability of the program’s impact in the increased skills and capacity of NGOs to deliver high quality services and access a variety of donor resources. 5. Reporting · It is unfortunate that Pact has never received feedback that its reporting has been inadequate, and is hearing this for the first time in the present final evaluation of the program. Pact has always responded to the Mission requests and at various stages of the program requested standard reporting guidance. Pact has also sought more measurable and quantifiable indicators over the last few years, and communicated with the mission that the indicators it was reporting on did not fully capture the breadth of the program’s results. Annex 6 Evaluation Team's comments on Pact feedback After reading through our evaluation again, as well as the comments from Pact, here are the evaluation team's views: 1. We do report on all of Pact's sector activities, implicitly or explicitly. Indeed, with very few exceptions, Pact drafted (for USAID's approval) the list of local NGO's, government officials, NGO networks, and other people we saw. We discussed with these people activities in each of the fields (sectors) in which the ENSEI project worked. I also believe we fairly assessed and reported on the information we obtained (in Addis and the field) in all the areas covered by the project, even if we did not enumerate each of the fields. 2. Certainly Pact had selection criteria for their 80-plus partners, and they shared them with us. We make no criticism of the criteria per se. However, the fair, unbiased, and open application of these criteria is the issue. We did hear accusations of decisions being made on the basis of favoritism or cronyism, and we reported this. We did not condemn Pact's entire selection process because of it; rather, we wanted to leave the impression the whole program would have benefited from greater transparency (before, during and after the selection process), because that is what we believed. 3. Re "adapting to changes in the Mission," once again as evaluators we only reported on what we heard from Mission technical offices. If the USAID officers were wrong in what they concluded (and we reported), it is beyond our scope to try correcting any erroneous impressions they may have had. 4. Exit Strategy. The evaluators both recognized and acknowledged Pact's efforts to prepare some (though not all) partner-recipients for a post-ENSEI world. However, the question here is whether more could have and should have been done, despite the constraints and difficulties that Pact cites. We determined that some partners had, on their own, devised and employed imaginative and effective ways to circumvent problems related to NGO fund raising and resource mobilization in Ethiopia. Once again, while recognizing the difficulties Pact cites (and the steps they did take), we concluded that Pact could have done more to prepare its partners for the end of ENSEI. 5. Re Reporting, the response is basically the same as in #3, above.