Final Evaluation: The Liberia Transition Initiative (2004-2006) Submitted by: Richard Blue, Ph.D Beverlee Bruce, Ph.D SOCIAL IMPACT, INC. HDA-I-23-03-00124-00 October 19, 2006 Social Impact, Inc Enhancing Development Effectiveness Social Impact, Inc. October 19, 2006 i TABLE OF CONTENTS ACRONYMS................................................................................................................................ iii PREFACE...................................................................................................................................... v ACKNOWLEDGEMENTS ........................................................................................................ vi MAP OF LIBERIA ..................................................................................................................... vii EXECUTIVE SUMMARY....................................................................................................... viii I. THE LIBERIAN CONTEXT............................................................................................... 1 The Conflict ................................................................................................................................ 1 OTI’s Mandate ............................................................................................................................ 1 II. PURPOSE OF THE EVALUATION.................................................................................. 2 Scope of Work ............................................................................................................................ 2 Time Frame ................................................................................................................................. 2 The Evaluation Team.................................................................................................................. 2 The Methodology........................................................................................................................ 3 Validity of the report findings, generalizations and lessons learned .......................................... 3 III. THE OTI PROGRAM IN LIBERIA: October 2003 to June 2006 .................................. 3 The OTI Liberia Transition Initiative Program: A Macro-level Overview................................ 4 IV. LTI YEAR 1: Findings and Intermediate Conclusions..................................................... 7 LTI Year 1: Grant Making Strategy and Implementation .......................................................... 7 Intermediate Conclusions.......................................................................................................... 11 LTI 1 Operational Effectiveness............................................................................................... 12 Summary for LTI 1 ................................................................................................................... 13 V. LTI 2 PROGRAM: YES Plus ............................................................................................ 13 Youth Education for Life (YES)............................................................................................... 13 YES Plus: Linking YES training to a LTI grant funded community project............................ 15 Timeline for Implementation of YES Plus ............................................................................... 17 Reintegration of war affected communities.............................................................................. 18 Targeting communities for YES Plus ....................................................................................... 18 Findings from Evaluation Field Visits: Conflict and reintegration .......................................... 19 YES Plus Projects ..................................................................................................................... 20 YES Project Impact and Problems Assessed ............................................................................ 20 YES Plus Intermediate Conclusions ......................................................................................... 21 VI. GENERAL CONCLUSIONS ............................................................................................ 22 VII. LESSONS LEARNED ........................................................................................................ 23 Selected Bibliography .................................................................................................................. 25 ANNEX 1: Scope of Work.......................................................................................................... 26 ANNEX 2: Liberian Context...................................................................................................... 31 ANNEX 3: Answers to SOW Part 2 .......................................................................................... 33 ANNEX 4: Interview List........................................................................................................... 35 ANNEX 5: Site Visits.................................................................................................................. 38 ANNEX 6: YES Question Guide ............................................................................................... 39 ANNEX 7: Program Timeline .................................................................................................... 40 EXHIBITS: Exhibit 1: Map of Liberia with Counties, # and Value of LTI Grants and # of IDPs……………...5 Exhibit 2: Comparison of LTI Grants by Program Objective……………………………..6 Exhibit 3: Comparison of LTI Grants by Focus Area….………………………………….6 Social Impact, Inc. October 19, 2006 ii Exhibit 4: Total Expenditures for LTI by OTI and IPs.…………………………………...7 Exhibit 5: Liberian NGOs Receiving Grants Valued at $30,000 or more, April 2004 to June 2006……………………………………………………………………..…...8 Exhibit 6: YES Module Completion Rate……………………………………………….14 Exhibit 7: Projects Closed or Completed by Type listed……………………………..…16 Exhibit 8: OTI LOE compared to UNHCR Estimate of # of Returning Refugees……...19 Social Impact, Inc. October 19, 2006 iii ACRONYMS ALP Accelerated Learning Program APU Abuse Prevention Unit CA (or CAII) Creative Associates International, Inc. CENTAL Center for Transparency and Accountability in Liberia CPA Comprehensive Peace Accord CSO Civil Society Organization CYPEP Community Youth Peace Education Program DART Disaster Assistance and Response Team DDRR Disarmament, Demobilization, Rehabilitation and Reintegration ECOMOG Economic Community Military Observer Group ECOWAS Economic Community of West African States FFP Food for Peace FIND Foundation for International Dignity FLY Federation of Liberian Youth GDB Grants Database GOL Government of Liberia HIV/AIDS Human Immunodeficiency Virus / Acquired Immunodeficiency Syndrome IDP Internally Displaced Person IP Implementing Partner IQC Indefinite Quantity Contract IRDO Integrated Rural Development Organization LF Learning Facilitator LTI 1 Liberia Transition Initiative 1 LTI 2 Liberia Transition Initiative 2 LUCNA Liberians United to Cultivate Natural Artists LURD Liberians United for Reconciliation and Democracy MCSS Monrovia Consolidated School System MIB/NIB The Ministry and NGO in a Box Program MODEL Movement for Democracy in Liberia NGO Non-governmental Organization NHRCL National Human Rights Center of Liberia NTGL National Transitional Government of Liberia OFDA Office of Disaster Assistance OLMY Organization of Liberian Muslim Youth OTI Office of Transition Initiatives PAL Project Authorization Letter PLU Program Liaison Unit PPR Program Performance Review SOW Scope of Work SWIFT Support Which Implements Fast Transactions TDY Temporary Duty UN United Nations UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund UNMIL United Nations Mission in Liberia Social Impact, Inc. October 19, 2006 iv US United States USAID United States Agency for International Development USG United States Government WAS Consortium of World Vision, Action Aid and Search for Common Ground YES Youth Education for Life Skills YMCA The Young Men’s Christian Association YRTEP Youth Reintegration Training and Education Program Social Impact, Inc. October 19, 2006 v PREFACE Evaluation is a risky enterprise, for the client and for the evaluator. The client may be apprehensive that the project or activity may be found wanting when judged by the evaluators against some set of expectations, objectives, and performance standards, usually set out at the beginning of a project or program. Evaluators are very much aware that the people who designed and implemented a program always know more about the details of how things happened, and will have their own views about why and who was responsible. The evaluator runs the risk of “getting it wrong.” Evaluators are asked to determine the effectiveness of a program, and whether or not it achieved its intended results, if not, why not, and whether there are recommendations or larger lessons to be learned from the experience. These are difficult questions to answer in the best of circumstances, and evaluators are rarely provided the resources that are needed to develop the kind of irrefutable evidence that can stand up to the most critical policy and social science scrutiny. The best evaluations occur when there is sufficient trust and openness to inquiry and differences of viewpoint between the evaluators and the clients. When the client is both forthcoming and responds thoughtfully to potential criticism in a dialogue with the evaluator, the final product will be closer to the truth, and of more utility for all. If the evaluator approaches the job as “the inspector general,” the opportunity for dialogue is lost and the final report suffers. In the case of the USAID Office of Transition Initiatives, we believe that a dialogue was established which helped us to identify errors of fact and interpretation, leading to a better final product. We have tried to respond to most of the comments made on the working draft; but, in some cases, we simply lacked the evidence or knowledge to make an informed response. As evaluators, we also recognize that the “facts” can always be interpreted in many ways, and we can only do our best to apply our knowledge and experience to these facts to reach conclusions about the program’s merit. We will not satisfy all of our readers, in OTI and elsewhere, but we do hope that this evaluation will provoke more thoughtful design of programs in the future, as well as demonstrating that the kind of programs mounted by OTI can have appreciable, observable, albeit difficult to measure results. Social Impact, Inc. October 19, 2006 vi ACKNOWLEDGEMENTS As is always the case, were it not for the many individuals with whom we spoke in Washington and in Liberia, this document could not have been written. We begin by thanking our colleagues at OTI who gave unstintingly of their time to explain how the organization came to be – its creation story – and how it operates in societies recovering from conflict in transition to democracy and the rule of law. We received information from documents, maps, charts and databases as well as inspiration from their shared hope, and ours, “that Liberia is a dramatic success.” Creative Associates in Washington not only discussed their role in Liberia with candor but provided electronic copies of the curriculum we were to evaluate before our eminent departure. In Monrovia, OTI and Creative organized our interviews, village visits, and site visits in local neighborhoods flawlessly. The American Ambassador, the Political Officer and the USAID Supervisory Program Officer were also helpful, especially, the USAID representative who underscored information the Senior Education Policy Advisor in the Africa Bureau had provided about the Mission building on the present OTI momentum to continue the Accelerated Learning Program for the next three years. Liberian government representatives, Liberian and international NGO administrative and field staff, as well as villagers and their urban counterparts in Monrovia neighborhoods could not have been more welcoming or forthcoming. To each category of persons who made our work so much easier than it could have been we express our sincere appreciation and acknowledge that any shortcomings in this endeavor are ours alone. Social Impact, Inc. October 19, 2006 vii MAP OF LIBERIA SINOE GRAND GEDEH NIMBA BONG LOFA GBARPOLU BOMI MONTSERRADO MARGIBI GRAND BASSA RIVER GEE MARYLAND GRAND KRU RIVER CESS GRAND CAPE MOUNT Social Impact, Inc. October 19, 2006 viii EXECUTIVE SUMMARY This evaluation report commissioned by the US Agency for International Development’s (USAID) Office of Transition Initiatives (OTI) was conducted under the auspices of Social Impact Inc. by Dr. Richard N. Blue and Dr. Beverlee Bruce. The evaluation began in July 2006 with documentation review and interviews in Washington DC with OTI and other senior USAID staff. Field work in Liberia took place in August, followed by additional analysis and report drafting. The final report was presented to OTI on October 19, followed by a formal briefing for USAID OTI and invited parties on October 20, 2006. The Scope of Work (SOW) for the evaluation contained two sets of questions; the first for 90 percent of the effort focused on assessing the impact, successes, weaknesses, operational issues, and strategic and political responsiveness of the program, and included a request for lessons learned. The second set of questions, for 10 percent of the effort, were more general with regard to whether OTI’s Liberia program supported US foreign policy, peace and democracy, and was fast, flexible and opportunistic in providing such support.1 Because of the relatively short time frame allowed for field work, answering the questions required a combination of data collection, analysis and a heavy admixture of the judgment and experienced based reasoning of the evaluators, who benefited greatly from the comments and observations made by OTI officers and from other stakeholders and the many Liberians who shared their views with us. For purposes of analysis, the evaluation divided OTI’s Liberian Transition Initiative into two main phases – LTI 1, the first year which we describe as “classic” OTI transition grant making, and LTI 2, which we designate YES Plus, to describe the second year which focused 90 percent of OTI’s resources on a single youth education and cooperative grant funded program which took place in 367 communities spread over 10 Liberian counties. The evaluation covers OTI activities from December 2003 to June 2006, although OTI continues to fund activities at the time of this writing. Creative Associates International, Inc. was the primary contractor for grant administration. Mercy Corps, World Vision, Action Aid, and Search for Common Ground were Implementing Partners (IPs) under two cooperative agreements with OTI. The latter three organizations undertook the implementation of the YES training program, while CA managed the grant process. LTI 1 expended roughly $3.7 million on 142 Grants, while LTI 2 YES Plus and others minor programs expended over $2.8 million on 326 Grants. Overall, the project budget was $26 million, including OTI costs at $2.5 million, Creative Associates at $17.2 million, Mercy Corps at $2.5 million, World Vision at $2.3 million, and others at $1.8 million. Of the $17.2 million actual grants constituted $7.8 million and YES training cost $4.8 million. At the request of USAID, OTI also implemented the Accelerated Learning Program (ALP), with supplemental funds from USAID in the amount of $2.7 million. OTI is currently planning a third year in Liberia. 1 The complete Scope of Work can be found in Annex 1. Social Impact, Inc. October 19, 2006 ix Conclusions for LTI 1 Substantive Impact LTI 1 did what OTI does best and generally succeeded in helping government, civil society and media start to become functional by giving these sectors basic tools followed by program grants to address issues such as human rights, reconciliation, anticorruption, leadership, responsible and free media and the like. It also responded quickly and effectively to emerging threats, such as the Liberia University riots. By taking on the challenge posed by the student situation at the university, OTI’s immediate involvement resulted in a visible outcome. Strategic Responsiveness OTI Creative Associates, succeeded in implementing a classic OTI program in LTI 1, which was attuned to both US foreign policy objectives of helping produce confidence in and momentum towards a stable and democratic peace in Liberia. Within the OTI strategy, it managed to retain its basic operational code of being receptive, quick to commit resources, and opportunistic in response to highly volatile and changing circumstances on the ground. The Community Youth Peace Education Program (CYPEP) program is a good example. The CYPEP development may be considered strategic in that while responsive to an immediate need, the program had sufficient structure that was continued throughout the balance of OTI’s two-year program because it addressed a longer term problem of how to constructively engage large numbers of disaffected and restless urban/peri-urban youth in Monrovia and in other cities and towns. Civil Society Less dramatic but still significant over the longer term was the LTI 1 strategy of rekindling and supporting the development of civil society organizations in general. Testimony from several organizations underlined the critical importance of their relationship with OTI in 2004, and the importance of OTI’s willingness to award follow-on programmatic grants. One NGO leader said that more than the money, LTI’s willingness to stand with them during the difficult times built the confidence of other funding organizations in the NGO’s competence. Learning from Experience: Focus on Youth Finally, in response to the growing concern over how to reintegrate displaced persons back into Liberia, OTI decided early in the process to develop a program modeled after a similar program in Sierra Leone, and others in Burundi and the Democratic Republic of the Congo. In November 2004, LTI 1 mounted an important new initiative with the YES Pilot Program. The Pilot phase was set up to test the concept of an expanded youth for life initiative that had been deemed successful in Sierra Leone. The SOW for this evaluation asks whether OTI “properly” used the lessons learned from that experience. We will address this in the next section focused on LTI 2 and YES Plus. Social Impact, Inc. October 19, 2006 x LTI 1 Operational Effectiveness OTI used the SWIFT Indefinite Quantity Contract (IQC) to select CA as the contractor for LTI 1. Respondents from both Creative Associates and OTI stated that they had a close and cooperative partnership in LTI 1. This is consistent with the language used by OTI Washington leadership to describe its preferred modus operandi with contractors in general. OTI was slow in fielding a permanent country representative, as well as in getting Liberian administrative staff in place. However, some of the interim Country Representatives came to the job with a wealth of experience and a “can-do, problem solving” approach that fit well with OTI’s mandate and general operational principles. Another major problem in LTI 1 was the time it took to begin grant making. Although CA was in place as early as February 2004, the first real tranche of grants was not awarded until April/May 2004. With OTI’s mandate and reputation for disbursing grants rapidly, and the build-up of other assistance flows, it is reasonable to assume that a four month delay at the beginning of a two year commitment had negative consequences for OTI’s reputation, and perhaps for its ability to carry out its mandate. OTI had to work within a complex array of US and international organizations. The relationship between the USAID Mission and OTI seems to have been a positive one, with OTI being responsive to USAID requests with regard to school rehabilitation and curriculum and materials development for the ALP. OTI and CA also worked quickly and effectively to develop support for selective ministries and civil society organizations. Some OTI respondents agree that certain aspects of the operational record of OTI from November 2003 to May 2004 were not up to general OTI standards and may have had negative consequences for LTI 1. In sum, substantial grant making was slow to materialize, a permanent country representative was not on the ground until July 2004, and hiring local staff was excessively slow. On the positive side, OTI did field some very experienced TDY staff who were able to act quickly and creatively to the various threats to the transition process, and who were able to work closely with LTI to enable them to move money and, in some cases, provide technical assistance to government and Civil Society Organizations (CSO) much in need of simple, direct assistance. Summary for LTI 1 We conclude that the LTI 1 strategy, as described above, was appropriate and corresponded to Liberian needs, as well as being fully supportive of US foreign policy objectives. The impact of the program, as far as can be determined by the data available from interviews with various beneficiaries, was substantial, both materially and psychologically. Government and civil society were able to begin functioning and potentially dangerous disaffection and discontent among young people in and around Monrovia was redirected and channeled toward more positive and constructive behavior. The program focused on important transition issues including the reconciliation process, improving the public’s understanding of the peace process, providing an alternative to idleness and potential violent outbreaks that might threaten the peace, and establishing a new agenda associated with the introduction of democratic elections and accountability and transparency in government. Social Impact, Inc. October 19, 2006 xi On the operational side, the delay in grant making start up and other personnel delays certainly put pressure on OTI and made it more difficult to live up to its own operational standards. A more serious consequence may have been opportunities lost, and a delayed start of the YES program, resulting in having less time than desirable to mount a much more complex program in LTI 2. Conclusions for LTI 2 – YES Plus YES Training Modules There is considerable evidence that YES participants, both youth and older people, found YES training to be interesting and, in specific areas, valuable to them in both practical terms (learning to write and count) as well as in relational and psychological terms. Although the drop out rate remained high in terms of formal YES participants, there is little doubt that YES training reached a much larger audience in the communities, as interest and time warranted. It is also the case that some of the conceptual and language difficulties noted in the pilot assessment were not addressed and some modules, especially the one dealing with sexuality and HIV/AIDS, were difficult to negotiate. A major part of the justification for YES was that it would help solve the problem of reintegrating youth into local communities. Aggregate level statistics about the number of IDPs, ex-combatants, and the percentage of youth in the population as a whole can be misleading when used to describe the character of each community where YES Plus was implemented. Of the 14 communities visited, some did have ex-combatants, but for most, these people had left for other places. Many others reported that they had been dispersed into several locations, including refugee and IDP camps, but they had come back to face the job of rebuilding their homes, reclaiming their farms, and getting on with life. The “reintegration of youth” into the village did not stand out as the most pressing problem faced by the YES Plus communities visited by the evaluation team. Another objective for YES Plus was to contribute to conflict resolution. Remarking on what they learned from YES, young people especially said they had learned to respect each other and tried to start dialogues over differences. This level of conflict seemed to us to be of a type typically associated with adolescents or between adolescents and older generations found in all societies. In short, the level and character of the problem of “conflict” as described in program documents did not match up well with what we learned about the character of the YES communities we visited. In a few cases, the introduction of voting systems for selecting the type of project, as well as issues about where the project would be located, and who would control any benefits, led to divisiveness and may have exacerbated conflict. YES training did have a positive impact, but the project experience did not contribute much to lessons learned in YES. Under the best of circumstances, one very positive experience is not sufficient to fundamentally change attitudes and behaviors. Many YES Plus projects were rushed, unresponsive to variations in needs and on the ground conditions, timing, and costs, thereby leading to huge management problems, conflicts and infighting among IPs and sometimes disappointment for communities. Implementation was very management intensive. Moreover, in some cases, when it came time for YES participants to do the actual work of Social Impact, Inc. October 19, 2006 xii assembling sand and gravel, and making bricks, difficulties set in and elders took over. Local labor was frequently paid, voluntary contributions notwithstanding. General Conclusions 1. OTI’s contribution to the Liberian transition process was substantial, especially in the first year of the program. OTI worked closely with the US Embassy and USAID to meet the critical political needs of restoring government and civil society functions, increasing public knowledge and understanding of the issues, and promoting community integration in those parts of the Liberian population that posed the greatest danger to the fragile peace process. It used its special mandate to address these objectives by working closely and quickly to support Liberian governmental and civil institutions, most of which were not functioning. OTI demonstrated innovative and effective responses to transition crises in the CYPEP and other programs. Overall, US foreign policy interests and objectives were well served by the OTI LTI program, especially in LTI 1. 2. OTI’s effort to mount a national program focused on youth and community reintegration was less successful, although many positive benefits were provided at the individual and community levels. The YES Plus program was, in its design and implementation, not well matched to Liberian realities. OTI planners were rushed, and underestimated the operational, resource mobilization and time factors that would affect the program’s implementation. Extraordinary efforts had to be made to successfully complete most of the projects, but with questionable impact for the psychological and community integration objectives of the program. What emerged was substantially less than what was expected. YES Plus bore a close resemblance, in theory, to a donor assisted community development program, with too little time to substantially change attitudes, behaviors, and institutional arrangements supportive of longer term development transformation. Since OTI’s mandate is to design and implement “fast and flexible” programs, it is difficult to grasp why it undertook such an ambitious and complex program to implement as YES Plus. Lessons Learned 1. OTI was a creative solution to the emergence of war torn, post-conflict situations. Given extraordinary authority to move quickly, OTI’s mandate was not long term development, but to bring immediate evidence of benefits of peace to a war stressed population, and to reinforce that peace by using the “grant” mechanism to foster effective governance and social and political cooperation. For OTI, the process of helping communities and governments become functional again was more important than the product, whether it be a rehabilitated school, a playground, or a guest house. OTI best serves US foreign policy interests when it demonstrates American commitment to peace and fostering cooperation through democratic processes as evidenced by the Liberia program. 2. OTI’s mandate may be best served not just by making grants, but by engaging with its grantees in a purposeful and consistent way. In Liberia, OTI’s commitment to the Truth and Reconciliation Commission is one of several examples. For the local OTI Representative, there are four elements to this tactical approach: first, do the due diligence on the problem and on the potential grantee, second, start small and assess results; third, develop partnerships and networks of grantees with a more strategic objective, and fourth; be prepared to make multiple grants to Social Impact, Inc. October 19, 2006 xiii those organizations that demonstrate competence, commitment, and the ability to produce results. The other side of this tactical approach is that it fits well with OTI’s operational mandate of being opportunistic, risk taking, innovative, and political. 3. In its second year in Liberia, OTI chose to carry out what most observers and implementing partners agreed resembled a community development program similar to what the USAID Mission might have undertaken. Several lessons emerge from this experience. First, OTI should avoid commitments that tie up its budgetary and staff resources to the extent that YES Plus did. Second, given limited time and staff, avoid formulaic concepts and complex program designs that require far more time and resources than are typically available to OTI. Third, be cautious of adapting lessons learned from other programs if there is not time to test and adjust to the cultural and social realities of the country. Fourth, do not commit all or most of the resources to one, long term program; rather experiment, be flexible, and use resources to learn as you go, and to reward your counterparts’ progress. The lesson learned is that OTI should be wary of straying too far from its mandate and its proven track record of implementing programs that result in immediate benefits for the populations of societies emerging from devastating conflict or natural disaster. 4. OTI can play a role in testing out innovations that could become development programs funded by USAID or others. In this respect, the pilot phase of OTI LTI YES was such an effort, and it was this that allowed OTI to move quickly into the CYPEP program, by all accounts a classic quick response to a serious political problem stemming from peri-urban riots. OTI’s serial grants to indigenous organizations dedicated to fostering human rights, advocating for tough anticorruption measures, and development of responsible journalism make good use of OTI's mandate and, in some ways, allow for greater flexibility and engagement then USAID can exhibit in its civil society and democracy programs. The lesson learned is that OTI can be an innovator, incubator, and pilot tester of programs that do have long term development consequences. As OTI and USAID work together, it would be instructive for USAID to adopt some of the flexibility and speedy commitment that OTI exhibits. 5. There is a relationship between a successful process and a beneficial product. OTI programs need to pay attention to both, especially at the local level. The idea behind YES, and for that matter, some of the other classic grant making OTI did in Liberia, was that active engagement in a useful enterprise would help to reinforce the lessons learned from the training process. This is a sound idea, but if people see little or only temporary benefits from their engagement in the enterprise, or if the enterprise itself was not a high priority for them and they do not really own it, the product may well undermine the positive learning that derived from the process. 6. Organizational learning is a critically important process which requires discipline, constant dialogue, and some way of correctly assessing past experience. LTI YES built on recommendations from a similar program in Sierra Leone, Burundi and Congo. But there were other factors operating in the Liberia context that should have been considered before OTI committed to a one year national program such as YES Plus. Many of these were known towards the end of the YES Pilot phase, but not enough was done to address them. Social Impact, Inc. October 19, 2006 1 I. THE LIBERIAN CONTEXT The Conflict For the past 26 years the Liberian experience has been rife with social instability caused by civil conflict. Events contributing to this state of affairs include a military coup (1980); two civil wars (1989 and 2003); two contentious multi-party elections (1985 and1997); three interim governments; and 14 aborted peace agreements amid intervention by the Economic Community of West African States (ECOWAS), which, from 1990 to 1999, fielded its Military Observer Group (ECOMOG) in an attempt to establish peace. Finally, in August 2003, following former President Charles Taylor’s exile in Nigeria, representatives from the belligerent groups, Liberians United for Reconciliation and Democracy (LURD) and the Movement for Democracy in Liberia (MODEL), the Government of Liberia (GOL), major political parties and civil society signed the Comprehensive Peace Accord (CPA) in Accra, Ghana. At the same time, they selected the National Transitional Government of Liberia (NTGL) to govern the country while preparing for elections in October 2005. Since the election, Ellen Johnson Sirleaf, Africa’s first elected woman head of state, has established a popularly elected government. In the past two years, over 100,000 combatants have been disarmed, and 200,000 Internally Displaced Persons (IDPs) and 43,000 refugees have been resettled. But unemployment remains high and illiteracy is well over 50 percent, especially among young men and women who make up nearly one-third of the current population.2 The United States Agency for International Development’s Office of Transition Initiatives (USAID/OTI), working in collaboration with other US Government (USG) agencies, the United Nationals (UN), and other donors, confronted two major issues when they began working in Liberia shortly after the NTGL was inaugurated. The first challenge was to help the government and civil society start to function by rapidly dispersing material and financial assistance. The second, and more difficult, challenge was to develop positive engagement programs with Liberia’s youth. This large, disaffected, illiterate and unemployed segment of the population needed to be provided opportunities for reintegration and positive work that would contribute to the development of a peaceful and democratic Liberia.3 OTI’s Mandate Based on the Transition Initiatives legislation, OTI’s mandate is “…to support transition to democracy and to long-term development of countries in crisis: Provided, That such support may include assistance to develop, strengthen, or preserve democratic institutions and processes, revitalize basic infrastructure, and foster the peaceful resolution of conflict…” 4 In sum, OTI’s mission is to support U.S. foreign policy objectives by helping local partners advance peace and democracy in priority countries in crisis. It seizes critical windows of opportunity and works on the ground to provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs.5 In the US foreign assistance community, OTI is seen, and some of its staff describe it as a “SWAT” team because it moves into post-conflict situations where humanitarian assistance does 2 See Mission Performance Plan: FY 2008, U.S. Mission to Liberia, Department of State. February 13, 2006. 3 See Annex 2 for a more detailed analysis of the Liberian context. 4 From http://www.usaid.gov/our_work/cross-cutting_programs/transition_initiatives/. 5 Also from http://www.usaid.gov/our_work/cross-cutting_programs/transition_initiatives/. Social Impact, Inc. October 19, 2006 2 not reach, and before the more long-term development programs of USAID and other donors take hold. OTI is noted for being on the ground quickly, providing grant assistance promptly taking risks, being opportunistic, and for being free of the rhetoric and aspirations of long-term development strategies and objectives. OTI’s task after a long period of civil conflict is to demonstrate to a traumatized population that peace has dividends; that government and civil society can function, and that conflict can and should be resolved peacefully through democratic processes. II. PURPOSE OF THE EVALUATION The purpose of this evaluation is to assess OTI’s success in implementing a strategy that was responsive to the evolving political needs of the post-settlement transition period in Liberia from November 2003 to the end of May 2006.6 OTI expects the evaluation to identify impact of the Liberia Transition Initiative (LTI) program, as well as lessons learned and best practices that might benefit future programming. Scope of Work The OTI Scope of Work (SOW) for this evaluation contains two parts. The first part focuses on OTI’s success in its implementing strategy in Liberia and how OTI’s work could be improved. It requests specific examples of impact, strengths and weaknesses, operational successes and shortcomings, and, specifically, whether OTI Liberia “properly” used the lessons from a similar program in Sierra Leone. The programs predominantly focused on in this evaluation include the Youth Education for Life Skills program (YES), the Community Youth Peace Education Program (CYPEP) and the Accelerated Learning Program (ALP). The second part of the SOW asks whether OTI supported USG foreign policy objectives by providing “fast, flexible” assistance, adapted to “changing circumstances,” had a “clear strategy” and had significant impact. 7 (Annex 3, SOW Part 2 Answers) Time Frame The evaluation began July 17, 2006 with document review and interviews with OTI staff in Washington, DC from July 24 to 26, 2006. Field work in Liberia took place from August 12 to 23, 2006. Report drafting, debriefings, reviews, comments and rewrite occurred from September to October 2006, with the final report submitted to OTI on October 19, 2006 and a final Power Point briefing on October 20, 2006. The Evaluation Team The two-member evaluation team was led by Dr. Richard N. Blue, a former USAID Senior Officer, Director of the USAID Office of Evaluation, and since 1997, a consultant on more than 30 USAID requested evaluations and assessments. Dr. Beverlee Bruce has more than 30 years experience in Liberia as a scholar, teacher, Peace Corps Director, and frequent participant in United Nations and World Bank assessments in Africa. 6 See Section III of this report for an explanation of the time line for OTI’s involvement in Liberia, and for the reasons for selection of the November 2003 – May 2006 time frame for purposes of this evaluation research. 7 The evaluation team’s answers to Part 2 of the SOW are found in Annex 3. The evaluation answers to SOW Part 2 questions are summaries of the findings and conclusions from Part 1 presented in this report. The reader is strongly urged to read the report first, then the Annex with Part 2 Answers. Social Impact, Inc. October 19, 2006 3 The Methodology The team reviewed OTI documents, interviewed key stakeholders in Washington, DC and Liberia, as well as program beneficiaries in Monrovia and in 14 rural communities. Of the 11 working days available to the team in the field, five days were used to interview program implementers, government ministries and NGO leaders, while six were used to visit 14 rural communities selected by the evaluators from the OTI Grants Database (GDB) and with the advice of OTI’s IPs. (Annex 4, 5 and 6)8 Validity of the report findings, generalizations and lessons learned In spite of the best efforts of the evaluation team, this evaluation suffers from a number of weaknesses. First, there was no baseline data laid down by OTI against which to measure change resulting from the OTI program. This was especially a problem for assessing the follow￾on YES program, also known as the LTI 2YES Plus program.9 Second, the time allotted for field work was insufficient to conduct a systematic sample of either LTI 1 or YES Plus grantees for analysis. Third, for the same reason, no effort was made to examine control communities for assessing whether YES Plus communities changed/benefited more than non-YES Plus communities. All generalizations in this report, while supported by what we observed, and by our best judgment based on long experience, should be treated cautiously as based on the available evidence. III. THE OTI PROGRAM IN LIBERIA: October 2003 to June 2006 Following the August Peace Accords, OTI sent an assessment team to Liberia in October 2003, followed in early December by the first of four OTI long-term Temporary Duty (TDY) personnel with authority to begin programming based on the assessment report produced in November 2003. (Annex 7 Program Timeline) 10 Using an already competed OTI contract mechanism called “Support Which Implements Fast Transition” (SWIFT) IQC, OTI was able to secure the services of Creative Associates International, Inc. (CA) as an implementing partner which was on the ground by February 2004. CA received a Project Authorization Letter (PAL) that allowed them to become operational even before they received the money. However, according to OTI substantial grant making was delayed until April 2004 when the first major tranche of OTI grants was made. The first permanent OTI Representative assumed control in July 2004 – eight months after OTI had established a presence in Monrovia. OTI is continuing its program in Liberia with an 8 Annexes 4,5 and 6 contain (a) a list of all persons interviewed, (b) a list of communities visited, and (c) a Site Visit Question Guide used by the team to structure community group discussions. 9 In retrospect, the evaluation team may have erred in focusing its limited field time on assessing YES Plus communities. However, given the emphasis given to YES Plus, at the time we felt that six of the eleven days was the least we could devote to this effort. This left only five days to meet with US Embassy, USAID, Liberian Government officials, interview LTI 1 grantees, assess CYPEP, ALP, and conduct interviews with YES IPs in Monrovia. 10In the working draft of this report, one reviewer stated that “(OTI)…should have realized that the window of opportunity was from the assessment and go decision as laid out in the timeline. OTI, as part of Democracy Conflict and Humanitarian Assistance (DCHA), has a reputation of being a SWAT dealing urgently with post peace agreement issues. That’s what the US Ambassador and the USAID Mission Director expected of us. The Peace Accord was signed in August 2003. OTI sent a four person assessment mission in October. An implementing partner was selected in February, but had no substantial money to operate until April – eight months later.” Social Impact, Inc. October 19, 2006 4 extension of certain components covered by this evaluation, while designing a new one-year approach for FY 2007. 11 The OTI Liberia Transition Initiative Program: A Macro-level Overview Program Content OTI’s Liberia program was known by two titles reflecting a major shift in emphasis from Year One to Year Two. Year One’s program, the Liberia Transition Initiative 1 (LTI 1), was a classic OTI program that included grant making designed to: support the political transition to peace and an elected government; to build momentum for democracy through support for civil society and responsible media; and for a peaceful resolution of conflict including reintegration of communities. LTI 1 also included implementation of the ALP, funded at the request of USAID. The LTI program continued into Year 2 serving largely as the community grant component of the YES Plus program, as explained below. From the beginning, OTI had decided that an important part of its strategy was to focus on Liberian young people, especially the reintegration of both combatants and non-combatants into Liberian society. LTI 1, therefore, included the pilot phase of the YES program, which was a five-month training program adopted and modified from the Youth Reintegration Training and Education Program (YRTEP) program implemented in Sierra Leone and focused on life skills, basic literacy, health, conflict management, family, and HIV/AIDs. The purpose of YES was to assist young people, defined as ages 18 to 35, to return to and reintegrate into their communities as productive and peaceful members. YES became the almost exclusive emphasis in Year Two which we will call LTI 2 or YES Plus, the “plus” referring to the OTI strategy of linking a community grant program to the YES experience.12 This was one of the principle recommendations of an earlier evaluation of the Sierra Leone YRTEP program. Under separate cooperative agreements with OTI, YES training was implemented by Mercy Corps and World Vision, acting as the principal partner in a consortium, known as WAS, which also comprised Action Aid, and Search for Common Ground’s African affiliate, Talking Drum Studio. Creative Associates continued to be responsible for grant administration including input to the OTI GDB. Although 90 percent of LTI 2 funds were targeted on YES Plus grants, CA continued with limited “classic” grant making through year two. It is important to note that from October 2003 OTI Liberia was forward operating in a complex and fast moving environment with many donor organizations, as well as the UN organized 11The Scope of Work prepared by OTI is structured as an Impact Evaluation, but did not specify a close date of the program. Therefore, the evaluation team selected May 31, 2006 as the close date for its review of OTI programs for several reasons: 1) OTI’s internal decision to close all grants associated with the second year YES program by that date; 2) the team was conducting “impact focused” field work in August 2006, just two months after the grant close out date; and 3) the post May 2006 program had been modified from the one that had been underway from July 2005 to May 2006. An assessment of impact or results of the ongoing activities would have been meaningless. 12 Some OTI reviewers disliked the term “YES Plus” as it was not a term used within OTI. The evaluation team selected the term to accentuate three things of importance: 1) that OTI made a conscious decision to invest 90 percent of its resources in LTI 2 to this program; 2) in response to an evaluation of a similar Sierra Leone evaluation OTI added a grant component (the Plus) to the YES program; and 3) after May 31, 2006, OTI continued to fund YES without the grant component. Since this third iteration was still being implemented, the evaluation team chose not to speculate on its effectiveness or impact compared to YES Plus. Social Impact, Inc. October 19, 2006 5 Disarmament Demobilization Rehabilitation and Reintegration (DDRR) and United Nations Mission in Liberia (UNMIL) activities associated with demobilization and peacekeeping. Within the US Government apparatus, besides the US Embassy, USAID’s office of Food for Peace (FFP) and the Office of Disaster Assistance (OFDA) were also in the field. This required both coordination as well as responsiveness to political objectives, a key to which was finding some way to address the problem of demobilized and returning young people. By May 31, 2006, LTI had completed and closed 468 grants totaling $6,533,473. 13 Of these, 142 grants for $3.7 million were made in LTI 1, mostly for government and civil society related activities in Monrovia, but including the YES pilot phase grants. LTI 2 involved 326 grants for a total of $2.8 million, mostly connected with the YES Plus program (see below). LTI 1 and 2 grants were made in counties outside Monrovia, however this was more prevalent during LTI 2 – Bomi received 80 grants, Grand Cape Mount 73, Bong 58, Nimba 45, Grand Bassa 44, Grand Gedeh 43, Gbarpolu 23, Sinoe 15, and Lofa three. Of these, YES Plus managed 326 grants, or about 43 percent of the LTI grant activity. Forty-nine YES Plus grants, or about 15 percent, were ultimately cancelled for non-performance by the communities. Exhibit 1: Map of Liberia with Counties, # and Value of LTI Grants and # of IDPs 13 Source: The OTI Grant Database (GDB) shows different numbers depending on the ‘filter’ used to create sub-sets of grants. Using the ‘all grants’ filter, the total number of grants is 547 for $7,676,047. This appears to be the total of all grants through August 2006. It is difficult to identify precisely the number of YES Plus grants, as the GDB has no specific filter for that sub-set. Social Impact, Inc. October 19, 2006 6 Exhibit 2: Comparison of LTI Grants by Program Objective Grants by Program Objective $0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 Restore Critical Transitional Governance and Civil Society Functions Increase Public Understanding of Key Political Transition Issues Promote Community Reintegration and Peaceful Resolution of Conflict Program Objective Grant Amount ($) Closed Cleared Completed Cancelled Exhibit 3: Comparison of LTI Grants by Focus Area Grants by Focus Area $0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 Civil Society Organization Support Community Impact Activities Conflict Managament Election Processes Excombatant Reintegration Justice/ Human Rights Media Transparency/ Good Governance Focus Area Grant Amount ($) Closed Cleared Completed Cancelled Social Impact, Inc. October 19, 2006 7 Exhibit 4: Total Expenditures for LTI by OTI and Implementing Partners YES Pilot training programs had been implemented in 40 communities, and the Year 1 YES spin-off, known as CYPEP had been given in several Monrovia outlying communities and in five other Liberian towns. The YES Plus training and grant cycle one had been completed in 326 communities by May 31, 2006. IV. LTI YEAR 1: Findings and Intermediate Conclusions LTI Year 1: Grant Making Strategy and Implementation By June 2005, LTI 1 issued 142 grants for a total of $3.8 million. The grants were awarded to government, civil society, and media organizations, primarily to get them functioning again and to demonstrate positive benefits to peace, raise hopes, and build confidence.14 These grants supported the political objectives of the international, and especially the US, commitment to the democratic process and building momentum for peace in Liberia. A significant part of the program was to assist the justice and re-conciliation effort, including funding important research on Liberian attitudes and expectations with regard to this sensitive issue. LTI 1 made many small grants to NGOs, but more than 20 indigenous NGOs received grants totaling $30,000 or more, some with multiple grants. The Young Men’s Christian Association (YMCA), National Human Rights Center of Liberia (NHRCL), and others received more than $100,000, with YMCA receiving the most with $252,000. Exhibit 5: Liberian NGOs Receiving grants valued at $30,000 or more, April 2004-June 2006 14 The GDB lists the objectives as follows: “1) restore critical transitional governance and civil society functions; 2) increase public understanding of key political transition issues; and 3) promote community re-integrations and peaceful resolution of conflict. Total Expenditures Creative Associates, $17,280,375, 65% OTI, $2,419,611, 9% Mercy Corps, $2,583,614, 10% World Vision, $2,274,200, 9% Other, $1,833,496, 7% Total Program Costs: $26.39 million Social Impact, Inc. October 19, 2006 8 Name of Organization Number of Grants by OTI Total $ value 1. YMCA 10 252,706 2. NHRCL 4 196,267 3. BUCCOBAC 3 125,279 4. DEN-L 4 119,512 5. CENTAL 6 109,932 6. SEARCH/T.D. 4 99,909 7. ILDO 4 91,627 8. OLMY 5 86,872 9. LUCNA 3 66,195 10. MAWORDA 2 54,349 11. LIBERIAN FOOTBALL ASSOC. 1 53,193 12. FLY 4 46,715 13. SUSTAINABLE AG. 2 39,970 14. PRESS UNION 3 37,205 15. MLE 1 33,996 16. TJWG 1 33,425 17. MARWOPNET 1 32,399 18. GREEN 2 31,754 19. NAT. EX-COMBATANTS 1 31,545 20. LIBERIAN WOMEN’S NATIONAL POLITICAL FORUM 1 27,463 TOP 20 GRANTEES 62 1,618,096 Examples of successful OTI LTI 1 programs The SOW directed the evaluation to present examples of “successful” OTI programs. LTI 1 demonstrated the best traits of OTI – its ability to move quickly to put programs in place produced substantial and immediate impact.15 Four examples stand out in Liberian and US observers’ minds. These are the response to University student riots in 2003, the response to urban youth violence in October 2004, the fast implementation of the ALP, and the Ministry-in￾a-Box and NGO-in-a-Box programs (MIB/NIB). Curtailing University Riots First, OTI’s effort to reorient university student riots into a very successful “clean-up” effort helped the University become operational and gave a sense of purpose and self confidence to student leaders. As described by one former OTI Liberia staff person: The US Ambassador called on a Thursday to find out what OTI could do in response to the pending crisis. That Friday OTI met to discuss the issue, Saturday they were on campus to discuss the situation with the students – who conducted an assessment and submitted a grant proposal. On Monday OTI cleared the grant. A second intervention was in response to the situation at the five Monrovia Consolidated School System (MCSS) high schools in Monrovia. The MCSS teachers were threatening a boycott over late pay and the students planned to join them in a massive demonstration signaling their support. The University 15 Even though there was a time lag for the IP to be able to kick off the grant program in LTI 1, OTI was able to have its representatives in-country lay the groundwork for immediate implementation once the IP was ready. Social Impact, Inc. October 19, 2006 9 students with whom OTI had worked were called to intervene and did so averting an event that could have resulted in street violence. Community Youth Peace Education Program: CYPEP, Adapting YES for Urban Youth Another success was OTI’s development of an urban youth life skills training program (adapted from the YES curriculum then in pilot testing) called CYPEP. This is generally cited as a very quick and effective response to peri-urban gang violence that broke out among disaffected youth in Monrovia. Following the October 28, 2004 youth riots when churches and mosques were burned in and around Monrovia, OTI reached out to youth organizations for their assistance in defusing the situation. While UNMIL identified neighborhood hotspots, community leaders, religious, youth and sports groups had conducted their own assessments as well. YMCA, an international Christian organization that has been in Liberia for over 100 years, implemented the program in five neighborhoods during CYPEP’s first phase. The Federation of Liberian Youth (FLY), an umbrella organization of all youth groups in Liberia, did the same. The Organization of Liberian Muslim Youth (OLMY) was inaugurated in 1972 with chapters in every county. Since 2001, the group has been rebuilding the structures that broke down during the war. At present it has chapters in Bong, Lofa, Grand Cape Mount and Montserrado Counties. OLMY implemented CYPEP programs during the first phase in two Monrovia neighborhoods. According to FLY, OLMY and YMCA representatives the impact of CYPEP has been multifaceted and includes: · A commitment to volunteerism among youth – Seeds for Peace, Youth in Action and Youth for Truth; · Violence-free pre- and post-election periods; · An holistic approach to understanding the role of youth in a democratic society; · Participants know the reality of AIDS as opposed to hearsay about the disease; and · Participant use of condoms as a preventative measure against becoming HIV positive. In addition, community leaders in the targeted neighborhoods report that “the children have been transformed.” As an example many young people take their grievances to a community leader for resolution rather than resorting to violence. During the war, youth with guns were empowered and intergenerational conflict became the norm. Now these same youth, along with their peers, are more civil than they were. The Accelerated Learning Program (ALP) A third success was OTI’s implementation of the ALP at the request of USAID Mission. This initiative demonstrates OTI’s utility to longer term development programs mounted by USAID Missions. The ALP allows students from ages 8 to 18 to complete primary school in three years instead of six. In many of the villages the evaluation team visited someone would make note of being able to write his or her name (frequently girls) and in saying so indicates an interest in more literacy. One person reported how it was unnecessary to make a mark on the ballot during the recent election but rather to write one’s name instead. In another village we were told that the villagers had already spoken to the district education officer about enrolling several young Social Impact, Inc. October 19, 2006 10 women who are mothers but interested in returning to school now that the war is over. Being over age and in school is routine in today’s Liberia. The ALP momentum on which USAID will build includes LTI’s rehabilitation of 11 ALP schools in six counties for a total cost of $2.7 million; 750 teachers trained by Master Trainers trained by LTI; 9,000 ALP students supported by grants from LTI to Save the Children, the United Kingdom; the International Rescue Committee, the Christian Children’s Fund and selected parochial schools; LTI institutional collaboration with the Ministry of Education, UNICEF and UNESCO; as well as its purchase and distribution of 15,000 textbooks. “In the Box” Program Lastly, the Ministry and NGO in a Box (MIB/NIB) program was remembered by current Ministers and NGO respondents as being a necessary condition to becoming in any way operational during the transition government period. Also, OTI was far quicker in supplying necessary equipment than other donors who had promised similar assistance. Focus on Monrovia in LTI 1 LTI 1 concentrated most of its efforts in and around Monrovia. This strategy made sense for several reasons. First, Monrovia was the seat of government and ministries had almost nothing to work with, including chairs, desks, and computers. Second, Monrovia was where the most volatile parts of the population were located, e.g., University, peri-urban areas with high populations of disaffected youth who were unlikely to return to the rural communities. Third, it allowed OTI and CA to work closely to restart, create and energize Liberian CSOs, which had been prominent in Liberia earlier, but had been devastated by the years of conflict. Moreover, the post-conflict crisis and transition period generated a new agenda for CSOs, including advocacy for human rights, prevention of abuse to women and children, issues of justice and resolution of the difficult question of “who is to be blamed.” Finally, OTI/LTI was functioning in a still very insecure environment outside of Monrovia. Creative Associates did not have much familiarity with the micro-conditions in other counties like Grand Cape. In the few cases where LTI did make grants in smaller towns, such as Robertsport in Grand Cape Mount, the experience was not positive.16 Operational timeliness After the national transition government was installed in August 2003, OTI quickly fielded assessment teams and had experienced people in Monrovia by December 2003 with OTI interim country representatives. However, OTI was slow in getting a full-time country representative on the ground. The permanent country representative started in July 2004 – eight months after the OTI assessment team concluded its work. After receiving the contract for LTI, Creative Associates was able to field key people within a few days, but unresolved issues between the USAID Office of Acquisition and Assistance and Creative Associates prevented any substantial grant making until April 2004, or five months after the OTI assessment team concluded its work. 16 See OTI Grant Data Base for early grants in Grand Cape Mount County municipality of Robertsport. The impact statements in these early reports suggest a number of problems, later confirmed by interviews with CA officers. Social Impact, Inc. October 19, 2006 11 An OTI Strategy session was held at the end of May 2004 and it confirmed the basic direction of the existing program. The YES program was alluded to at this meeting but it was not prominent in the June draft of this strategy. Other documents and interviews reveal that a YES type program had been on the planning table from the beginning, but the dominant nature, scope and means for implementing this program had not been fully developed until 2005. Intermediate Conclusions Substantive Impact LTI 1 did what OTI does best and generally succeeded in helping government, civil society and media start to become functional by giving these sectors basic tools, followed by program grants to address issues such as human rights, reconciliation, anticorruption, leadership, responsible and free media and the like. It also responded quickly and effectively to emerging threats, such as the Liberia University riots described elsewhere. OTI, in taking on the challenge posed by the situation with the students at the University, was involved immediately and the outcome was visible. Strategic Responsiveness OTI and its contractor, Creative Associates, succeeded in implementing a classic OTI approach in LTI1, one which was attuned to both US foreign policy objectives of helping produce confidence in and momentum towards a stable and democratic peace in Liberia. Within OTI's strategy, it managed to retain its basic operational code of being responsive, quick to commit resources, and opportunistic in response to highly volatile and changing circumstances on the ground. The CYPEP program is a good example. The CYPEP development may be considered strategic because it was not only responsive to an immediate need, the program had sufficient structure that it was continued throughout the balance of OTI’s two-year program because it addressed a longer term problem of how to constructively engage large numbers of disaffected and restless urban/peri-urban youth in Monrovia and in other cities and towns. Civil Society Less dramatic but still significant over the longer term was the LTI 1 strategy of rekindling and supporting the development of civil society organizations in general. Testimony from three organizations underlined the critical importance of their relationship with OTI in 2004, and the importance of OTI’s willingness to award follow-on programmatic grants. One NGO leader said that more than the money, LTI’s willingness to stand with them during the difficult times built the confidence of other funding organizations in the NGO's competence. Learning from Experience: Focus on Youth Finally, in response to the growing concern over how to reintegrate displaced persons into Liberia, OTI decided early in the process to develop a program modeled after a similar program in Sierra Leone, and others in Burundi and the Democratic Republic of the Congo. In November 2004, LTI 1 mounted an important new initiative with the YES Pilot Program. The Pilot phase was designed to test the concept of an expanded Youth for Life Skills initiative that had been Social Impact, Inc. October 19, 2006 12 deemed successful in Sierra Leone. The SOW for this evaluation asks whether OTI “properly” used the lessons learned from that experience. We will address this in the next section focused on LTI 2 and YES Plus. LTI 1 Operational Effectiveness The OTI Scope of Work required the team to examine OTI’s “operational successes and short￾comings.” We have interpreted this to include OTI’s relationship with its implementing partner in LTI 1, Creative Associates. With a staff of four persons, two Americans and two Liberians, OTI cannot implement anything by itself. It follows the same practice as other USAID units, relying on contracts and cooperative agreements to bring in the necessary person power, expertise and organizational resources needed to implement OTI’s programs. In LTI 1, this was Creative Associates. OTI used its SWIFT Indefinite Quantity Contract to select Creative Associates as the contractor for LTI 1. Respondents from both Creative Associates and OTI stated that they had a close and cooperative partnership in LTI 1. This is consistent with the language used by OTI Washington leadership to describe its preferred modus operandi with contractors in general. OTI operations OTI was slow in fielding a permanent Country Representative, as well as in getting Liberian administrative staff in place. However, some of the interim Country Representatives came to the job with a wealth of experience and a “can-do problem solving” approach that fit well with OTI’s mandate and general operational principles. Another major problem in LTI 1 was the time it took to begin grant making. Although Creative Associates was in place as early as February 2004, the first real tranche of grants was not awarded until April 2004. In a fast moving situation such as was the case in Liberia in 2004, this was an objectionable delay, putting much pressure on the interim Representatives. It is impossible to say what the cost of these delays was in terms of missed opportunities. Given OTI’s mandate and reputation for fast disbursing grants, and the build-up of other assistance flows, it is reasonable to assume that a four-month delay at the beginning of a two-year commitment had negative consequences for OTI’s reputation, and perhaps for its ability to carry out its mandate. OTI had to work within a complex array of US and international organizations. The relationship between the USAID Mission and OTI seems to have been a positive one, with OTI being responsive to USAID requests with regard to school rehabilitation and curriculum and materials development for ALP. OTI and CA also worked quickly and effectively to develop support for selective ministries and civil society organizations. Some OTI respondents agree that certain aspects of the operational record of OTI from November 2003 to May 2004 were not up to general OTI standards and may have had negative consequences for LTI 1. In sum, substantial grant making was slow to materialize, a permanent country representative was not on the ground until July 2004, and hiring of local staff was excessively slow. On the positive side, OTI did field some very experienced TDY staff who were able to act quickly and creatively to the various threats to the transition process, and who Social Impact, Inc. October 19, 2006 13 were able to work closely with CA to enable them to move money and, in some cases, provide technical assistance to government and CSOs much in need of simple, direct assistance. Summary for LTI 1 We conclude that the LTI 1 strategy, as described above, was appropriate and corresponded to the Liberian needs, as well as being fully supportive of US foreign policy objectives. The impact of the program, as far as can be determined by the data available from interviews with various beneficiaries, was substantial, both materially and psychologically. Government and civil society were able to begin functioning, and potentially dangerous disaffection and discontent among young people in an around Monrovia was redirected and channeled toward more positive and constructive behavior. The program focused on important transition issues including the reconciliation process, improving the public’s understanding of the peace process, providing an alternative to idleness and potential violent outbreaks that might threaten the peace, and establishing a new agenda associated with the introduction of democratic elections and accountability and transparency in government. On the operational side, the delay in grant making, start-up and other personnel delays certainly put pressure on OTI and made it more difficult to live up to its own operational standards. A more serious consequence may have been opportunities lost and a delayed start of the YES program, resulting in having less time than desirable to mount a much more complex program in LTI 2. V. LTI 2 PROGRAM: YES Plus The YES Plus program dominated LTI 2, and was marked by the rollout of an ambitious effort to help young people return to their communities and become productive and peaceful participants in the overall rebuilding effort. YES was the training component; Plus was the LTI grant funded component. Our findings cover first, the YES training component, followed by YES Plus, with emphasis on the implementation of the grant project component. YES Plus ended May 31, 2006, two months before this evaluation. YES training continues under Mercy Corp and Action Aid, while World Vision has been redirected by OTI towards implementation of the CYPEP program. The post-May 31, 2006 activities are not covered by this evaluation. Youth Education for Life (YES) The YES Plus program was ambitious as illustrated by the anticipated number of beneficiaries and the complex structure of the program. One early estimate of beneficiaries was 30,000 youth from more than 700 communities. This was adjusted downward as OTI and its partners began implementation. By May 31, 2006, 367 communities had received YES training, and 326 completed both training and grant projects. We estimate there were between 12,000 and 14,000 direct beneficiaries of YES training. The program’s main purpose was the reintegration of the internally displaced youth who migrated to Monrovia or elsewhere; young people who may have joined one or more militia; and those who remained during the crisis. It was these populations that are generally of major concern to donors, policymakers and government officials in societies undergoing post-conflict reconstruction and such is the case in Liberia. Drawing on what was characterized as a successful informal education program in Sierra Leone that assisted in the reintegration of Social Impact, Inc. October 19, 2006 14 40,000 youth into their respective communities; members of OTI Washington staff hypothesized a similar outcome in Liberia, the Congo and Burundi. Using a training-of-trainers model the program trains Master Trainers who in turn train Learning Facilitators to serve as peer educators in establishing a participatory learning environment for 35 participants who meet for a period of five months in four sessions a week for 2 hours per session, 8 hours a week. The curriculum includes seven modules, each linked to basic literacy and numeracy skills. However, in response to the absence in Sierra Leone of a follow-on activity that would have linked the program in some concrete way to community reintegration, a small grants project was added to the Liberia program with the intention of providing a beneficial intergenerational bonding experience.17 The Pilot From November 2004 through April 2005, Creative Associates used grants to contract with Mercy Corps and Action Aid to conduct a YES Program pilot in 40 communities in Grand Cape Mount, Grand Bassa, Montserrado and Margibi Counties. It also contracted two monitoring and evaluation agencies, Subeh-Belleh Associates and Third World Consultants, to determine the level of understanding participants derived from exposure to YES program modules and at the same time it required its Master Trainers to track the appropriateness of training sites and the adequacy of the training materials. In addition, Creative’s Program Liaison Unit18 (PLU) visited 26 of the pilot communities to determine enrollment and retention rates of program participants and completion rates of curriculum modules. It is difficult to determine whether the findings from the PLU’s assessment were fully taken into account in mounting the YES Plus roll-out. These findings were important in that they showed that Pilot phase participants were relatively knowledgeable about some of the topics, but had difficulty with others, such as “peacebuilding.” Literacy and numeracy training had been highly valued, but more was needed to make these tools truly useful. Sexuality and HIV/AIDS modules were very difficult, and took time to work out a way to discuss. Learning facilitators also had difficulty with a number of the concepts introduced such as “income generation,” self-esteem, values, and attitudes. Finally, two critical findings might have given the program’s advocates cause to restructure and re-calibrate the length, scope and complexity of the YES curriculum. First, pilot phase respondents could not retain the content of the program over time, and second, except for the first module where the completion rate was 58 percent, the completion rate for the rest never exceeded 50 percent, and dropped to about four percent for the last two modules. Exhibit 6: YES Module Completion Rate MODULE % of Community that Completed Training Module 1: My Identity (13 Sessions) 58 % completed Module 2: World of Work (6 Sessions) 46% completed Module 3: Health and Us (15 Sessions) 35% completed Module 4: Peaceful Living (10 Sessions) 27% completed Module 5: Good Governance (11 Sessions) 15% completed Module 6: Our Environment (5 Sessions) 08% completed Module 7: Next Steps 04% completed 17 SOW Question II.5 asks, did OTI “properly use” lessons from the Sierra Leone YRTEP evaluation. As stated, the grant funded YES Plus project was a direct response to one of the recommendations made. 18 Creative established and staffed a Program Liaison Unit responsible for coordinating every aspect of the YES Project. Social Impact, Inc. October 19, 2006 15 Unfortunately, many of the problems identified in the CA assessment of the pilot phase including high drop-out rates, difficulties with the curriculum content for some modules and logistical support issues continued in the roll-out of YES Plus. Also, as seen below, management, implementation, community participation, and rising costs problems were either not anticipated or were largely under-estimated. Evidence of Impact of YES training In the course of the evaluation the team visited five villages in Grand Cape Mount, five in Lower Bong and five in Bomi and Margibi. Our findings are not unlike those of the CA PLU summarized above. In most villages, more women than men attended our group discussions, and it was clear that both had difficulty remembering the specific content of the modules. Yet they were frequently enthusiastic about what they had learned. For many literacy was important because they understood the concept “if you can read, no one can cheat you.” Others observed that “the program was important because we have had no previous schooling.” When asked whether anyone fell asleep in class after a hard day’s work, one woman said, “I never closed my eyes because I wanted to learn what they were teaching.” Of the seven modules that comprised the YES curriculum “Who Am I” was a clear favorite. Other favorites included protecting the environment, personal hygiene, resolving palaver, communicating with the elders, the meaning of citizenship, learning the lesson about HIV/AIDS which is to “be mindful of ourselves by not going around but to stay with one’s wife or husband,” learning how to live in a family and how to love one anther and forgive. In one village, participants said they were not embarrassed to talk about issues related to sexuality, whereas in another they said at first they were but later they were empowered to discuss the ways to protect themselves from the “deadly disease.” One young woman volunteered that as a result of the YES curriculum, she was “more assertive and self-confident.” Others mentioned having learned the importance of equal rights for men and women on the grounds that women are as capable as men. With regard to completion rates, our observations were somewhat better than those of the PLU, but we would still estimate that no more than 66 percent of the cycle I participants actually completed all the modules. Related to that was a surprising finding. YES training was not just for youth. In fact, it appears from our group discussions that everyone participated in at least some of the training, mostly held in the village town hall late in the day, a time when work was done. Although we did not get really consistent or forthcoming answers as to why the drop out rate was so high, we suspect that the length of the program, five months, four nights each week, may have taxed the level of commitment and the available time and interest for many.19 For some communities, where people had to walk some distance from a satellite hamlet, the distance and insecurity of a night walk was a bit too much. In one, jealous husbands were afraid the Learning Facilitators were womanizers, and they said they were “too big to learn.” YES Plus: Linking YES training to a LTI grant funded community project A central proposition of the evaluation done for the Sierra Leone YRTEP program was that the lessons of the YES curriculum needed to be consolidated by a community experience for which the YES youth would take the lead. This became a central concept of YES Plus, with mixed results. 19 This explanation was seconded by several of the IP leaders during our exit briefing in Monrovia, August 23, 2006. Social Impact, Inc. October 19, 2006 16 Learning about elections: selecting a project Near the end of the YES training cycle, the Master Trainer would come to the community and organize a so-called decision session to determine what kind of follow-on project would be implemented with the required participation of the community. During this session and with guidance, the villagers were provided a menu of projects that could be undertaken. 20 The menu included village infrastructure, such as rehabilitating a school house, a hand pump, a guest house, a community hall, or a latrine. Or villagers could choose skills training in soap making, cultivating vegetable gardens, animal husbandry or rice/cassava mills. With guidance, an election was held to determine the first, second and third choices. Once a choice had been made, the community went through a planning process to develop a time line and assignments for completion. The YES Management Chair and committee were crucial to this part of the exercise. Exhibit 7: Projects Closed or Completed by Type listed in order of frequency (GDB) Project Type Frequency Skills Training 68 Cassava/Rice Mills 38 Sporting Goods 27 Guest Houses 25 Multi-purpose Halls 22 Hand Pumps 21 Market Hall 19 Latrines 9 Town Hall 7 Bridge and Road 6 School Rehab 5 Musical 3 After the selection had been made, the initial proposal was prepared by the IP and sent to CA, who would then inspect the site and make an engineering estimation of whether the project was feasible, what was needed in material and local contribution, and the total value/cost of the project. Then the whole vetted proposal was sent to OTI, including the three village choices in rank order of voting. OTI could clear or reject the grant.21 Although most of the time villagers got their first choice, later in the program OTI sometimes rejected the first and the second choice, approving the third one. According to statements made by IPs and LTI, cost and time to 20 The use of the term ‘menu’ was disputed by one OTI commentator. However, this term was used by a former OTI officer as follows: “What were the consequences of Creative’s menu of possible small grant activities for communities to choose from?” Another, very knowledgeable OTI respondent used the term ‘menu’ and said that it limited villagers choices believing as they did that there were no other possibilities than those presented them. That is villagers did not see the menu as just a list of suggestions and the IPs seem not to have informed them otherwise. 21 It should be noted that the word “grant” as used by OTI in this context is misleading. OTI does not transfer money to a community for it to manage. Rather, OTI uses “grant” funds controlled by its contractual partner, in this case Creative Associates, to purchase equipment, material, transport, hire skilled labor, and pay for local labor as needed. Creative Associates provided blueprints for buildings, and bought the machinery for rice and cassava mills. In YES Plus, CA estimated the dollar value of the local contribution, which was mainly making bricks, hauling sand and gravel, to meet the 25 percent participation requirement of the program. In LTI 1, however, OTI did make substantial program and core operational grant funds available to Liberian NGOs, such as the YMCA. Social Impact, Inc. October 19, 2006 17 construct considerations became an important factor in the approval of projects as the program was coming up on its cut-off date of May 31, 2006. Timeline for Implementation of YES Plus It is important to understand the time frame OTI had to work with for implementing the YES Plus program. OTI made YES Plus Cooperative Agreement awards to Mercy Corps and World Vision as head of a consortium directly in March and April 2005. The YES Plus operational plan was completed in April 2005; however, OTI, Creative Associates and all IP staff, stated that severe problems arose from a lack of clarity about roles, procedures, responsibilities and accountability among the several implementing organizations, and that the coordination issues had been greatly underestimated, causing a good deal of confusion and hostility, some of which continues. Several IP respondents noted that the rush to get the YES Plus program underway meant that major program decisions had to be made before the Pilot program was completed. As a result, they say that lessons learned from that program were not fully integrated into the YES Plus roll out. To alleviate some of these problems, OTI prepared a detailed grant making manual by June 2005, making it possible for CA and the IPs to better coordinate. May 2005 marks the first group of grants (11) for YES-type projects that follow from the Pilot Program. In July 2005, OTI's permanent Country Representative cleared 14 grants for YES Plus, including Training of Trainers for YES Curriculum. In addition to the issuance of the Grants Manual, the Training of Trainers was one of the first steps in the roll-out of the YES program. Once trainers were trained, Learning Facilitators selected and trained, the five month YES training in the communities was able to begin. From August to December 2005 OTI cleared 308 YES grants totaling $2.28 million, mostly for YES small grants. The OTI data base page where actual implementation notes are entered by LTI field staff indicates that most projects did not start until January and February 2006, with May 31, 2006 as the date set by OTI for completion, which meant that more than 300 projects spread over nine counties had to be completed within four to five months. All implementing respondents reported that the task of completing the projects in such a short time frame absorbed almost all the energy of the project, leaving little left over for considerations of quality or whether expected results were being achieved. Time was too limited, and the difficulties of getting materials to projects, insuring that local contributions had been completed, and dealing with escalating costs put pressure on everyone in the project. CYPEP training (without projects) continued with additional neighborhoods around Monrovia, and expanded to five other cities using local NGOs as implementers with grants directly from CA. CA worked with the YMCA and other NGOs to engage CYPEP youth in follow-up activities such as campaigns to inform voters of the election process and of their rights and obligations as citizens.22 22 The OTI Grant Data Base lists 31 grants between December 2004 and June 2006 that were not directly connected to YES Pilot or YES Plus community projects. Of these 16 were for the ALP and 9 were for CYPEP or YES Plus related such as two grants to IRDO for providing skills training to YES Plus communities. Other than ALP, it appears that most of the 10 percent of grant funds reserved for other OTI purposes was for the expansion of CYPEP into smaller towns such as Buchanan. Social Impact, Inc. October 19, 2006 18 Reintegration of war affected communities According to OTI respondents, the OTI decision to focus on youth was a strategic one, based on two factors that became apparent in 2004. First, many young people were not returning to their rural communities and second, the number of ex-combatants grew from an initial estimate of 50,000 to over 100,000. The US Ambassador to Liberia had urged all US agencies to “drastically step up reintegration activities very early in 2004,” which was a major reason given for OTI’s involvement in such a large effort. An appeal by UNMIL added to the pressure beginning as early as 2003. These factors contributed to OTI’s development and expansion of CYPEP and to the significant commitment of OTI resources and time to the YES Plus program in LTI 2.23 Targeting communities for YES Plus Was the YES Plus program well targeted? This depends on the objective of the program. The underlying motivation for YES Plus, so far as we can determine, was not always clear or well thought out. Targeting youth, as defined in Liberia, meant anyone from 1 to 35 years of age. Demographic estimates for Liberia are that 55 percent of the population falls into this category. If half of those were mature youth at age 17 or above, than the total is about 935,000 persons. Of those, the UN estimate of former combatants was about 103,000, of which about 20,000 were young men and women. The YES Plus program, at best, sought to reach between 12,000 and 15,000 youth, whatever their situation. Moreover, by relying on Mercy Corps, Action Aid, and World Vision for implementation of the program, it was also accepting the county locations where those programs were already operating on the reasonable assumption that these organizations knew the territory and could move quickly to implement YES Plus. The down side is that these counties generally speaking were not in the highest rank of IDP populations as estimated by UNHCR and, moreover, the criteria by which communities were selected did not include any analysis of whether the problem of reintegration or potential violence was particularly pressing in any given community. We were informed that selection criteria included accessibility, number of youth, availability of suitable Learning Facilitators, and commitment of the community to the entire YES Plus project. A fair assumption was that all communities had been displaced, and all were facing severe problems of reconstruction, etc, but reconstruction and development were not the stated purposes of the YES Plus program. An examination of UNHCR data on the county by county distribution of refugees IDPs shows that while there was a good fit between YES Plus efforts and the numbers of IDPs in Bomi and Bong, the YES Plus programs in Grand Cape Mount, Grand Bassa, Grand Gedeh, and Margibi were much less congruent with a refugee reintegration problem, at least at the county level. Lofa County, which had by far the largest number of IDPs, received no attention from YES Plus, and only a few LTI grant funded projects.24 23 This point was made forcefully by OTI commentators reviewing the working draft of this evaluation. 24 OTI commentators stated they were aware of this, but distance/time considerations as well as the presence of other programs decided OTI against any effort in Lofa County. Social Impact, Inc. October 19, 2006 19 Exhibit 8: OTI Level of Effort compared to UNHCR Estimate of Number of Returning Refugees By County (All LTI Grants to August 2006) OTI Level of Effort Rank Order by $ value per County OTI Level of Effort by # of grants UNHCR Estimate # returning refugees UNHCR Returnee Rank 1 highest – 5 lowest Montserrado $3,959,304 169 38,900-89,300 2 Bomi $986,478 50 1,700-14,600 4 Bong $887,038 68 14,600-38900 3 Grand Cape Mount $882,482 73 14,600-38,900 3 Nimba $835,863 46 14,600-38,900 3 Margibi $679,063 44 1,700-14,600 4 Grand Bassa $543,368 44 1,700-14,600 4 Grand Gedeh $482,290 43 14,600-38,900 3 Gbarpolu $346,930 23 1,700-14,600 4 Sinoe $133,447 16 1,700-14,600 4 Lofa $111,080 3 89,300-89,400 1 Maryland $ 0 0 38,900-89,300 2 Grand Kru $ 0 0 1,700–14,600 4 River Cess and Gee $0 0 400-1,700 5 Findings from Evaluation Field Visits: Conflict and re-integration Communities selected for evaluation field visits represented considerable diversity in the level of war related distress, need for reintegration, and potential for future conflict. They ranged in size from 45 people to between 1,000 and 2,000. All but one that we visited had lost population from the war; in some as much as one half had not returned. Communities visited in Bong and Bomi, especially communities along roads and in strategic points (Bong Mines), suffered considerable devastation, others less so. When asked, few would describe their communities as having had serious internal conflicts resulting from the war.25 In general, we found that all villages had suffered and populations had to flee, most going to IDP camps and, in border areas, to other countries. They returned to wrecked homes and overgrown fields, but the issue of internal community conflict was not as apparent as were the communities’ desire for education, income and a better material existence. There were allusions to peer and intergenerational disputes, and in larger, more dispersed villages, conflicts arose between satellite and core (older) parts of the community. It was also apparent that by the time the YES Plus project got underway, many villages had already received one or more foreign assisted projects, ranging from schools to village hand pumps; unlike YES projects, these did not require village participation. 25 The evaluators are very much aware that this is a delicate subject about which people would not be willing to give specifics. However, when we asked in very general terms about conflict in the past and present, most communities stated that they did not have serious problems. Some did, however, citing conflicts such as between villages, among peer groups (young men and women, for example), and over who would contribute and how much to the required YES Plus project. Social Impact, Inc. October 19, 2006 20 YES Plus Projects Operational Issues A heroic effort was made by everyone to complete most projects on time. There were 49 cancellations, but many more are not actually completed according to specifications. Costs increased as YES projects had to purchase goods in a market place in competition with all other donor projects and the market could not respond adequately to this surge in demand. The limit of $5,000 per project was often exceeded. Technical problems emerged. Creative Associates made design errors in blueprints for Guest Houses and Town Halls, Rice/Cassava Mills and Animals for Skills projects had to be imported which were not suitable for Liberia. The mills had to be modified at OTI expense. Many communities still do not have mills in place. Many of the animals died; in some villages all the animals died, according to one IP field manager.26 The contractual structure of the program also created problems. Creative Associates was responsible for grant approvals and managing the implementation process, but each of the IP also had responsibilities, mainly managing the project selection process and making a determination that communities had completed their contribution. Since many communities had difficulties in this, Creative Associates had to intercede with OTI to get changes in the budget to pay for unanticipated costs. When projects were not completed, or there was a difference of view about what participation meant, the Implementing Partners felt that they would receive the complaints of the communities because of what they viewed as arbitrary decisions made elsewhere. All respondents agreed that the grant aspect of the project was very management intensive and gave rise to much hostility and difficulty between all implementing stakeholders. That these issues did not show up during the Pilot Phase is testimony to having clear lines of authority and well defined responsibilities for all players. YES Project Impact and Problems Assessed Problems and Issues For some communities, the projects were as divisive as they were constructive. Questions of ownership (of goats, for example), sharing the benefits, and managing the projects once they came to fruition were not addressed in the project’s design or implementation process. Larger and more dispersed communities had difficulty scheduling voluntary work on the YES Plus project, forcing elders to intervene and direct labor contributions. The grant program was too ambitious, under funded, and did not allow enough time to do quality work. In August 2006, three months after the OTI Grant Database recorded all projects completed, the evaluation team found that guest houses had no furniture and were not functional, and, in some cases like Bomi, not completed. Mill houses did not have mills, animal husbandry projects were faltering (animals died, pens broken down and animals “free ranging”), and latrines were not being used. Pumps, town halls, market halls, and multi-purpose buildings were more successful. Soap making projects seemed more successful, but communities were still 26 IP Field Manager stated that imported goats were not well suited to Liberian conditions, and many died. Access to animal health care was limited. CA also stated that mills had to be refitted for Liberian usage, and were gradually being installed. Social Impact, Inc. October 19, 2006 21 using original supply of free inputs. Whether they will continue with this enterprise or not is open to future observation. The theory of YES Plus project (reinforcing lessons of cooperation and democracy) was not supported by what happened in most communities. This is a critical finding. Many factors intruded, including the gap in time between selection and the actual realization of the project during which the initial enthusiasm waned. Other contributing factors were cited and included the following: the difficulties in organizing labor and getting materials to meet the community contribution; the competing demands of rebuilding homes, getting crops in, and the normal, labor intensive household tasks; the sense for some communities that the project was not their real choice and was not going to do much for them. Elders in the community often took over the process in order to get the work done. Learning Democracy Asked how they would decide on a new project, a few communities said they would decide using the election process, but others already knew what they wanted, or would discuss with elders and decide that way. For those communities where people said they would use the election process taught by YES, when asked what if the outcome was something other than what many of them wanted, all but one said they would not accept the majority rule. This suggests that communities learned how to use voting to make decisions, and even enjoyed it, but that it would not be used again, or, if the outcome was not acceptable to elders, it would not be respected. Economic benefits Little thought was given to the economic rationale for projects. In some areas, everyone wanted a guest house. IP field workers were generally not attuned to dealing with economic management issues. Additionally, little thought was given to the business/management side of projects, i.e. how would the guest house be furnished and maintained? How would the productive enterprises be managed? These questions are now being asked by some of the communities, but they are receiving no answers. YES Plus Intermediate Conclusions YES Modules There is considerable evidence that YES participants, both youth and older people, found YES training to be interesting and in specific areas, valuable to them in both practical terms (learning to write and count) as well as in relational and psychological terms. Although the drop out rate remained high in terms of formal YES participants, there is little doubt that YES training reached a much larger audience in the communities, as interest and time warranted. It is also the case that some of the conceptual and language difficulties noted in the pilot assessment were not addressed and some modules, especially the module dealing with sexuality and HIV/AIDs, were difficult to negotiate. A major part of the justification for YES was that it would help solve the problem of reintegrating youth into the local communities. Aggregate level statistics about the number of IDPs, ex-combatants, and the percentage of youth in the population as a whole can be misleading when used to describe the character of each community where YES Plus was implemented. Of Social Impact, Inc. October 19, 2006 22 the 14 communities visited, some did have ex-combatants, but for most, these people had left for other places. Many others reported that they had been dispersed into several locations, including refugee and IDP camps, but they had come back to face the job of rebuilding their homes, reclaiming their farms, and getting on with life. The “reintegration of youth” into the village did not stand out as the most pressing problem faced by the YES Plus communities visited by the evaluation team. Another objective was for YES Plus to contribute to conflict resolution. Remarking on what they learned from YES, young people especially said they had learned to respect each other and to try to start dialogues over differences. This level of conflict seemed to be of a type typically associated with adolescents or between adolescents and older generations found in all societies. In short, the level and character of the problem of “conflict” as described in program documents did not match up well with what we learned about the character of the YES communities we visited. In a few cases, the introduction of voting systems for selecting the type of project, as well as issues about where the project would be located, and who would control any benefits, led to divisiveness and may have exacerbated conflict. YES training did have a positive impact, but the project experience did not contribute much to lessons learned in YES. Under the best of circumstances, one very positive experience is not sufficient to fundamentally change attitudes and behaviors. Many YES Plus projects were rushed, unresponsive to variations in needs and on the ground conditions, timing, and costs, thereby leading to huge management problems, conflicts and infighting among implementing partners and sometimes disappointment for communities. Implementation was very management intensive. Moreover, in some cases, when it came time for YES participants to do the actual work of assembling sand and gravel, and making bricks, difficulties set in and elders took over. Local labor was frequently paid, voluntary contributions notwithstanding. VI. GENERAL CONCLUSIONS 1. OTI’s contribution to the Liberian transition process was substantial, especially in the first year of the program. OTI worked closely with the US Embassy and USAID to meet the critical political needs of restoring government and civil society functions, increasing public knowledge and understanding of the issues, and promoting community integration in those parts of the Liberian population that posed the greatest danger to the fragile peace process. It used its special mandate to address these objectives by working closely and quickly to support Liberian governmental and civil institutions, most of which were not functioning. OTI demonstrated innovative and effective responses to transition crises in the CYPEP and other programs. Overall, US foreign policy interests and objectives were well served by the OTI LTI program, especially in LTI 1. 2. OTI’s effort to mount a national program focused on youth and community reintegration was less successful, although many positive benefits were provided at the individual level community levels. The YES Plus program was, in its design and implementation, not well matched to Liberian realities. OTI planners were rushed, and underestimated the operational, resource mobilization and time factors that would affect the program’s implementation. Extraordinary efforts had to be made to successfully complete most of the projects, but with questionable impact for the psychological and community integration objectives of the program. What emerged was substantially less than what was expected. YES Plus bore a close resemblance, in theory, to a donor assisted community development program, with too little time Social Impact, Inc. October 19, 2006 23 to substantially change attitudes, behaviors, and institutional arrangements supportive of longer term development transformation. Since OTI’s mandate is to design and implement “fast and flexible” programs, it is difficult to grasp why it undertook such an ambitious and complex program to implement as YES Plus. VII. LESSONS LEARNED 1. OTI was a creative solution to the emergence of war torn, post-conflict situations. Given extraordinary authority to move quickly, OTI’s mandate was not long-term development, but to bring immediate evidence of the benefits of peace to a war stressed population, and to reinforce that peace by using the grant mechanism to foster effective governance and social and political cooperation. For OTI, the process of helping communities and governments become functional again was more important than the product, whether it be a rehabilitated school, a playground, or a guest house. OTI serves US foreign policy interests when it demonstrates American commitment to peace and the fostering of cooperation through democratic processes as evidenced by the Liberia program. 2. OTI’s mandate may be best served not just by making grants, but by engaging with its grantees in a purposeful and consistent way. In Liberia, OTI’s commitment to the Truth and Reconciliation Commission is one of several examples. According to the OTI Representative in Liberia, there are four elements to this tactical approach: 1) do due diligence on the problem and on the potential grantee; 2) start small and assess results; 3) develop partnerships and networks of grantees with a more strategic objective; and 4) be prepared to make multiple grants to those organizations that demonstrate competence, commitment, and ability to produce results. The other side of this tactical approach is that it fits well with OTI’s operational code of being opportunistic, risk taking, innovative, and political. 3. In its second year in Liberia, OTI chose to carry out what most observers, and implementing partners agreed resembled a community development program similar to what USAID might have undertaken. Several lessons emerge from this experience. First, OTI should avoid commitments that tie up its budgetary and staff resources to the extent that YES Plus did. Second, given limited time and staff, avoid formulaic concepts and complex program designs that require far more time and resources than are typically available to OTI. Third, be cautious of adapting lessons learned from other programs if there is not adequate time to test and adjust to the cultural and social realities of the country. Fourth, do not commit all or most of the resources to one, long-term program; rather, experiment; be flexible, and use resources to learn as you go, and to reward your counterparts’ progress. The lesson learned is that OTI should be wary of straying too far from its mandate and its proven track record of implementing programs that result in immediate benefits for the populations of societies emerging from devastating conflict or natural disaster. 4. OTI can play a role in testing out innovations that could become development programs funded by USAID or others. In this respect, the pilot phase of OTI LTI 1 YES was such a worthwhile effort because it allowed OTI to move quickly into the CYPEP program. By all accounts this was a classic quick response to a serious political problem stemming from peri￾urban riots. OTI’s grants to indigenous organizations dedicated to fostering human rights, advocating for tough anticorruption measures, and development of responsible journalism make good use of OTI's mandate and, in some ways, allow for greater flexibility and engagement than USAID can exhibit in its civil society and democracy programs. The lesson learned is that OTI Social Impact, Inc. October 19, 2006 24 can be an innovator, incubator, and pilot tester of programs that do have long-term development consequences. As OTI and USAID work together, it would be instructive for USAID to adopt some of the flexibility and speedy commitment that OTI exhibits. 5. There is a relationship between a successful process and a beneficial product. OTI programs need to pay attention to both, especially at the local level. The idea behind YES, and for that matter, some of the other classic grant making OTI did in Liberia, was that active engagement in a useful enterprise would help to reinforce the lessons learned from the training process. This is a sound idea, but if people see little or only temporary benefits from their engagement in the enterprise, or if the enterprise itself was not a high priority for them and they do not really own it, the product may well undermine the positive learning that was derived from the process. 6. Organizational learning is a critically important process which requires discipline, constant dialogue, and some way of correctly assessing past experience. LTI YES built on recommendations from a similar program in Sierra Leone, and Burundi and Congo. But there were other factors operating in the Liberia context that should have been considered before OTI committed to a one year national program such as YES Plus. Many of these were known towards the end of the YES pilot phase, but not enough was done to address them. Social Impact, Inc. October 19, 2006 25 Selected Bibliography CAII. 2004. Youth Education for Life Skill, Final Design Team Report CARE/CAII. 2002. Final Evaluation of the OTI Program in Sierra Leone. Mercy Corps. 2004. Youth Education for Life Skills (YES) Pilot Program, July 1, 2004 – January 31, 2005, Proposal. Office of Transition Initiatives. OTI Special Tenth-Year Edition, 1994-2004 OTI. 2003. Assessment Report, Liberia Pockenpaugh, Jason. 2004. “On the Heels of Disaster.” Government Executive (pp 55-57). Rotberg, Robert I. 2005. The First Ten Years: An Assessment of the Office of Transition Initiatives. USAID/DART/APU/Liberia. 2003. “Operation Closing Time,” Exacts its Toll: A Comprehensive Analysis of Protection Problems in Liberia. USAID/DCHA/OTI. 2003. Congressional Report – Liberia. ________________. 2004. Strategic Plan for the Liberian Transition Initiative (LTI) Draft. USAID/Liberia. 2005. Youth Education for Life Skills (YES) Community Grants Procedures Guide. USAID/OTI. 2006. Annual Report, FY 2006. _______________. Community-Focused Reintegration Programs in the Democratic Republic of Congo and Burundi, Final Evaluation. ______________. Community Focused Reintegration Programs in Burundi, The Democratic Republic of the Congo and Liberia. USAID/OTI/CAII. Accelerated Learning Program Handbook. USAID/OTI/LTI. 2004. Accelerated Learning Program Strategic Framework. ______________. 2006. Community Youth Peace Education Program (CYPEP), Mid-Term Evaluation, Final Report. World Vision, Inc. Washington DC. 2005. Youth Education for Life Skills (YES), Annex II, Technical Proposal. Social Impact, Inc. October 19, 2006 26 ANNEX 1: Scope of Work FINAL EVALUATION LIBERIA TRANSITION INITIATIVE USAID/DCHA/OTI/LIBERIA STATEMENT OF WORK PDQ TASK ORDER UNDER IQC HDA-I-00-03-000124 Introduction USAID's Office of Transition Initiatives’ (OTI) mission is to support U.S. foreign policy objectives by helping local partners advance peace and democracy in priority countries in crises. Seizing critical windows of opportunity, OTI works on the ground to provide fast, flexible, short￾term assistance targeted at key political transition and stabilization needs. OTI’s program in Liberia was established in February 2004 to help advance an inclusive, peaceful, political transition in Liberia following a 14-year civil war that ended with the 2003 Comprehensive Peace Agreement. The three core objectives27 of this program have been to: 1. Restore critical transitional governance and civil society functions, 2. Increase public understanding of key political transition issues, and 3. Promote community reintegration and peaceful resolution of conflict OTI works closely with implementing partners Creative Associates International, Inc., Mercy Corps, and the World Vision Consortium in Liberia to implement four distinct but mutually reinforcing program components to achieve these core objectives: · The Youth Education for Life Skills (YES) program, a community-focused reintegration activity. The goal of YES is to enhance peace through non-formal education. YES helps youth, one of the largest and most war-affected populations of Liberia, become productive members of their communities. An offshoot of the YES program, entitled the Community Youth Peace Education Program (CYPEP) is a more abridged reintegration program targeted specifically at Liberia's urban youth · Small grants through a participatory process that reinforces community cohesion, assists in reconstruction and rural development, and facilitates youth leadership development. · Small grants to national organizations in the fields of justice, media, conflict management, good governance, and human rights · Technical assistance to the Ministry of Education and USAID/Liberia's Accelerated Learning Program (ALP) With the successful inauguration of a new democratically elected government, OTI’s current program focus is ending while another strategy and program is currently being devised. OTI is working with the USAID Mission to clarify which OTI’s programmatic elements might be incorporated into the Mission’s long term development strategy. 27 As defined in the March 2006 Fact Sheet on OTI’s Liberia webpage, http://www.usaid.gov/our_work/cross￾cutting_programs/transition_initiatives/country/liberia/fact0306.html Social Impact, Inc. October 19, 2006 27 Additional information on OTI’s program in Liberia, including reports, success stories, and fact sheets, can be found at OTI’s website at www.usaid.gov, keyword OTI or at http://www.usaid.gov/our_work/cross-cutting_programs/transition_initiatives/country/liberia/index.html Purpose OTI seeks an independent evaluation for the Liberia program between February 2004 and September 2006. The nature of OTI’s mandate, including its short-term objectives, shall be a driving factor in the evaluation. The evaluation will help document lessons learned that may assist in other OTI country programs, and may also be used as a tool when planning similar programming in Liberia and other countries. There are two sections to the evaluation. The approximate weighting of each section is indicated below. 1) OTI Liberia Program (90% of the final report) a) The core of OTI’s strategy in Liberia was to advance prospects for an inclusive, peaceful, political transition in Liberia in the context of the 2003 Comprehensive Peace Accord. Did OTI succeed in implementing this strategy, and in what ways could its efforts have been improved? Evaluators shall provide specific examples of where OTI demonstrated significant impact strengthening democratic processes or increasing momentum for peaceful resolution of conflict. b) Was the program strategic in responding to shifts in the transition process in terms of the evolving political and security situation inside Liberia? Specifically state reasons behind an adaptation to the strategy and the outcome. Within this context, did the program meet its stated goals and objectives? c) What were the strengths and weaknesses of OTI’s community focused reintegration programming; and did they properly use lessons learned from the Youth Reintegration Training and Education for Peace Program (YRTEP) program in Sierra Leone28 d) Discuss the program’s operational successes and shortcomings and identify lessons learned and best practices from which future programming will benefit. 2) OTI’s Mission Statement and the Liberia Program (10% of the final report) This section of the final report should not repeat facts, observations, or findings from the first section. Rather it should answer the questions and justify the answer with a summary, referring back to Section 1 if necessary. These questions are for OTI’s Annual Report and other reporting needs. a) Did the program support U.S. foreign policy objectives?29 If yes, did it accomplish this by helping local partners advance peace and democracy? b) Did OTI work on the ground to provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs? c) Did the program adapt, as necessary, to ongoing political developments/milestones relevant to the premises of the OTI program in Liberia? Put another way, did OTI’s program seizing critical windows of opportunity?30 28 Evaluations of the OTI programs in Sierra Leone can be found on OTI’s website under ‘Publications’ or at http://www.usaid.gov/our_work/cross-cutting_programs/transition_initiatives/pubs.html 29 As defined in USAID/Liberia website and in the 2005 Annual Report found on that site at http://www.usaid.gov/locations/sub-saharan_africa/countries/liberia/index.html Social Impact, Inc. October 19, 2006 28 d) Did the program, within 18 months of startup, have a clear strategy and plan for continuation of activities/actors relevant to the specific objectives of the OTI program in Liberia? e) Does the evaluation find significant impact in strengthening democratic processes and/or increasing momentum for peaceful resolution of conflict? Note that for all questions, strategy in this context is more concerned with short-term ingenuity and constancy in achieving medium-term objectives than with long-term strategic vision. Tasks This scope of work is for the following tasks: 1. Recruit two-person final evaluation team; 2. Determine, together with OTI, an appropriate methodology (including questionnaires) for the evaluation; 3. Evaluate the performance and impact of the Liberia program through: a. Interviews in Washington DC with current OTI staff, as well as staff from partners Creative Associates International, Inc., Mercy Corps, and World Vision b. A field review with interviews with OTI, USAID Mission, US Embassy, and other USG personnel as needed; Government of Liberia representatives; implementing partners; and program beneficiaries. 4. Provide an out-briefing to OTI in Liberia before departure 5. Document, in a draft evaluation report for OTI distribution and comment, findings, conclusions and lessons learned from the program, as well as recommendations for the future (details on the report are presented below). 6. Provide a briefing to OTI upon completion of the draft report (including PowerPoint). 7. Provide a final evaluation report for public distribution. 8. Provide an official presentation in Washington on the final report. A substantial amount of information, documentation, and data exists on the program. The evaluation team will first conduct a desk study of existing documentation on the program before conducting the field review and interviews. OTI staff will provide electronic versions of all relevant documents. Team Composition 1. One senior-level evaluator with extensive experience designing and conducting evaluations of programs in fragile states. The senior level evaluator will serve as team leader and be responsible for the field review, interviews, the draft and final evaluation reports, and for debriefs in Washington, DC. 2. One mid-level evaluator with experience conducting evaluations of programs in fragile states. Both evaluators shall have experience with political transition / post-conflict programs such as those implemented by OTI. In addition, at least one evaluator shall have experience working in Liberia. Other West African experience will be beneficial. 30 For more information on this as a criteria for OTI’s engagement, refer to the Criteria for Engagement under “About OTI”, found at http://www.usaid.gov/our_work/cross-cutting_programs/transition_initiatives/aboutoti3.html Social Impact, Inc. October 19, 2006 29 Deliverables The contractor shall provide the following deliverables: 1. Brief outline of methodological approach for assessments, including proposed itinerary, schedule for interviews, and identification of all logistical support needs; 2. Draft evaluation report, not to exceed 20 pages single-sided, with Times New Roman 12 point (or equivalent) font, plus additional annexes (report and annexes to be submitted electronically). Examples of past evaluations are available on the publications section of OTI’s website. The format of the report is flexible. However, the report shall include photographs (to be taken by the evaluators and/or to be selected from OTI and Implementing Partners’ photograph collections). The report shall also the following sections: Table of Contents, Acronyms, Executive Summary, Background (OTI’s mission and general approach to programming, country context, evaluation objectives and methodology, overall observations, findings (answers to questions in scope), conclusions, recommendations. 3. Brief Draft Power Point presentation summarizing conclusions and recommendations (for the first presentation); 4. Final evaluation report, deliverable no later than two weeks after receipt of all comments from OTI on first draft. A total of 20 bound copies should be delivered to OTI in addition to an electronic copy (Microsoft Word). 5. Final PowerPoint presentation to be submitted at the same time as the final report (to be submitted electronically). *Note that the PowerPoint presentation and final document should adhere to USAID’s branding policy (www.usaid.gov/branding). Proposed Time frame Task Order period June 15-September 30, 2006. Field Visit in July 2006; draft report submitted by August 31; final report (after receiving OTI’s comments) due by September 30, 2006. Suggested Level of Effort Task LOE (days) Initial meeting with Program Team and Database Training 2 Desk review of program documents 7 Preparation of proposed methodology and meeting with OTI 3 Initial interviews with staff in Washington DC 4 Field Review and interviews (14 days) 28 Additional interviews in Washington DC as needed 2 Preparation of draft report 10 Presentation of findings, recommendations and discussion of draft 4 Preparation of final report 10 Total 70 Social Impact, Inc. October 19, 2006 30 Government Furnished Documents and Assistance 1. Suggested Interviewee List (Washington and Liberia) with contact information. OTI will provide the VOIP number for USAID/Liberia 2. OTI will provide database training on OTI’s Grants Database. 3. Liberia Grants database, Monitoring and Evaluation documentation, Program Performance Review (PPR) documents, country strategy documents, previous assessments, partners’ statements of work, contractor monthly reports, etc. 4. Monthly reports, success stories, and other documents are available on the OTI website 5. OTI will assist in arranging in country logistics and the itinerary, however, the evaluation contractor would be responsible for all expenses including domestic flights and accommodation. Social Impact, Inc. October 19, 2006 31 ANNEX 2: Liberian Context The Conflict For the past 26 years the Liberian experience has been one of social instability caused by civil conflict. Events contributing to this state of affairs include a military coup (1980); two civil wars (1989 and 2003); two contentious multi-party elections (1985 and1997); three interim governments and 14 aborted peace agreements amid intervention by the Economic Community of West African States (ECOWAS), which, from1990-1999, fielded its Military Observer Group (ECOMOG) in an attempt to establish peace. Finally, in August 2003, following Former President Charles Taylor’s exile in Nigeria, representatives from the belligerents, LURD and MODEL (Liberians United for Reconciliation and Democracy and the Movement for Democracy in Liberia), the Government of Liberia (GOL), major political parties and civil society signed the Comprehensive Peace Accord (CPA) in Accra, Ghana and, at the same time, selected the National Transitional Government of Liberia (NTGL) to govern the country while preparing for elections in October 2005. The Humanitarian Response As the result of Civil War 200,000 Liberian civilians were killed; 500,000 were displaced inside the country and 800,000 became refugees in neighboring countries. Schools, clinics, roads, and markets were destroyed or severely damaged. At the same time, agricultural production was limited as was access to basic social services. In response to the devastation of the country’s infrastructure and to the displacement of its population the United Nations and its myriad agencies as well as the European Union, USAID, other bilateral donors and international NGOs provided the civilian population protection, food, water and sanitation, shelter, health services, education and skills training. In many cases these multilateral and bilateral donors worked with local NGOs, churches, mosques, and other civil society organizations to provide basic needs and to address human rights violations and security issues. Implementation of UN Resolution 1509 In September 2003 the Security Council passed UN Resolution 1509 authorizing the deployment of 15,000 peacekeepers to Liberia. By December, 5000 or 1/3 of the 15,000 UNMIL troops had been deployed, which meant civilians in 85 percent of the country were still subjected to human rights violations by LURD forces in Lofa, Grand Cape Mount, Gbarpolu and Bomi; by MODEL in Sinoe, Grand Kru, River Gee and Maryland; and by the GOL and MODEL in Nimba. It also meant that areas beyond Monrovia and its immediate suburbs remained insecure and inaccessible to humanitarian aid agencies until the end of December 2004 when the full complement of peacekeepers was deployed throughout the country. The deployment of peacekeepers had implications for Disarmament, Demobilization, Rehabilitation and Reintegration (DDRR) as well. Initially, scheduled to begin in December 2003, the exercise was postponed until April 2004 when proper planning and logistical support avoided the violence that had occurred in December. In any case 103,000 combatants were disarmed: 33,342 males; 22,370 females; 8,532 boys; 2,440 girls. Observers of the process charged the UN with having focused on disarmament and demobilization to the detriment of rehabilitation and reintegration programs, a charge the UN has denied. Social Impact, Inc. October 19, 2006 32 Post-Conflict Reconstruction The task of rebuilding Liberia has required massive assistance from the international community. As a consequence, the agencies, donors, international and local NGOs along with civil society organizations, referred to above, are assisting the Liberian government initiate post-conflict reconstruction activities designed to restore normalcy. As might be expected the USAID is a major party to this effort both on its own terms and in relation to its specialized offices. For example, USAID’s Disaster Assistance and Response Team (DART) as well as OTI have been involved in providing assistance in the post-conflict environment and continue to be. In August 2003, just prior to Taylor’s departure and the signing of the CPA, DART entered Liberia from Sierra Leone. And in response to OTI’s request to have the Abuse Prevention Unit (APU) represented on DART an APU representative was assigned to Liberia that September. By December 2003 the first Temporary Duty (TDY) personnel began OTI’s longer term presence in Liberia. Significantly, the CPA that established the NTGL led to UN Resolution 1509, which allowed the Security Council to field 15,000 Peacekeepers. As the UNMIL deployed throughout the country security and access to humanitarian assistance improved for populations in areas where fighters associated with LURD and MODEL had continued to harass civilians. Since the inauguration of Ellen Johnson-Sirleaf in January 2006 as the first woman elected president of an African country, several hundred thousand internally displaced persons and refugees have returned to their respective communities. Although they have come back to depopulated villages where adequate housing, potable water, health care, education, agricultural inputs for farming and transportation to local markets are in short supply, returnees are hopeful circumstances will improve. Reportedly, some villagers are still refugees in neighboring countries where children are in school, others are in Monrovia, which is overcrowded and, in spite of UNMIL’s presence, experiencing an increase in crime, particularly armed robberies. Social Impact, Inc. October 19, 2006 33 ANNEX 3: Answers to SOW Part 2 The Scope of Work for this evaluation focused mainly (90 percent) on the design and implementation of OTI’s Liberian Transition Initiative (LTI) Program. The SOW established a limit of 20 pages for the entire report. The evaluation team concluded that the LTI was of such complexity and importance that the report should use the entire 20 pages to report on the team’s findings, conclusions and lessons learned. Also, since answers to Part 2 questions, according to the SOW language repeated below, were to be derived from the findings of Part 1, the team was justified in preparing a separate annex to present the summary answers to Part 2 questions. The OTI SOW language setting out instructions and questions for part 2 as follows: OTI’s Mission Statement and the Liberia Program (10% of the final report) This section of the final report should not repeat facts, observations, or findings from the first section. Rather it should answer the questions and justify the answer with a summary, referring back to Section 1 if necessary. These questions are for OTI’s Annual Report and other reporting needs. a) Did the program support U.S. foreign policy objectives? If yes, did it accomplish this by helping local partners advance peace and democracy? b) Did OTI work on the ground to provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs? c) Did the program adapt, as necessary, to ongoing political developments/milestones relevant to the premises of the OTI program in Liberia? Put another way, did OTI’s program seizing critical windows of opportunity? d) Did the program, within 18 months of startup, have a clear strategy and plan for continuation of activities/actors relevant to the specific objectives of the OTI program in Liberia? e) Does the evaluation find significant impact in strengthening democratic processes and/or increasing momentum for peaceful resolution of conflict? Note that for all questions, strategy in this context is more concerned with short-term ingenuity and constancy in achieving medium-term objectives than with long-term strategic vision. Answers to Part 2 questions follow in summary form a. Did the program support U.S. foreign policy objectives? If yes, did it accomplish this by helping partners advance peace and democracy? The OTI Liberian Transition Initiative did support U.S. foreign policy objectives as demonstrated in the main body of the evaluation report. This conclusion is confirmed by testimony of U.S. embassy officials as well as the evaluation team’s analysis. The report provides evidence and examples of how OTI’s programs supported the interim government, helped civil society to function, addressed important issues such as reconciliation, public understanding, and government accountability. Contributions to peace were manifest in youth organizational efforts, especially the CYPEP program. b. Did OTI work on the ground to provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs? Social Impact, Inc. October 19, 2006 34 The LTI program was especially effective in this regard during LTI 1; with the exception of delays in getting the grant program underway until five months after the first OTI Representative arrived in country. OTI’s response to one key political problem, how to re-direct the energies of Liberian youth toward constructive engagement and re-integration, was less successful, and moved OTI away from being ‘fast, flexible and short term’. c. Did the program adapt, as necessary, to ongoing political developments/milestones relevant to the premises of the OTI program in Liberia? Put another way, did OTI’s program seizing critical windows of opportunity?(sic) It is not clear exactly what is meant by “ongoing political developments/milestones relevant to the premises of the OTI program...” The evaluation did find evidence of the program having seized critical windows of opportunity. The best example of ‘seizing critical windows’ may be the Liberian University intervention as well as the CYPEP program, but other smaller grants for media development and public interest advocacy were also notable. d. Did the program, within 18 months of startup, have a clear strategy and plan for continuation of activities/actors relevant to the specific objectives of the OTI program in Liberia? The start-up of the OTI program in Liberia is a matter of some discussion between the evaluation team and OTI commentators. The team has selected December 2003 as the start date based on the arrival of the first OTI Interim Representative. After 18 months of active programming, OTI had settled on the YES strategy by July 2005, although the evidence clearly shows OTI was moving towards some sort of replication of its youth oriented program experience in Congo, Rwanda, and Sierra Leone at an early point in its Liberian engagement. The YES strategy in its various forms had expended over $3.6 million by August 2006 on promoting community reintegration, substantially more than any other OTI objective. However, by the time of this evaluation in August 2006, OTI had moved beyond the YES program in favor of a different strategy, which is still under development. e. Does the evaluation find significant impact in strengthening democratic processes and/or increasing momentum for peaceful resolution of conflict? Finding significant impact in any independent, systematic or social-scientifically valid way was well beyond the capability of a two-person team working within a ten day field research time frame. As noted in the main body of this report, the team was unable to use random selection methods or make comparisons between OTI programs and other programs or regions. Based on the limited evidence we were able to gather, it is likely that OTI programs did contribute to “strengthening democratic processes and/or increasing momentum for peaceful resolution of conflict,” although we are unable to determine with any degree of precision whether these contributions can be said to have had “significant impact.” Again, we conclude that this was more likely to have occurred in LTI 1, than in the YES Plus program, although the latter did provide other benefits of value to rural Liberian communities. Social Impact, Inc. October 19, 2006 35 ANNEX 4: Interview List I. Washington, D.C. Creative Associates, LTI Bobadilla, Paola. Program Associate, LTI Project Backstop Clark, Kim. Senior Associate, Former HQ LTI Project Director McCall, Richard. Vice President for Programs and SWIFT II Corporate Monitor Villalobos, Noy. Management Associate, HQ LTI Project Director Office of Transition Initiatives (OTI) and USAID Africa Bureau Briggs, Jeanne. OTI Program Officer Clemens, Musu. OTI Liberia Country Representative Coleman, Carolyn. Senior Education Policy Advisor, Africa Bureau Gattorn, John. OTI Liberia Program Manager Hall, Linda. Former OTI Liberia Desk Officer Henry, Daniel. Technology Sector Specialist Jenkins, Robert. OTI Acting Director Kachra, Galeeb. Program Office Program Manager Kearns, Jason. IT Programmer Kerner, Donna. Manager and Deputy Team Leader OTI Africa Krumm, Donald Senior Field Advisor / DCHA / OTI (December 2003) Langlois, John. OTI Senior Media Advisor Martin, Angela. Former OTI Team Leader for Africa Swift, Sarah M. Country Development Officer AFR/WA World Vision Brenneman, Leland. Interim Director Koomson, Anthony. Technical Support Matavel, Argentina. Departing Director Siganda, Carolyn. Finance Support, West Africa II. Monrovia American Embassy Callender, Elizabeth. USAID/OTI Deputy Country Representative Booth, Donald. Ambassador Myers, Alfreda. Political Counselor Pauling, Sharon. USAID Supervisory Program Officer Fahnbulleh, Louise. Program Assistant, USAID/OTI Liberia Creative Associates: Fischer, Shannon. Chief of Party, LTI Yarsiah, James M. LTI Program Manager Clinton-Varmah, Princetta. Grant Development Officer; Bomi, Montserrado and Gbarpolu Deond, Joe. Grant Development Officer Jensen, Kulubuh. Grant Development Officer; Rural Montserrado and Margibi Nimineh, Thomas. Grant Development Officer; Grand Bassa and Sinoe Warner, Mardia. Grant Development Officer; Bomi and Grand Cape Mount Social Impact, Inc. October 19, 2006 36 Yennego, David. Grant Development Officer; Grand Gedeh, Nimba and Bong GOL Representatives Allen, William. Director General of the Civil Service Agency, Former Minister of Information Conteh, Al-Hassan. President, University of Liberia Davis, Natti B. National Coordinator of National Reconstruction and Development Committee Mulbah, Elizabeth, Liberian Agency for Community Empowerment (LACE) Wisseh, Medina. Office of the President. Former OTI Project Development Officer Wolokolie, Jamesetta Howard. Minister of Youth and Sports Implementing Partners (CYPEP) Federation of Liberian Youth (FLY) Wisner, George Organization of Liberian Muslim Youth (OLMY) Dukuly, Ibrahim Young Men’s Christian Association (YMCA) Grey, Lyn Kerkula, Joe Paye, Lester Implementing Partners (YES) Action Aid Crayton, Massa. Director Gaie, Ernest. Director and former OTI M&E Mercy Corps Ewert, Tom. Director Doe, Michael. Program Manager Search for Common Ground, Talking Drum Studio Bloh, Oscar. Country Director Konneh, Sekou. Community Mobilizer Newland, Beatrice. Outreach Assistant Sonii, Shirk. Producer / Media Production World Vision Tarpeh, Ettie. World Vision Director Abdullai, Emma. Program Manager Siaffa, Emmanuel. Master Trainer Siminyu, Albert. Chief of Party, WAS Consortium Liberian NGOs Center for Transparency and Accountability in Liberia (CENTAL) Cummeh, G. Jasper III, Director Social Impact, Inc. October 19, 2006 37 Foundation for International Dignity (FIND) Speare, Jackson II, Program Officer Integrated Rural Development Organization (IRDO) Mentee, Sam Director Chiefs, Deans of Elders, Elders, YES Chairpersons, Participants and citizens in 14 Liberian Communities and 3 CYTEP classes- estimate 680 persons. Social Impact, Inc. October 19, 2006 38 ANNEX 5: Site Visits YES Villages Visited Grand Cape Mount County: Implementing Partner, Action Aid · Mambo · Jenewonde · Kenema · Weilor · Mami Lower Bong County: Implementing Partner, Mercy Corps · Kitikoe · Yarsiah Estates · Zangbayama · Venda · Bassa Town Margibi and Bomi Counties: Implementing Partner, World Vision · Demeah (M) · Todien · Gbanba · Sass Town CYPEP Sites Visited · Susan Berry Elementary School, Congo Town · Mile 72 · Muslim School in Clara Town on Bushrod Island YES Implementing Partner Field Staff Action Aid, Boima Coleman Mercy Corps, Dixon Wlehbo World Vision, Arthur Peters, Emmanuel Siafa CYPEP Field Staff Position, Monrovia Neighborhood Assigned To Learning Facilitators · E. Theophilus Allen, YMCA · J. Lassannah Kanneh, Clara Town · J. Bismark Karbiah, King Peter’s Town · G. Cecilia Washington, Bong Mines Master Trainers · Vivien Beh, Parker Paints · G. Baccus Karpeh, YMCA · Sidney Williams YMCA Victor Jah, Project Officer LTI Joe Kerkula, Program Coordinator, YMCA Social Impact, Inc. October 19, 2006 39 ANNEX 6: YES Question Guide OTI Liberia YES Site Visit Question Guide (August 2006) The Community 1. how many people live here? 2. is this community smaller now, or bigger than before? 3. how many houses? 4. all farmers, cassava is main crop, diversified small agriculture? 5. proximity to market? 6. other foreign aid projects (before 1990 and after 2003)? (Are they working?) 7. what happened here during the war? 8. did everyone come back? 9. what about conflict in this community? YES Training (Observe age, gender, ‘presence or authority’ of LF…note who did most of the talking…who were the ’new men’ in the community? Any “new women”?) 1. How many people here took YES when it started? (show of hands: count, note male/female, approx. age.) 2. (To LF) How many showed up at first few sessions? 3. Why did people drop out? 4. (to LF) How many received certificates. 5. What do you remember that you liked (and/or could use).? 6. What did you not like (or what could be made better?)? 7. Why did you take YES course? The Project (Get initial data from OTI Grant Database) 1. How did you decide on which project to do? (Probe for process.) 2. What were your choices? 3. Before YES came, what did you ‘really want’? 4. Why did you want a ………….. (guest house)? 5. What problems did you have? 6. Did you have skilled people here, or did you have to hire them? 7. Were the materials on time, were they the right materials? 8. Are you satisfied with the results? 9. How are you going to use the …? (mainly guest house) 10. If you had to choose between two new projects next month, how would you do it? (do they use a traditional approach, or some form of ‘majority vote’… 11. What if the majority voted for a project many of you would not want, what would be your response? (Probe for acceptance of decision made by vote, especially elders.) Do you have questions for us? Social Impact, Inc. October 19, 2006 40 ANNEX 7: Program Timeline Date Transition Event Description People involved June and July 2003 Multiple attempts to take over Monrovia by LURD rebel faction. Charles Taylor’s arrest warrant unsealed in Sierra Leone. Charles Taylor Leaves Liberia for asylum in Nigeria. Rebels Shell and loot Monrovia. US Embassy hit by mortar fire. Thousands dead related to insurgency. To date, up to 750,000 IDPs and refugees from decades of war and an estimated 250,000 dead. http://www.usaid.gov/our_work/humanitari an_assistance/disaster_assistance/countri es/mano_river/fy2004/Liberia_CE_SR05_ 10-29-2003.pdf August 6,2003 USAID DART enters Liberia from Sierra Leone OTI in talks to have the Abuse Prevention Unit (APU) represented on DART. See press release: http://www.usaid.gov/press/releases/2003/ pr030806.html Leah Werchick, Angela Martin August 11, 2003 Charles Taylor Steps down as President August 18, 2003 Comprehensive Peace Agreement signed in Accra, Ghana between LURD, MODEL, and GOL. Gyude Bryant selected by all 3 factions as transitional President 3 days later. Text of CPA: http://www.usip.org/library/pa/liberia/liberia _08182003_toc.html September, 2003 Jeff Drumtra joins the DART in Liberia as an APU representative Report written: “Operation Closing Time Exacts Its Toll” Jeff Drumtra, Leah Werchick September 19, 2003 15,000 Blue Helmet Peacekeeping Mission Approved by UNSC. October 14, 2003 NTGL begins their mandate October 22- November 5, 2003 OTI sends in a four person Assessment Team to Liberia Report written which leads to present program in Liberia Larry Meserve November 24, 2003 Congressional Notification goes to Congress explaining OTI wishes to get involved in Liberia December 8, 2003 SWIFT II Liberia IQC Hits the Streets December 8, 2003 SWIFT II Liberia Task Order Hits the Streets for IQC holder to bid on Ceiling is at $111,111,196 December 9, 2003 DDR begins and is cancelled immediately from poor cantonment site management December 9 2003- June 20 2004 First Long Term TDYers begin OTI long term presence Start Up Team. Multiple meetings with US Ambassador Blaney (substantial talk about DDR planning). Don Krumm (Dec 03-22 2003, Feb 19 to May 5), Denise Dauphinais (Jan 15 2004 to 30 March then 15 APR to 7 June), John Gattorn (Dec 03-17 2003, Mar-Jun 04), Donna Kerner (?) February 11, 2004 Creative Associates Awarded SWIFT II Contract February 14, 2004 Creative has Start-up Team in Liberia Sandy Schuster, Kim Clark, Social Impact, Inc. October 19, 2006 41 Date Transition Event Description People involved March 2004 OTI begins working with the Transitional Justice Working Group. Large scale Transitional Justice grants begin, including working with Greenberg Research on focus groups nation wide John Moreira, John Gattorn (on HR and TJ TDY for OTI) March 26, 2004 Pre-Authorization Letter (PAL) given to Creative to incur $540,000 worth of program expenses PAL needed because Creative was operational yet OAA negotiations took a very long time April 2, 2004 SWIFT II negotiations between OAA and Creative finally completed Full Funding of incremental funding begins. Paves way for $725,000 burn rate in April. Amanda Downing April 15, 2004 DDR begins again under a more coordinated system UNMIL April 2004 Powerpoint presentation given to USAID Mission Director on YES as part of the USG reintegration strategy Don Krumm, Anna Korula, Shannon Fischer April 2004 YES Program Operational Plan Completed Report lead the way for how the YES program will be implemented in Liberia, including youth targets, geographic locations, philosophy, and impact targets Anna Korula, Leslie Long, Martin Hayes June 2004 First Strategic Planning Meeting Takes Place Social Impact Moderated the activity. 4 day event. Musu Clemens arrived for it before her contract was signed to be Country Representative. Denise Dauphinais, Musu Clemens, John Gattorn, Social Impact, Anna Korula, all LTI staff, Ambassador Blaney Keynote speech, Donna Kerner. July 1 2004 Musu Clemens Begins as OTI Country Representative in Liberia July 14, 2004 Modification to SWIFT Task Order Raised ceiling to $15,255,029, added language in contract to incorporate YES August 2004 YES Pilot Phase begins To gain lessons learned for full roll-out of YES, YES Pilot small grants were implemented via Creative small Grants Denise Barrett, Mercy Corps, Shannon Fischer, Musu Clemens. September 2004 OTI given $1.5 million by USAID/Liberia of Supplemental funds To implement the ALP program in Liberia with a no competition waiver. Ed Birgells, Denise Dauphinais, Donna Kerner, Shannon Fischer October 14, 2004 Full YES RFA hits the streets $4.5 million activity. Limited competition to only organizations presently in Liberia. Deadline for proposals with November, 17, 2004. John Gattorn, Angela Martin. Sharon Bean wrote the RFA October 20, 2004 Riots disrupt Monrovia, schools and property burn down, 6 people dead. OTI engaged instigators, talked to youth. The idea for CYPEP was born. CYPEP became big focus on small grants. World Vision contract modification one year later gave CYPEP a big focus. JY, Musu Clemens October 31, 2004 DDR officially ends with over 100,000 officially disarmed. The UN initially thought 50,000 combatants would show up for DDR. UNMIL December 4,2004 Angi Yoder becomes OTI Program Manager/Deputy Country Representative in Liberia Social Impact, Inc. October 19, 2006 42 Date Transition Event Description People involved February 22, 2005 1 of 2 YES RFAs awarded to Mercy Corps Mercy Corps begins implementation of YES in Montserrado and East and South Liberia March 3, 2005 Second Strategic Planning Session for OTI in Liberia Marked huge switch in the direction of OTI in Liberia where entire 90% of focus went to YES. OTI Tdyers included Donna Kerner, John Gattorn, Mary Stewart. 30 people in attendance including World Vision, Search for Common Ground, Creative Associates, Action Aid. March 8, 2005 2 of 2 YES RFAs awarded to World Vision World Vision begins implementation of YES in Montserrado and West and North Liberia (under a consortium called WAS, standing for World Vision, ActionAid, and Search for Common Ground. June, 2005 YES Grants Manual completed A collaborative effort by Creative Associates, WAS, Mercy Corps on how to coordinate Creative Small Grants with the Mercy Corps and World Vision YES program. July 13, 2005 Ambassador John Blaney leaves Post in Liberia Embassy July 29, 2005 Ambassador Don Booth takes over as US Ambassador Embassy August 2005 Angela Martin resigns as Africa Team Leader for OTI in DC Angela Martin August 2005 Konrad Huber begins as Africa Team Leader for OTI in DC Konrad Huber October 11, 2005 First Presidential Elections Held with George Weah and Ellen Johnson Sirleaf going towards a a run-off election November 8, 2005 Run-off election with George and Ellen. Ellen wins with 59% of vote November 13, 2005 Angi Yoder resigns as OTI’s Program Manager/Deputy Country Representative in Liberia Moves on to be South Sudan Team leader for OTI December, 8, 2005 Modification to Contract Raised ceiling from $15,255,029 to $16,937,897 and moved close out date from 2/11/06 to 9/30/06 December 28, 2005 Elizabeth Callender takes over from Angi Yoder as Program Manager/Deputy Country Representative in Liberia Previously OTI Program Manager for ANE team in Washington DC January 16, 2006 Inauguration of Ellen Johnson Sirleaf as President of Liberia. Mandate of the NTGL dissolved. February 23, 2006 Third Strategic Planning Session held in Monrovia Decided to focus away from YES programming and back to small grant activities. STTA activities and strategic communications for EJS begins to unravel. Eleanor Bedford (moderator), Musu Clemens, Elizabeth Callender, JY, Mission Director, US Ambassador Social Impact, Inc. October 19, 2006 43 Date Transition Event Description People involved March 29, 2006 Charles Taylor Arrested trying to flee Asylum in Nigeria Taken to War Crimes Court in Sierra Leone March 30, 2006 Modification of Contract Increased ceiling from $16,937,897 to $17,868,343. March 30, 2006 Given $930,446 of Mission Money To Continue ALP activities until 9/30/06 Sharon Pauling April 15, 2006 John Langlois TDY to begin work on Strategic Communication activities John Langlois June 2006 Charney Research grant signed to begin Strategic Communication assistance to the Executive Mansion John Langlois June 2006 SWIFT holder told by SWIFT IQC that future activity in Liberia after September 2006 will be from the BRDG mechanism Jeanne Briggs, John Gattorn