FINAL EVALUATION OF THE USAID/OTI HAITI RECOVERY INITIATIVE October 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Eric Benschoter, Yves-Francois Pierre, and Amalia Prado on behalf of The QED Group, LLC, under Task Order #71 of the Program Development Quickly II (PDQ-II) IQC, contract number DOT-I-00-08- 00017-00. The author’s views expressed in this document do not necessarily reflect the views of USAID or the United States Government. …we should not lose sight along the way that all of these efforts, at the end of the day, should help the victims of disasters in the poor countries of the world, whose reserves are small and whose survival often depends on fast, effective, and equitable aid. Independent Review of the US Government Response to the Haiti Earthquake, Final Report (March, 2011) FINAL EVALUATION OF THE USAID/OTI HAITI RECOVERY INITIATIVE October 2013 Program Development Quickly II (PDQ-II) IQC, Task Order #71 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iv ACKNOWLEDGEMENTS The evaluation team would like to thank all of the OTI leadership and program management staff in Washington and Port au Prince for their assistance with providing the necessary documents, arranging meetings and interviews, and making themselves available when necessary to keep the team informed before and during a full schedule of site visits and interviews in country. The team acknowledges the tireless efforts and extra hours given freely to us in Haiti by all of the grantees, government officials, implementing partners, and other stakeholders involved in HRI, many of whom are mentioned in the interview lists in Annex B at the end of this report. Particularly, the efforts of Chemonics staff were invaluable in arranging for and transporting the team to the many site visits, interviews, and focus groups. We would also like to thank former Haitian officials for their time, advice and assistance, including former Prime Minister Jean-Max Bellerive, former Executive Director of the Interim Haiti Recovery Commission (IHRC) Gabriel Verret, former Secretary General Fritz Longchamp, and former Governor of the Haitian National Bank Fritz Jean. Finally, this report would not have been possible without the generosity and insights of the community members, CBO leaders, farmers, artists, students, entrepreneurs, and other Haitians who benefitted from HRI and shared their opinions and experience with us frankly and honestly. The evaluation team consisted of Evaluators from The QED Group, including Ms. Amalia Prado as Team Lead, Mr. Eric Benschoter, Mr. Yves-Francois Pierre and Focus Group Facilitator Mr. Jean Denis Lys. Cover photo credit: Marc Lee Steed. Cash-for-work activities funded through HRI in Port au Prince. The program slogan of “Ann Leve Kanpe!” seen on the yellow t-shirts translates from Creole as “Let’s Stand up Together!” v CONTENTS Acronyms ..................................................................................................................................................... 1 Executive Summary .................................................................................................................................... 3 Background .......................................................................................................................................................................................... 3 Evaluation purpose and methodology ........................................................................................................................................... 3 General observations ........................................................................................................................................................................ 4 Findings by program phase .............................................................................................................................................................. 5 Recommendations ............................................................................................................................................................................. 6 I. Introduction .............................................................................................................................................. 7 Haiti background: trends before and after the earthquake ..................................................................................................... 7 USAID and the Haiti Recovery Initiative ................................................................................................................................... 10 Evaluation objectives and methodology ..................................................................................................................................... 11 II. General observations on HRI I and II ................................................................................................. 13 On HRI’s fit to the OTI model and approach .......................................................................................................................... 13 On coordination ...............................................................................................................................................................................19 III. Findings: HRI I ...................................................................................................................................... 21 Community stabilization ................................................................................................................................................................. 21 Increasing citizen engagement ....................................................................................................................................................... 24 Enabling the GOH to function ...................................................................................................................................................... 26 IV. Findings: HRI II ..................................................................................................................................... 29 Relevance of mission statements .................................................................................................................................................29 Community engagement and the value of collective action .................................................................................................. 30 Increasing communication between communities and government ................................................................................... 34 Increasing awareness of access to economic opportunity .................................................................................................... 35 Seizing windows of opportunity: IDP resettlement strategy ................................................................................................37 V. M&E good practices, lessons learned, and recommendations .......................................................... 39 Activity level ......................................................................................................................................................................................39 Program level ....................................................................................................................................................................................39 Recommendations ........................................................................................................................................................................... 41 VI. Conclusions, program challenges, and recommendations ............................................................... 42 Program challenges and recommendations ............................................................................................................................... 43 Annexes ...................................................................................................................................................... 46 Annex A – OTI/Haiti program performance evaluation statement of work .................................................................... 47 Annex B – List of interviewees and focus groups ................................................................................................................... 53 Annex C – Interview questions and discussion guidelines .................................................................................................... 57 Annex D – Evaluation questions matrix ..................................................................................................................................... 66 Annex E - References ..................................................................................................................................................................... 83 1 ACRONYMS AIA American Institute of Architects ASEC Assemblée des Sections Communales CASEC Conseil d'Administration de la Section Communale CBO community-based organization CEP Permanent Electoral Council CHF CHF International CIAT Comité Interministeriel d’Ammenagement du Territoire CIP Caracol Industrial Park CONCERN Concern Worldwide CfW cash-for-work DAI Development Alternatives, Inc. DART Disaster Assistance Response Team DDANE Ministry of Agriculture Departmental Directorate DINEPA Direction Nationale de l'Eau Potable et de l'Assainissement DOS Department of State ENDK Enfomasyon Nou Dwe Konen FER Final Evaluation Reports GOH Government of Haiti HASOW Humanitarian Action in Situations Other than War HRI Haiti Recovery Initiative IDB Inter-American Development Bank IDP internally displaced persons IDEJEN Haitian Out-of-School Youth Livelihood Project IHRC Interim Haiti Recovery Committee INGO international non-governmental organization INL Bureau of International Narcotics and Law Enforcement Affairs IOM International Organization for Migration IP implementing partners IT information technology LAPOP Latin American Public Opinion Project LOP life of project M&E monitoring & evaluation MINUSTAH United Nations Stabilization Mission in Haiti MOCC Ministry of Culture and Communication NGO non-governmental organization OFDA Office of United States Foreign Disaster Assistance OIG Office of Inspector General OTI Office of Transition Initiatives PM Prime Minister PNH Haitian National Police PPR Program Performance Review SMS short message service STTA short-term technical assistance TA technical assistance TDY temporary duty UN United Nations 2 UNOPS United Nations Office for Project Services USG United States Government USAID United States Agency for International Development 3 EXECUTIVE SUMMARY BACKGROUND The Haiti Recovery Initiative (HRI) is the third OTI program in Haiti since 1994. Although it is not entirely unique in being served by OTI in multiple instances, it is atypical and says a great deal about Haiti’s need for improved socio-political and economic stability. This initiative was different from prior efforts, however, in that it was part of the post-earthquake USG response at a time when USAID/Haiti was going through a thorough revision of its country strategy and had few operational programs and staff in place. Both facets – responding to an emergency, not a political transition, and its fit within the mission – set it apart, highlighting OTI’s speed and flexibility, as well as the challenges posed by filling in for some of the upcoming longer-term programs. Up until the earthquake, Haiti appeared to be making strides in institutional capacity and governance after decades of dictatorships and instability, a floundering economy, extreme poverty, and inequality. USAID was on the verge of releasing a new five-year strategy for the country, produced in collaboration with national authorities, and new programs would be gearing up to support it. On the eve of its public presentation, the unthinkable happened, and the nation was thrown into near chaos. Nineteen of twenty ministries collapsed in the earthquake, and thousands of people were instantly displaced from their homes and communities. With few active programs to respond to the needs this event created, USAID turned to OFDA and OTI as their best tools for an immediate response. Senior program design personnel were on the ground within days, implementation contracts were made effective in less than a week, and the first grant was cut five days after. Since that time, the program was extended toward the end of its expected 18 month life-of-project and HRI II continued working under a new USAID strategy until now, as it winds down and expects to be fully closed by November 2013. EVALUATION PURPOSE AND METHODOLOGY This report is designed as a program performance evaluation covering the entire LOP from January 2010 to September 2013. The evaluation documents key accomplishments, outcomes and lessons learned, and should serve to share knowledge gained from the HRI experience with both the new long term development programs starting up at the Mission and other OTI programs. The evaluation also identifies key opportunities that were seized upon or missed, as well as any unexpected results, both positive and negative. Organizational learning is noted throughout the report and the main points are gathered at the end, along with accompanying recommendations. The evaluation team notes data limitations in the report, as well as methods employed to guard against bias. The dynamic nature of the program, with the ability to adjust its strategy to fulfill immediate needs and fill gaps, coupled with a large number of diverse activities meant that the team did not have a baseline or common indicators against which to measure impact. Thus, much of it is inferred on the basis of qualitative information collected in retrospect by the final evaluation team and on the program’s own M&E products. Over the course of two field trips – the first focused on HRI I and the second on HRI II – the team conducted 78 individual interviews with key informants, nine group interviews with roughly 80 participants, and 19 focus groups with over 150 participants. 4 GENERAL OBSERVATIONS …regarding fast, flexible, and purposeful programming. Overall, the team discerned that the program provided a remarkably fast response in an extremely challenging environment. Although the heavy demand and minute management from different levels of the USG on HRI as a primary response tool at times impacted the program’s ability to be purposive in setting a coherent and consistent strategic direction, it was still a highly valued mechanism for its dynamism and flexibility. There was widespread consensus that OTI’s identification of its three main program components of stabilizing communities, increasing citizen engagement, and enabling the government to function was on target. The first phase of the program can be qualified as effectively responding to emergency needs such as damage assessment, rubble removal, short-term employment opportunities, reestablishing communications capacities, and rapidly helping key government entities to get back on their feet, even without a GOH administrative structure in place. As the program shifted into a second phase in mid￾2011, there was a slowdown in grant activity as implementation shifted to one partner and the corridors strategy required greater coordination with other USAID offices. This occurred again in early 2012 with staffing changes and, later in the year, the design of the more intensive community engagement processes. …regarding windows of opportunity. Most sources confirmed that HRI I and II seized opportunities afforded by the evolving context and USG policy imperatives. For instance, it was able to contribute to the cholera response with highly regarded results and, although asked to intervene a month before the elections, rapidly implemented a series of televised presidential debates and a much-needed call center for voter information – after the earthquake, poll centers changed and citizens who had lost their IDs did not know how to proceed. In general, the HRI communications and outreach campaigns are considered highlights of the program’s strategic capacities. Activities include setting up a Press Center for the GOH, the “News You Can Use” radio broadcast for relief information, as well as the cholera response. In addition, the program carved out some breathing room after the disaster response phase and developed a more community-driven approach to programming, including support of the USG corridors strategy. This created opportunities to directly engage local beneficiaries in prioritization of activities, such as solar lamp distribution, which improved the impact of the various grants. It also led to improved ownership of activities, whereby in some areas community members actively protected project resources from theft or vandalism and formed informal committees to provide maintenance and care. The only missed opportunity brought up by several interviewees was the potential for greater engagement on technical assistance to the GOH, although some USG and GOH sources sustained there was not room for more. Other potential opportunities were working in other earthquake-affected regions, replication of the rental subsidy model for camp clearances, and a greater focus on gender from earlier in the project life. The first two, however, were subsequently covered by other donors, while the third was more of a cross-cutting approach the program later built in. What is striking is the extent to which external directives prevented the program from seizing these opportunities. …regarding coordination. Donor coordination at program startup was intense. There were daily meetings and phone calls that seemed quite taxing on time, albeit necessary as heard in many interviews. The various networks of coordination that OTI had to handle included Department of Defense, Department of State (DOS), Office of Haiti Special Coordinator, Department of Treasury’s Office of Technical Assistance, other 5 USAID operating units, United Nations, GOH, Inter-American Development Bank, World Bank, and other international aid agencies and domestic and international private sector. There was generally positive feedback from stakeholders in the USG and GOH, as well as at the Inter-American Development Bank – the only other donor HRI had coordinated with that the evaluation team was able to meet. In fact, only one interviewee had the impression that the program was not responsive to USG priorities and did not make time to meet as often as wished for. Likewise, on the GOH side, only one interviewee held a negative opinion of the program team’s performance given multiple implementation difficulties throughout a complex grant. FINDINGS BY PROGRAM PHASE With regards to community stabilization, the program’s prioritization of short-term employment opportunities and its assistance on rubble removal, as well as small infrastructure repair was effective. Organizational learning gradually refined its approach to beneficiary selection and community engagement – and both processes were also maximized by other mission programs. The evolution from a dispersed emergency response mode to more targeted geographic areas also followed good practices. On increasing citizen engagement, all evidence points to the media interventions’ success in getting information out to affected populations – including on cholera and elections, both on the government and non-government sides. The more medium- and long-term investments on building local media and research organizations’ capacities for a better indigenous emergency response show mixed results. HRI I effectively allowed the GOH to resume core functions through support for infrastructure – temporary and rehabilitated permanent buildings – and equipment. While the program was responsive to GOH and USG by fulfilling GOH technical assistance requests and fielded a significant number of experts to different offices, initial expectations at startup as to seizing an opportunity for reform were not upheld. In addition, the program allowed the USG to fill gaps as the earthquake caught most USAID offices without active programs and short on staff. Once HRI transitioned to “laying the foundation for longer-term development” in line with the new USG strategy, and as the regular programs were delayed, the team cast a wide net in terms of variety of activities, identifying good local partners, and putting forth an intensive and committed effort to help build collaborative frameworks in the target areas. At that point, the HRI team brought to bear its local knowledge and the OTI bottom-up approach in crafting regional mission statements that spoke to local perceptions, concerns, and priorities. Although fraught with difficulties, this approach produced results that were widely acknowledged as positive, contributing to the communities’ sense of well-being and inclusion. In line with one of the main principles undergirding both the “whole of government response” and the new USG country strategy, HRI sought to actively involve government in its activities. During the first phase, given the nature of its interventions and the context, government protagonism was achieved, despite occasional difficulties. During the second phase, results are mixed: where willing government partners were found, program effects on governance were evident in terms of increased access and communication. Where partners were less engaged, beneficiaries and members of the communities attribute the benefits of the activities to the donor and the CBOs. Government officials’ presence only at inaugurations did not seem to count as engagement in the communities’ view. For the Northern corridor, in a narrow sense, HRI II succeeded in laying the foundations for long-term development more evidently in those sectors where clearer guidance and more concrete plans for the follow-on programs were provided. In a broader sense, however, the program’s impact on communities in terms of “readiness for development” was amply proven throughout the final evaluation team’s field work. Solar lamps allowed women greater freedom of movement, children the opportunity to improve 6 their academic performance without having to migrate to better-serviced towns, and the creation of micro-enterprises that took advantage of the nighttime socializing. Rehabilitation of recreational spaces increased people’s pride in their town, influx of visitors, and a perception of greater regional integration. Rehabilitation of public offices motivated officials to work longer hours and provided their constituencies with a clear reference point of government presence. Vocational training provided young women with a highly valued opportunity to change their attitudes, behavior, and work opportunities. As staff described, the program brought about a sense of a “nouvelle ère” accompanied by a perception of inclusion in development, that is, tangible signs of incremental progress. RECOMMENDATIONS 1. Where programs fulfill a substantial gap-filling function, this should be explicitly acknowledged, while continuing to carve out “breathing space” to review openings for greater input on strategic direction and consistency. 2. Whether resulting from armed conflict or disaster, contexts where reconstruction work will be unavoidable merit consideration and discussion of alternative ways to implement, including infrastructure work in the initial contracts, using an engineering firm as a sub-contractor, or others. 3. Identification and in-depth discussion of opportunities for TA support to the host government should be part of the “strategy review breathing space.” Should TDY support be needed – as was the case for Haiti, given the limited number of OTI staff that could be in country – the areas of expertise should be made explicit. 4. With intensive community engagement processes, in keeping with the participatory dynamics set in motion, substantial changes and difficulties throughout implementation should be consulted or at least communicated with said communities. 5. Training on project design and management inevitably raises expectations as to future funding of the initiatives developed, regardless of the disclaimers. Planning ahead to build in a “very small grants” mechanism in the training grants can help manage these expectations and create incentives for participants. 6. Establishing a clear communications protocol facilitates relationships with grantees and ensures accountability. 7. The division of roles and responsibilities on M&E between program officers, the internal M&E, and external evaluation resources should be reviewed to balance the focus on the activity and cluster levels and to maximize the knowledge gained. 7 I. INTRODUCTION This final evaluation of USAID/OTI’s Haiti Recovery Initiative documents program results throughout its lifetime against the backdrop of the country situation immediately following the January 12, 2010 earthquake and the evolving USG country strategy for Haiti. Program outcomes are assessed to its objectives during the first phase and to its regional mission statements during the second. Its approaches, flexibility, and analysis feeding into programming decisions are assessed to the OTI business model. Organizational learning is noted throughout the report and the main points are gathered at the end, along with accompanying recommendations. HAITI BACKGROUND: TRENDS BEFORE AND AFTER THE EARTHQUAKE The donor community and analysts widely agree that Haiti had made strides on institution building and governance in the years preceding the earthquake – references to the inaugural docking of the largest cruise ship at the Labadie Port in December, 2009 abound as a sign that the country had turned a corner. The disaster was followed by a breakdown of the moral order, along with increased citizen demands and an ensuing crisis of local governance. In effect, the challenges of widespread violence, job creation, the restoration of hope and social order were on local authorities’ agendas for at least three months after the earthquake before “refondation” (”building back better”) took over as the ultimate goal for the Plan for National Recovery and Development. With regards to institution building before the earthquake, the focus is on Haiti’s tradition of centralization and the development of the 1987 Constitution that aims at a new democratic order. The post-earthquake context presents information on violence, corruption, and victimization by violence before presenting a brief review of citizens’ demands for services and satisfaction with them after the earthquake – all of which points the program addressed. A tradition of centralization and authoritarianism. In Haiti, centralization has meant that the great majority is excluded from access to goods and services, which have historically been concentrated in urban areas. Following independence from France in 1804, the newly created Haitian state redistributed arable land according to seniority in the indigenous army and ethnicity. The state, comprised of dominant Creole groups, felt no obligation to deliver services to the newly freed peasants, maintaining the practices of exclusion that the colonial administration used to subjugate slaves and freemen. Privileges and state services were, and still remain, concentrated in the main cities, the loci for “civilized” people. Residing largely outside the highly symbolic gates of these cities, the peasantry could not exercise any civic rights by participating in state affairs to gain access to goods and services available to urbanites (Théodat, 2003). Despite sporadic attempts at devolving power to local levels, the communes, Haiti has been marked by a tradition of centralized management and administration through the discretionary appointment of civil, military, and paramilitary authorities whose main role was to funnel local resources to the capital city (Jadotte and Pierre, 2008). The persistent failure to respond to the needs of the large majority of rural dwellers coupled with their exclusion from participation in any political decision-making ended up eroding their confidence in the state apparatus at all levels of government. The Constitution of 1987: towards the new democratic order and decentralization. After the fall of the Duvalier dynasty in 1986, a widely representative constituent assembly, including various sectors of civil society, union leaders, and political parties, developed the Constitution of 1987, which in principle marked a turning point towards decentralization, local governance, and the institutionalization 8 of a new democratic order. Not only the commune, but also the communal section and the department were to become autonomous instances of power (Deshommes, 2004). The Constitution emphasized universal suffrage and citizen participation at all levels of government, creating multiple assemblies and councils to facilitate such participation.1 In addition, it emphasizes the right of local peoples to work towards the development of their localities. The new standards set by the Constitution, however, are far from being applied. The departmental assemblies were never implemented, nor was the Interdepartmental Council. The CEP (the Permanent Electoral Council) is still an interim entity. Post-1987 presidential decrees did not respect the new constitutional safeguards and even ran counter to decentralization. For instance, the Ministry of Finance received financial control over local government units following a March 13, 1987 decree, although article 217 of the constitution established the ”decentralized funding” of local government (Smucker et al., 2000). Haiti is far from having a ‘Decentralized Unitary State’ declared Deshommes seventeen years after the Constitution and the ensuing enthusiasm passed. Political crisis and the rise of organized violence since 1987: towards the breakdown of the moral order post-earthquake. From 1987 up to January 2010, Haiti has gone through many political crises including military coups, fraudulent elections, and political violence, with no credible security agencies and low levels of confidence in public institutions (LAPOP, 2006). As a result, criminality and gang related activities were on the rise. Under Aristide, distribution of weapons to youth groups in exchange for political support contributed to this increase: “With his departure, these gangs (which at that point were fully involved in criminal activity) quickly established control” of neighborhoods in a rapid escalation of violence, crime, and kidnappings (Kolbe, 2013). Data from Humanitarian Action in Situations Other than War (HASOW) shows that starting 2004 until 2006-07, the percentage of youth who claimed having affiliation with armed urban groups was on the rise. It started declining sharply before 2010 and re-escalated timidly after (see Figure 1, adapted from HASOW). Concerted efforts against gang violence and kidnappings before the earthquake between MINUSTAH and the Haitian National Police (PNH) explain such a decline. Gangs were dispersed and order restored around 2007 (Marcelin, 2011). However, the general quality of moral life was deteriorating due to a combination of factors (Shamsie, 2011), among them are:  the prison breakout of more than 5,000 detainees, which brought back insecurity and fear of revenge;  the poor living conditions in the camps set up to house the IDPs leading to an increase in sexual violence against women and girls in and outside the camps; 1 For the Communal Section, for instance, there is the CASEC (the Administrative Council of the Communal Section) and the ASEC (the Communal Section Assembly); for the Commune, the CM (the Municipal Council) and the AM (Municipal Assembly); for the Department, the CD (the Departmental Council) and the AD (the Departmental Assembly). 9 Figure 1. Self-reported affiliation with armed groups by year 0 0.5 1 1.5 2 2.5 3 3.5 4 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Percentage of males (18-29) claiming affiliation with armed urban groups Bel Air Martissant Delmas 2 Gran Ravine Cité Soleil  the loss of capital by the informal sector which put thousands of self-employed people out of a “business” making them more vulnerable to all kinds of abuse. Corruption and victimization by corruption. Perceptions of corruption and reports of victimization by corruption can be used as additional indicators of Haiti’s breakdown of the moral order post-earthquake. Indeed, the LAPOP study of 2012 indicates that, as compared to 25 other countries of the Americas, the perception of corruption (which is relatively low in Haiti) did not correspond necessarily to the victimization of corruption, which is relatively very high (LAPOP, 2012). Corruption has been part of daily life to the point that it is perceived as being low whereas in reality it is high. After the earthquake, however, in Haiti the percentage of people victimized by corruption increased significantly by 13 percentage point from 2010 to 2012 (54 to 67 per cent). Post-earthquake local governance: citizen demands for and satisfaction with services and participation in municipal meetings. According to the Latin American Public Opinion Project, whereas the percentage of people demanding services from governmental instances was generally low as compared to other countries of the Caribbean and Latin America (second lowest in the regional ranking), it doubled after the earthquake, going from 10 per cent in 2006 to 21 per cent in 2012. The percentage of people who said they were satisfied with the quality of municipal services, however, only increased by 5 percentage points over the same period (from 33 to 38 per cent). Not surprisingly, the earthquake had triggered popular demands for help from municipal governments, which they could not satisfy. Citizen participation in meetings, however, increased. Whereas in 2006 citizen participation in meetings organized by the municipality over the preceding 12 months was among the highest fifth (13 per cent) in the Americas, it jumped to the first position in 2012 with 21 per cent, which roughly represent a two third increase. As an illustration, the former mayor of Carrefour declared that to escape the pressure coming from multiple demands after the earthquake, he would make a point of checking up on cash-for-work sites up in the hillside to avoid further demands and complaints in his office. 10 USAID AND THE HAITI RECOVERY INITIATIVE Throughout 2009 the USG strategy for Haiti underwent a thorough review after multiple social and political disturbances, coupled with declining social and economic indicators, called into question aid effectiveness. For USAID in the field, this meant programs phased out and USG personnel left, leaving behind a clean slate for the new strategy. In practice, this meant that when the earthquake hit there was a serious shortage of human resources and very few programs to accommodate shifting priorities to address needs beyond immediate relief. In this context, USAID/OTI began its program Haiti Recovery Initiative (HRI) less than a week after the earthquake as part of the “whole of government” post-earthquake response, supporting activities in Port au Prince in three components: stabilizing communities, increasing citizen engagement, and enabling the government to function. Most of the programming initially focused on short-term employment, support for emergency relief communications, and enabling the Government of Haiti to resume core functions. As the context evolved, the program also addressed hurricane preparedness, cholera prevention, and elections-related communications activities. In early 2011 the new five-year "Post-Earthquake USG Haiti Strategy" laid out two objectives: to catalyze economic growth through investments in agriculture, energy and infrastructure and to ensure long-term stability through investments in public institution. These were to be achieved through investment in three corridors (Port au Prince, Cap Haitien, and Saint-Marc) on four core development pillars: energy and infrastructure, food and economic security, health and other basic services, and governance and rule of law. Working within this strategy, in March 2011 HRI expanded its geographic focus to include Saint-Marc and Cap Haitien with the aim of laying the foundations for longer-term programs supporting decentralization of growth and population. As the program moved into its second phase, in August, 2011, the focus shifted significantly to the Northern corridor, around the Caracol Industrial Park. Activities implemented in the development corridors initially included temporary employment and livelihoods, a few agricultural projects, skills training for CIP factory workers, and access to safe drinking water. For the first phase, lasting until July 2011, there were two implementing partners, DAI and Chemonics, both of which were selected precisely because they already had operational teams in the field. For the second phase, the contract was bid again for only one implementing partner and Chemonics won. The transition between HRI I and II lasted a few months: both implementing partners closed out and one started up. In this context, the “Independent Review of the US Government Response to the Haiti Earthquake” commissioned by USAID cites as the most important outcome that “predictions of ongoing deaths, infectious disease outbreaks, mass migration, and political insurrection did not occur within the first six months.” The report also lays out the following as some of the main challenges for effectiveness and efficiency of the “whole of government” response in that time frame: policymakers involved in tactical decision-making in the field, overwhelming frequency of information requests, new managerial structures which resulted in multiple coordination instances and often overlapping roles and responsibilities, and a shortage of human resources available to the USG in Haiti. 11 EVALUATION OBJECTIVES AND METHODOLOGY Final evaluations in transition contexts, in this case, a post-disaster situation, face multiple constraints some of which include: shifts in objectives, widely diverse portfolios, often incomplete paper trails for decision-making, lack of baselines, weak grantee reporting capacities, and tacit theories of change and assumptions. The present endeavor is not exempt from limitations, but attempts to overcome them through triangulation of different qualitative sources of information: an in-depth desk review of hundreds of program documents and the database, interviews (individual and group) with stakeholders, focus group discussions, interviews on the street in the target communities, and site visits. The total tally is: 78 individual interviews, 82 in 9 group interviews, and 150 participants in 19 focus groups (see Annex B for the full list). Data limitations. Given that there was no explicit theory of change, a significant proportion of the indicators pertaining to key objectives of the program and used during the research were rather intuitive. There was no experimental or quasi-experimental design in the process, no comparison between intervention and non-intervention areas. Rather, given the diversity of a large portfolio of discrete activities, there could be no common indicators, except as surfaced by the qualitative research, thus the lack of baselines. Although a few studies called “baselines” were completed throughout the course of the program, these focused on a small selection of issues to inform programming and they were not structured in tandem with a final evaluation plan. Rather the program was evaluated against its objectives in the first phase and its regional mission statements in the second. The data also have limitations inherent to retrospective accounts and, possibly, to bias in the selection of interviewees. Other than triangulation of sources, the former is inevitable. To counter the latter, the final evaluation team selected the activities for site visits and, where possible, randomized selection of focus group participants. The team considers that overall there was enough divergence of views on enough topics to ensure there were no systematic biases in the information collected. Team Composition and Phase of the Evaluation. The evaluation was carried out by a research team of three people in the first phase and four in the second phase. For the first phase (HRI I), the team covers the analysis of existing information and primary information collection in key informant interviews with former and current program leadership and staff, key Mission staff, GOH employees and advisors, other donors or NGOs, operating at the time, and, where possible, with beneficiaries. The first phase was carried out in May with field visits and interviews in Port-au-Prince, Saint-Marc, and the North. For the second phase (HRI II), the research team used the program database to sample ongoing and recently completed activities for in depth review by selecting grants that work well and those that did not work as well as expected. The sample was consulted with in-country staff and leadership. The team targeted specific geographical areas in order to glean cumulative effects beyond the activity level. The second phase was carried out from July to August with field visits and interviews in Port au Prince and the northern communes of Cap Haitien, Limonade, Caracol, Trou du Nord, Terrier Rouge, Fort￾Liberté, and Ouanaminthe. Evaluation questions. The Scope of Work (see Annex A) required that the independent program performance evaluation document accomplishments, missed opportunities, outcomes, successes, and lessons learned. More specifically, the evaluation questions are the following:  Did OTI programming through HRI reflect a fast, flexible and purposeful response to the complex dynamics of the transition in target areas of Haiti and seize key windows of 12 opportunity to achieve the program goal and objectives? Were there issues that should have been addressed but weren’t?  How and how well were HRI’s activities coordinated at both the strategic and objective levels with USAID, Department of State, Government of Haiti, and other donor programs in Haiti? What factors helped or hindered coordination?  To what extent did HRI effectively lay the groundwork for other, longer-term investments by USAID, Government of Haiti and other donors?  Were there any unexpected effects of the program’s activities or implementation methodology, either positive or negative, on the economic and political situation in target areas of Haiti?  What has the program accomplished at the program goal and objective levels?  Did the results of HRI activities differ between regional focus areas? If so, what were the distinguishing factors between regions or between the approaches that HRI took?  What were the program outcomes and effects (intended and unintended) on the various beneficiary groups? Beneficiary groups should include: grantees, public officials and citizens.  What activities undertaken by the program were most and least valued by local partners (community, associations, government, etc.)? Why?  What lessons can be derived from the HRI experience to inform USAID/Haiti on strategic approach and methodology after OTI departs Haiti? To what extent are these lessons contingent upon OTI-specific operations and financial mechanisms, and are there aspects of the strategic approach that cannot be replicated for this reason? Data collection and analysis procedures. For individual interviews, the team collected data from stakeholders in key informant interviews as well as some street interviews with beneficiaries during site visits of certain community projects. Most interviews were carried out by two members of the team simultaneously. Scheduling conflicts sometimes led them to divide. Group interviews were conducted to save time and allow for feedback from more individuals. In these, the questions were not structured to provoke a discussion, nor was there facilitation involved. The focus group discussions used a more elaborate method. Generally, the size of the group varied from eight to twelve participants in order to provide ample opportunity for each person to talk – although there were a few where the team had difficulties in keeping the number of participants manageable. Mostly, participants were selected so as to reflect a wide range of opinions. While one member of the team acted as facilitator, the others would ask questions. Open-ended questions formed the discussion guide and varied somewhat by the type of group (see Annex C). Most information gathered was then organized into a matrix (see Annex D) and analyzed for patterns of commonalities and differences. Focus group responses were translated into French, summarized to some degree and provided to the team by the Focus Group Facilitator. 13 II. GENERAL OBSERVATIONS ON HRI I AND II ON HRI’S FIT TO THE OTI MODEL AND APPROACH This section addresses the answers to the over-arching evaluation question that speaks to OTI’s model and approach: “Did HRI programming reflect a fast, flexible and purposeful response to the complex dynamics of the transition in target areas of Haiti and seize key windows of opportunity to achieve the program goal and objectives?” The following sections contain a more specific discussion of the different program phases and illustrative activities. Overall, the analysis carried out by the evaluation team revealed that the program provided a remarkably fast response in an extremely challenging environment, proving its flexibility in adapting to changes in said context and to requests stemming from USG policy imperatives. The weight and operationalization of the latter, however, at times impacted the program’s ability to be purposive in setting a coherent and consistent strategic direction. There was widespread consensus that OTI’s identification of its entry points – the three components – was on target in the post-earthquake scenario. The program’s implementation speed and effectiveness stood out amongst the emergency response efforts. As the program shifted to more transitional programming, laying the foundation for longer-term development, it was asked to spearhead a wide variety of initiatives in the development corridors long before concrete plans for the regular programs were in place. This proved to be a serious constraint for ensuring follow-on for specific activities. In addition, the widespread perception that HRI would fill the gap between disaster relief and development work did not account for the complexities inherent in any such endeavor, much less for a program running on a small grants model. The evaluation team also found that, in drawing up the extension plan, when the HRI team was better able to carve out space for an approach that was more purposive and responsive to the local communities’ priorities, the results were all around evaluated much more positively in terms of contributing to the communities’ sense of well-being and inclusion. The impact on communities’ perceptions of the government, national and/or local, was restricted to those places where officials were actively involved. Fast and timely. Based on a previous OTI Haiti program, which found a clear pattern of negative association between short-term employment activities and the number of incidents reported by the UN, the start-up team prioritized this component as a means to prevent instability. In collaboration with a USAID engineer, the team also conducted an assessment of the damage to GOH buildings and found 19 of 20 ministries had completely collapsed. Another team member rapidly designed, in collaboration with the Minister of Culture and Communication, an emergency media relations venue next to the temporary GOH headquarters at a police station. Prior to that, a prolonged government silence had given way to rumors, increasing the population’s anxiety as no information on relief efforts and no reassurance from the leading authorities was had. HRI cleared its first grant ten days after the earthquake and, by the end of the first month had approved 27 additional activities for a total of $US 2.2 million. As the charts in Figure 2 illustrate, this rate was increased through the third quarter of 2010. Through July, 2013 the program cleared 998 grants for a 14 little over $US 90 million. Five per cent of the total number of grants cleared was later cancelled due to difficulties with grantees or in the operational context. Figure 2. Grants cleared and funding committed by quarter over the life of HRI The graphs also show that the transition from the first to the second phase (mid-2011) markedly slowed the program down, as it went from two implementing partners to one – something the OTI field team had warned was likely to happen. Once past the transition, there is an uptick in funding committed in large part due to four big-ticket grants cleared in November (two on resettlement, one on GOH work space, and one on media). In early 2012, the program again changed pace as it experienced staffing changes and then crafted more deliberately its regional approaches. This latter work then allowed it to resume speed as it entered its last year – with the late 2012 spike in funding committed more equally distributed among grants than the previous spike in late 2011. It must be noted that despite the OTI model’s prioritizing rapid start-ups for all its country programs, Haiti by far outstrips the averages both in terms of numbers of grants and funding. Flexible. The numerous types of grants and partners, the shifts in target areas, and the quick response to changes in the context – such as the cholera epidemic and a late request to implement elections￾related grants – illustrate, at a very basic level, HRI’s programming flexibility. As one interviewee stated, “We struggled to meaningfully address a wide range of activities: humanitarian, political transition, and long-term development.” The unforeseen opening of regional offices, the reconfiguration of technical teams, the team’s diligence in providing solutions to the complexities that arose – for example, how to manage payroll for the short-term employment grants, and the implementation of a large number of infrastructure grants – attest to its operational flexibility. As Mission staff noted, HRI’s role in the latter was critical as there was no infrastructure provision in the regular development programs. As the USG corridors strategy rolled out, the program’s gap-filling function grew, increasingly so as the longer-term programs were delayed. HRI was effectively flexible in responding to the evolving context and to top-down directives. The downside, many agreed, was that HRI became the “go-to” program, thereby decreasing its own opportunities to carve out a more consistent direction within the larger USG strategy. Purposeful. As noted above, the program’s entry points were relevant in addressing the more pressing needs prioritized by the GOH, the international community, and local population. Its ability to move quickly allowed it to start, before the end of January, on activity implementation of short-term employment in rubble removal, structural assessments of key GOH buildings, set up temporary work space for GOH officials – including a Press Center – and supported a wide-reaching radio station 15 network with emergency relief-related programming. Subsequently, HRI evolved to include other types of activities in response to community priorities, as expressed by partner community based organizations and program-commissioned focus groups. These include livelihoods and vocational training, small infrastructure repair, and improved service delivery, among others. It also implemented cultural and athletic activities aimed at promoting a sense of return to normalcy; diffusing critical messages on cholera prevention and, later, on elections. In addition, from very early on, the DOS requested that the Ambassador and Mission Director clear off on grants. As the approval system consolidated, according to some, the team stopped presenting proposals unlikely to get cleared. Part of the answer to the question as to the extent to which an OTI program is purposeful has to do with the development of innovative solutions to problems arising in the evolving context. The HRI team designed and implemented, along with its partners, several inventive initiatives. Two of them were widely referenced by all manner of stakeholders as groundbreaking and hugely effective: the “News You Can Use” emergency relief radio show and the use of a rental subsidy to support resettlement of two camps. In addition, when it came to outreach and information campaigns, the team’s reiterated emphasis on understanding local communities’ perceptions – rather than making assumptions – allowed the program to model more strategic messaging for both the Mission and the GOH. Formally, the program’s paper trail as to the extent of its purposefulness is remarkably complete throughout its first phase. Other than the initial program plan, there are nine updates through March 2011. Few OTI programs have this number of updates to the program rationale and plan documenting the nuances and shifts in direction. For example, in September, 2010 there is a recommendation to shift to doing more small infrastructure repair and rehabilitation as visible signs of recovery, aside from rubble removal. While the program had started doing canal clearing and watershed management – involving reforestation – as early as April, there was a more marked effort along these line following the recommendation as tracked against the grants cleared subsequently. After the new USG strategy for Haiti became official, in January 2011 and as HRI moved into its second phase, the key constraint to designing and implementing a purposeful vision was the conflicting guidance on the priorities for program resources in support of the larger USG strategy. The multiplicity of views, the funding required for each, and the time and operational constraints threatened to overwhelm the program. Based on an assessment of its operational capacity, lessons learned, and unique strengths, the team proposed focusing on targeted and catalytic initiatives touching on all four pillars of the USG strategy in key neighborhoods of Port au Prince and in Saint-Marc and the Northern corridors. Based on an initial agreement as to the general outline of these initiatives, HRI moved forward. As the program neared its end in 2012 and the longer-term programs were not yet online, it was again extended. The agreement arrived at on priorities for the extension maintained the three components and multi-sectoral work, with a greater focus on a key ingredient of the OTI model: community-driven programming, aimed at ensuring greater Haitian ownership and, ultimately, impact. These agreements are the only documents outlining the strategic vision for the second phase and its extension until late 2012, when the regional offices in Port au Prince and Cap Haitien drew up each of their mission statements. These succinctly present the rationale guiding the different sectoral initiatives and how they come together, prioritizing the community engagement processes based on internal analyses of the different contexts and desired changes. A major obstacle identified in both Port au Prince and the North was the absence of a sense of ownership by the community of its own development – a prevailing perception that it is somebody else’s responsibility. That the program objectives/components did not change through substantial shifts from disaster response to development, target areas, sectors, types of activities, and approaches cannot but draw 16 attention. As noted earlier, the tactical evolution was well documented during the first phase; the transition to the second was less so and the program’s gap-filling function seems to have prevailed. The extension plan attempts to lay out a more targeted vision that builds on the program’s unique strengths, but in practice it is not until the last stretch emerging from the mission statements that a more strategic, bottom-up process takes hold. Interviewees agreed that aside from the operational context, the main challenges faced in crafting and implementing a coherent strategic direction were the pressure for speedy implementation and the multiplicity of external directives. As for the former, the program team’s closeness to the field as activities are implemented gives OTI programs an edge, but in this case, the sheer volume of activities made actually accompanying implementation, seeing how activities played out, and interacting with the community as part of the process very difficult. As one team member stated, “Other-directed grants definitely took more time and effort to implement.” One example of this were the two complementary grants to the American Institute of Architects (AIA) and the Comité Interministeriel d’Ammenagement du Territoire (CIAT) to support the preparation of a master plan for the North. The grants were cleared in November, 2011 and initially the end date for the master plan was March, 2012, but AIA delays and coordination difficulties between the grantees delayed this until December of that year. AIA was perceived as not being sensitive to the host government’s priorities, CIAT was unsatisfied with the managerial work by the HRI implementing partner, and from the program team’s perspectives, both organizations were very difficult to work with. In the end, CIAT used part of the data gathered by the AIA process and produced its own plan. Meanwhile, HRI was also managing another 142 activities for a total value of approximately $18.3 million, of which 59 grants were managed out of its Cap Haitien office. In hindsight, several team members involved in different time frames wondered whether they could have done more to hold off some of the external pressures, while others – including staff from other programs – are adamant that there was no room for “pushing back.” The 2011 Program Performance Review recommended the program move beyond the wide net of experimental programming to a more strategic focus, probably further exacerbating the field team’s second-guessing as to what was perceived as a high level of other-directedness. The number of concerns expressed in interviews and internal documents as to whether the program was strategic or not is, by itself, a noteworthy point. Normally for OTI, the gap-filling function applies to discrete activities or, at a stretch, a cluster of grants, not the main thrust of a country program. The conditions in Haiti for USG ability to rapidly roll out its longer-term programs changed this, suggesting OTI develop guidance for field teams in similar situations. As one senior OTI advisor stated, “An interesting question for me is: is that okay? Is it a role we necessarily shrink from – being the guys our foreign policy dictates without necessarily working within a political strategy?” Additionally, one field team member noted that after the emergency response phase, the more transitional programming would have required strategy-related discussions that were less harried and recommended “intentionally creating breathing space” to review the strategic direction. A complementary observation was that the intensity of oversight from above and the level of public scrutiny expected also tended to make the field team leaders too attentive to the minutiae: “The whole team was involved in the nitty-gritty details. We could not take a step back to reflect on the evolving context.” As the “Independent Review…” cited earlier noted, variations of these problems were common across agencies. 17 A final note on the program rationale and its fit to the evolving context pertains to the use of a “stabilization” conceptual framework throughout its lifetime. Information gathered throughout this evaluation pointed to several reasons the team continued using this analytical framework – including the terms “drivers of instability,” “destabilizing forces,” and the like – even past the immediate post￾earthquake period, when outside observers instead noted that the country had remained remarkably stable, despite widespread fears to the contrary. As described by start-up team members, the notion of “stabilizing communities” in the initial period was more akin to helping communities recover while preventing violence. In subsequent periods, other reasons cited were the funding streams and a perception from the field that “Haiti is never fully unstable or stable.” Yet another reason was that if the program was not about stabilizing communities, what would make it OTI-like? Nonetheless, the final evaluation team found that, while the stabilization conceptual framework does not seem very appropriate, particularly as the program transitioned from emergency response to its second phase, the analysis did help the team hone its bottom-up approach and identify key interventions. The field team’s own misgivings as to its applicability suggest a thorough review of terms was due as HRI I shifted to the second phase. Windows of opportunity. The evaluation team found that, overall the program did seize key windows of opportunity in the evolving context. In the first phase, it rapidly built in cholera prevention programming as the outbreak spread and, with little advance notice, implemented elections-related information activities. In addition, the ongoing emergency-relief radio show also prioritized prevention messages and the strategic communications training provided to GOH ministries allowed the government to respond efficiently, marking a notable difference with prior experiences, as acknowledged by the former Minister of Culture and Communication. With elections, although staff had developed numerous ideas for activities, the program was asked to hold off and it was not until under a month before elections that there was a green light. In that limited time, HRI managed to organize the first-ever presidential debates on television, screen them in the camps, set up a call center for people to find out where they would vote and how, even when many had lost their identity documents and polling centers had changed. To a large extent, the larger USG strategy was part of the evolving context and, when the pressure for speedy implementation lessened somewhat, HRI was able to seize a window of opportunity and craft an interesting and effective approach to community-driven programming while addressing the sectoral work prioritized by the other USG partners. For example, from beneficiaries’ point of view, one of the most highly valued activities was the installation of solar lamps, an activity that started during HRI I in Cap Haitien and was widely replicated throughout HRI II in Saint-Marc, Port au Prince, and the North using INL funds. Aside from improving people’s perception of safety, the lamps allowed school children to study until later, improving their academic performance, small merchants to set up businesses under them, and, more generally, they brought some of these communities a respite from the isolation that set in with the dark. In Bel Air, for instance, youth organizations organized women’s night-time soccer tournaments on the streets where lamps had been set up. The team’s emphasis on the bottom-up approach yielded numerous signs the community really engaged and took ownership of the processes set in motion. Focus group participants confirmed their involvement in prioritization of activities. In some cases, their minute oversight allowed local authorities to address issues with the program and solve problems with the construction firms. Across geographic target areas, multiple interlocutors emphasized the importance of the activities in helping change the community’s own self-perception and the perception by others: “We don’t want Martissant to be a red zone anymore,” “Artists coalesce the community of Bel Air; we want to recover the Bel Air of old, birthplace of artists, writers, and figures of state,” and “Before, people from neighboring communes would look down on Terrier Rouge, saying we didn’t even have a municipal office. Now we have the 18 nicest one.” When local or national authorities could not be reached by program staff to sign grant agreements, the community representatives would hunt them down. Overall, beneficiaries of training activities showed a remarkable willingness to share their acquired knowledge and skills with others – to the point where some organizations resisting change broke up. The examples are numerous, all attesting to HRI’s commitment to overcoming the culture of dependency the team itself identified as one of the major obstacles to development. By allowing, and sometimes demanding, communities take ownership in very complex and challenging environments, HRI effectively lay the groundwork for development in a way that would not have been possible without the bottom-up approach. A few missed opportunities were also noted. The most widely discussed was the opportunity for greater engagement on technical assistance in support of the GOH. The program initially envisioned its support for the GOH to resume its core functions as two-fold: setting up functional, temporary work space and providing technical assistance for key ministries with a view towards governance reforms. The thinking was that other countries stricken by disaster had re-built their public sector more strategically – the slate having been wiped clean, they purposefully avoided going back to “business as usual.” Some considered this expectation was unrealistic given the magnitude of the devastation. A more intermediate position held that the window of opportunity for reform closed around mid-2010, as routines were re￾established and elections became imminent. From the perspective of the OTI field team, while greater engagement on technical assistance would have been ideal, without more OTI staff in the field, it was not possible. While the startup team included a governance expert who led initial negotiations with the GOH, the strategic thinking on this point was later more externally directed. The program was asked to focus on supporting the Interim Haiti Recovery Commission (IHRC) and, as two rounds of elections approached, technical assistance was put on hold. A September program rationale paper recommends the team bring to bear greater conceptual acuity to the design of the technical assistance for the government and reconsider the duration of the contracts (maximum six months), which was not well￾suited to the needs. The former Prime Minister Jean Max Bellerive offered yet another perspective: a large percentage of the more highly qualified public servants died in the earthquake as the ministries collapsed, leaving the administration with scant absorptive capacity for both technical assistance and other types of support. On top of that, in the initial stretch other donors found themselves with their hands tied with regards to provision of other types of support and vied with each other to second their in-house expertise. A USAID advisor present during that time frame also noted there were serious difficulties in general for procuring technical assistance willing to travel to Haiti. In many ways, the highly positive comments the evaluation team heard from former GOH representatives about their interactions with the start-up team sounded very much like the timely provision of technical assistance. HRI was also receptive to GOH requests for technical assistance for the IHRC and supported other USAID offices’ requests – with the Tax Office, for example. Should OTI wish to play a key role in the provision of technical expertise, prior experience by the field team or committed TDY support from headquarters would seem to be necessary. Other missed opportunities garnered much less discussion and were referenced less widely. These include working in other badly hit areas of Port au Prince like Leogane and Petit Goave – the argument was they did not fall within the planned corridors strategy and other donors would be working there, but many took a very long time to deploy. Other missed opportunities were incorporating gender as a cross-cutting factor into activities earlier, before gender-based violence became so prevalent in the post￾earthquake scenario and continuing to work on resettlement. For the latter, what one USAID advisor cited as a “huge missed opportunity” was when HRI was not allowed to proceed with a Saint-Marc housing project for women heads of households that had been displaced by the earthquake. Given the importance of resettlement, replication of its rental subsidy model with other camps would have helped the population affected and, quite possibly allowed the program to explore other alternatives in a more 19 timely fashion, rather than at the end, when it was asked to resume. What is striking is the extent to which external directives prevented the program from seizing these opportunities. ON COORDINATION There was generally positive feedback from stakeholders in the USG and GOH, as well as at the Inter￾American Development Bank – the only other donor HRI had coordinated with that the evaluation team was able to meet. In fact, only one of eighteen USG interviewees had the impression that the program was not responsive to USG priorities and did not make time to meet as often as wished for. Likewise, on the GOH side, only one interviewee held a negative opinion of the program team’s performance given multiple implementation difficulties throughout a complex grant. At start up, a team member coordinated with the Department of Defense to stand down “Command Solo,” a military initiative aimed at substituting local radio stations in broadcasting emergency relief information and initiated, instead, local production of the same information, thus strengthening the media network for the longer run. Also at start up, in the temporary absence of the UN coordination platform as it struggled to recover from its own losses, the team set up a geo-referenced database to track USG partners’ work and outputs on rubble removal. Information requests from the USG stateside required an intense meeting agenda and the team had a dedicated member for that purpose. As the emergency response became more established, the frequency of meetings decreased only slightly. The unusually high level of public scrutiny of relief efforts made donors and NGOs in general very reactive to media coverage – as noted by several books and articles about the time. The evaluation team could not ascertain whether there was any particular event that gave rise to the DOS request that the Ambassador and Mission Director clear off on all grants. As the 2011 Management Review found, this did not cause undue delays in approvals. From the former ambassador’s perspective, this system worked very well and kept him well informed of progress. From further away, at DOS, there were still some misgivings about particular activities (the public squares in the Northern communes, for example) and others were turned down. The adjectives used by USAID sectoral teams in describing collaboration with OTI are overwhelmingly complimentary. The team is described as nimble, un-bureaucratic, creative, and brings to bear valuable local knowledge and connections: “They opened doors for all of us.” “Performance was timely and effective,” and, as one interviewee put it, “the only thing we can show, everyone else is backlogged three years.” On the USG side, one of the factors that helped with coordination and which enhanced HRI’s visibility was its diligence on reporting and responding to information requests: “The program stood out because it had its information handy, they always knew where they were implementing and what the results were.” Another factor was that the program team’s local knowledge allowed them to collaborate with other offices in developing plans for HRI use of their funding. INL provided funds for road safety campaigns and installation of solar lamps. The Department of Treasury funded a technical assistance program for the Direction Générale des Impôts. Other offices, most notably Health, had very concrete plan for their upcoming program and HRI was able to accommodate as directed. In addition, the HRI team was very proactive in setting up coordination opportunities for follow-on by the new programs: it distributed lists of contacts, including local community-based organizations, 20 activities that could be built on, lessons learned, and helped set up a partners’ meeting to discuss all of these as it wound down. Finally, with regards to HRI I there were several references to a remarkable level of internal coordination among the two implementing partners and OTI, highlighting everyone’s commitment to the job – as reflected in very long hours for all offices. Despite the frenzied pace and opportunistic division of target areas and grants – Chemonics did infrastructure and DAI, IT and wiring given their in-house expertise – the relationship was collaborative and amicable, with no partiality shown by OTI. The following sections provide a more detailed analysis of the evaluation team’s findings by program phase. 21 III. FINDINGS: HRI I Recapitulating, HRI I started work on three components shortly after the earthquake aimed at stabilizing affected communities, increasing citizen engagement, and enabling the GOH to function. The choice of entry points was widely deemed relevant and their implementation timely and effective. The distribution of program funds between components was as follows: community stabilization with 487 grants for a total of $33.95 million; increasing citizen engagement with 61 activities for $6.11 million; and enabling the GOH to function with 123 activities for $10.91 million. That is, community stabilization had 67 per cent of the funding, citizen engagement 12, and GOH support 21 per cent. A systematic review of outcomes and shortcomings by component follows. COMMUNITY STABILIZATION This objective initially targeted short-term employment for restive communities of Port au Prince with the understanding that the socioeconomic disruption caused by the earthquake threatened a historically fragile stability. As the months passed, post-earthquake general discontent with the government, the perceived slow pace of reconstruction, the congested camps, and an imminent reconstitution of gangs, all had potentially destabilizing effects. Around April the program also started doing canal clearing and watershed management, gradually increasing the number of grants as another way of providing short￾term employment while decreasing potential for flooding during the rainy season. After Hurricane Thomas hit Haiti in November, 2010, the program’s M&E team visited the sites for these completed grants and found that residents reported the project work had decreased flooding. They also found 80 per cent of the seedlings planted had survived the rains. Internal planning documents prioritized labor intensiveness over speed and the use of heavy equipment; very quickly, however, skyrocketing rubble transportation costs skewed the labor/equipment ratio. As a bid to complement the GOH strengthening component and to facilitate rapid implementation, HRI looked to mayors as partners for site selection and worker recruitment. All USAID partners involved in rubble removal agreed to place the GOH at the forefront not only in consultations with the Ministry of Public Works, but also in the branding for activities – the yellow shirts worn by all workers depicted on the cover of this report. From the implementing partners’ perspective, standardizing employment conditions was important: this included safety training, provision of safety equipment, gender-sensitive training, health coverage, and managing payroll through a third party – a cash transfer company. After a short period, as cases of corruption came to light, both implementing partners also built in anti￾corruption training sessions. As one of the cash-for-work supervisors noted, “Our programming allowed us to understand how corruption works on the ground: we would adjust our regulations and they would find ways to circumvent them, especially the grassroots organizations.” As both staff members and external observers agreed, the proliferation of CBOs and NGOs following the earthquake only partially reflected goodwill: “Many CBO leaders create their own organization so as to look for funding and do not have a regular job,” was a common view. HRI reviewed and adjusted its beneficiary selection guidelines iteratively – to the point where some team members would complain they were not able to keep up with the changes. Later, the M&E team reviewed the changes and identified the following key components of an effective beneficiary selection process: 1) clearly defined criteria; 2) transparency and oversight (having all stakeholders represented in the selection committee); and 3) equitable access to information about upcoming opportunities. Different beneficiary selection methods were tried, mostly for short-term employment and training opportunities – at a later point, the program also set up a hotline to report corruption. In the long run, 22 HRI’s sharing of lessons learned with other USAID offices substantially influenced design of beneficiary selection processes for large-scale programs: “We learned not to directly select beneficiaries for housing projects, but have community leaders of different population groups do that. These organizations then held a press conference, with the GOH, to explain the selection criteria. We learned to give municipal authorities a key role.” Although estimates of the amount of debris in Port au Prince varied widely at the time, the donor community later agreed on the more conservative estimate of 10 million cubic meters. By the end of June, 2010, HRI had completed 96 rubble removal activities, providing temporary employment for close to 19,000 people, injecting $US 2.2 million in wages, and clearing a little under 500,000 cubic meters of debris – five per cent of the total. Of the people employed in the first six months, only 20 per cent were women – largely due to the physical demands. As it became evident that money shortages were forcing workers to go without meals throughout the day and as a way to redress the gender imbalance, the program provided food – contracted out to women street vendors – for the first few months. The level of devastation was such that generation of short-term employment addressed several critical needs at the same time. A population with scant savings capacity and largely dependent on the informal sector for a daily income was suddenly out of business. Rubble covered the city making circulation and a return to normalcy impossible. Manual removal of debris was, for many, the only way while people were still waiting for news of their loved ones. Fear spread that gang activity would increase given the absence of economic activity. The program’s very significant role in rubble removal and short-term employment was explicitly acknowledged by the Haiti Task Team and several Mission interviewees – in particular as it related to facilitating follow-on for housing and shelter programs. As Figure 3 illustrates, a month after the earthquake both implementing partners had started rubble removal activities in multiple sites (light blue and violet dots in first map), along with CHF and IOM working with other USG funds. Almost exactly a year later, the preponderance of HRI rubble removal sites (red dots) is evident. In June, 2010 an OIG Audit of all USAID cash-for-work activities found that OTI’s prioritization of manual removal made it inefficient; the labor/materials ratio initially planned and the employment estimates were not being met; and too much of the recruitment decision-making was left to local government authorities without oversight from grassroots organizations. The Audit recommended switching to doing more small infrastructure work, finding partners with rubble transport capacities to lower costs and improve the labor/materials ratio, and involving local organizations for recruitment. Once past the initial focus on rubble removal activities, the program gradually increased work on other community priorities in a smaller number of target areas in Port au Prince. These included vocational training, public safety, and access to services, including recreation to mitigate the potential for instability. Once implementation speed slowed down somewhat, consultation processes with the communities increased and, according to former staff members involved at that time, generated some of the more interesting activity ideas. For instance, as public opinion shifted to expectations about reconstruction and ensuing job opportunities, it became rapidly clear that many big companies were recruiting abroad because locally few were skilled in use of heavy machinery. A women’s organization from Bel Air suggested training in skills for heavy equipment and refrigeration – where, unlike most construction work, women need not be at a disadvantage. This training was later replicated during the program’s second phase for both Bel Air and Martissant. 23 Figure 3. USAID rubble removal points February, 2010 and February, 2011 24 Interestingly, during the round of interviews with people on the street in Bel Air, the evaluation team met one young man trained for heavy equipment use and during one of the focus group discussions with workers in direct implementation grants (HRI II) one of the supervisors had also been trained. The first had been able to get work sporadically and both reflected on the fact that there were never enough jobs to go around for all that needed them. From the very beginning, the program team ran up against the multiple complexities of operating in the target areas – prioritized precisely for the potential destabilizing effects of violence. Gangs posed overriding constraints for Port au Prince programming and discussions of outcomes were full of references to them, their links to political forces – particularly evident during elections, – their intimidation of community organizations, and community members’ pervading sense of powerlessness. Their corrosive effect on social fabric and rule of law in the capital’s target areas was a major concern. For instance, organizations providing training feared enforcing regulations as to tardiness and unexcused absences because of possible reprisals – they ended up attributing enforcement to the donor. Aside from that, CBOs were constantly vying for funding and discrediting each other. Both CBOs and the population thought almost exclusively of short-term needs and program staff found striking a balance challenging. Also, CBOs were widely viewed as looking out for their members, not necessarily the community at large. To a certain extent these challenges resulted in the program teams prioritizing work with what they considered a few “good partners,” which in turn led to further difficulties as other organizations disengaged or became openly antagonistic. This later led HRI II to explore other alternatives. In sum, the level of devastation and socioeconomic disruption in an already over-populated, poverty￾stricken, and hyper-centralized society plagued by instability ground it to a halt in many ways. As many interviewees, including beneficiaries, commented, without international aid those first few months would have been made infinitely worse. In that context, HRI programming provided short-term employment opportunities, assisted in rubble removal and small infrastructure repair – most notably watershed management, strengthened local and national GOH responsiveness, and later supported other more transitional activities. As the former country representative for OTI remarked and the “Independent Review…” agreed, “It is hard to measure impact by what did not happen.” In the emergency response period, HRI addressed known destabilizing factors, targeting some of the most vulnerable and complex areas. The evaluation team can only infer the program contributed to preventing the riots and looting that had been feared would happen. INCREASING CITIZEN ENGAGEMENT With approximately a tenth of HRI I funds, this component addressed critical and urgent information needs in the aftermath of the earthquake, more medium- and long-term capacity building for information providers, and elections-related work, as well as “one-off” youth and cultural events and festivals. The Haiti chapter of Internews, a media and communications-specialized INGO, collaborated with the OTI media specialist on the start-up team on the design of an emergency relief radio show (“News You Can Use” or “Enfomasyon Nou Dwe Konen”) broadcast daily for free on an ever-increasing network of Port au Prince stations. The program was funded by several donors, OTI being the largest. The following extract summarizes the organization’s vision for the grants: In humanitarian disasters people affected by an unfolding crisis have more than physical needs; they also have an urgent need for information. From earthquakes to armed conflicts, human survival can depend on knowing the answers to a few critical questions: Is it safe to go back home? What is the extent of the damage? Should I stay with my family or go for help? Where can I get clean water? 25 Where is the nearest health facility? Information in crises is as important as food, water and shelter. It can save lives, assuage fear and prevent violent conflict. It can arrest the spread of disease and political turmoil. Information empowers: It allows people to knowledgeably make critical choices about their safety and that of their families and communities. (Mandel and Sommerfeldt, 2013) Its huge success and listenership was instrumental in helping people access vital information and regain a certain sense of control, as well as NGOs to understand their target audiences’ needs. The guiding principle was that in a situation such as that of Haiti at the time, donors could not know or make assumptions as to the population’s information needs. Thus production of the daily show included intensive research – and in later grants an SMS call-in service for questions posed by the audience. By the time of the first Program Performance Review, OFDA interviewees commented the DART was considering building in an emergency-relief communication component to its disaster response package. On the UN side, an FM radio run by MINUSTAH continues to produce and broadcast a demand-driven show that maintains the ENDK name. At the time of the final evaluation, in a focus group discussion with ten former camp residents who had benefitted from an HRI resettlement activity, when asked whether they had heard of ENDK, they all assented, some of them very emphatically: “Of course! We listened to it every day in the camps! It helped us to prepare for the hurricanes, told us how to tie down our tents, helped with health information, a lot of things.” From the very beginning, aside from the radio show, Internews carried out a rapid assessment of damage to local stations and provided support to allow them to resume broadcast – including equipment. It also sequenced the production of the radio show, journalist training, and consolidation of a local research arm (for the demand-driven component). The latter two components would allow for indigenous capacity to respond to disasters with relevant information in higher quality format. Local radio shows, stated the grantee, tended to be long-winded and circular, letting interviewees ramble for long stretches, losing listeners and critical pieces of information along the way. An independent evaluation carried out in April, 2011, as the third of eight grants ended, noted three main findings: (1) the exclusive focus on Port au Prince for ENDK was a missed opportunity in that people who had fled the capital needed to know what was going on; (2) the Internews media activities were not integrated into the rest of the HRI program; and (3) the training component was too short to have the impact expected. It recommended expanding distribution of the show to stations in the provinces and decreasing the number of stations receiving training to the more committed partners with longer programs. The independent evaluation also recommended setting up the research arm on audience information needs as a stand-alone venture. These recommendations were heeded in subsequent grants. While the rationale for the sequencing is sound, it is unclear if the training objectives were met other than in a couple of radio stations – MINUSTAH and Tele Ginen, a large commercial station. The grantee tried more targeted and longer trainings held at the stations themselves through embedded advisors, but still faced significant difficulties – namely, station owners’ reluctance to give their journalists the time to train and practice the new formats. From the implementing partner’s perspective, there were concerns that OTI did not sufficiently hand over the Internews partnership as start-up wound down. Thus, when the grantee was a year late in submitting its programmatic and financial reporting, it was very difficult to resolve. 26 In addition to the listeners’ information needs, HRI I supported two other media research endeavors: media use surveys (6 activities) and media monitoring (5 activities). The former were carried out by a local company and results were widely distributed and presented to media outlets – partly as a complement to the programming quality improvements sought through Internews’ training, but mainly to provide them with information that would allow them to become more competitive. The latter was set up as national elections neared aiming at increasing transparency in election coverage, but ended up not being used outside of USG partners given the political sensitivities. Finally, increasing citizen engagement provided critical and timely support for the Ministry of Culture and Communication (MOCC), allowing the GOH to resume its information function – and, to a certain extent, its leadership after a prolonged silence. The MOCC received support for 13 grants for a total of $564,000. These included the first-ever government press center, training, strategic communications technical assistance, and equipment. The following quote summarizes GOH perception of HRI support: I cannot speak without a lot of emotion, it was extremely important. We were working under a tree… They [HRI] built the first Press Center, set up an office next to the tent, and provided all the necessary materials, and for two to three months we put out information 12 hours a day. The impact was enormous: all the government officials gave conferences at this Press Center, all the journalists sought information there, and all the donors were there, as well. It was a difficult moment, everyone was upset because of personal losses, stressed, and we had a tremendous responsibility. At the time we also had to answer the phone, people were looking for their relatives, so we also established a call center at the Press Center. -Marie Laurence Jocelyn Lassegue, former Minister of Culture and Communication The strategic communications expert drew up a plan used through the change in administration, said Ms. Jocelyn Lassegue, and provided the training for all the ministries’ communications directors. This support proved critical to enable the GOH to put forth a timely and strategic response to the cholera outbreak in October. In sum, OTI’s initial assessment of the importance of media work in responding to the disaster was highly accurate. It identified a critical need for the GOH and the population at large – and, indirectly, the donor community - and provided the immediate-, short-, and medium-term support that were required. On the very last point, however, the results are mixed. Journalists have been trained – and several left their media employers for NGO work; radio formats at two stations have become more sensitive to audiences’ information needs; the Internews research arm is legally consolidated, but face difficulties in getting enough work; and the two companies conducting the media use surveys and media monitoring services are leaders in their fields. ENABLING THE GOH TO FUNCTION HRI I initially envisioned the strengthening of GOH presence and capacity by improving its ability to perform critical functions through infrastructure, equipment, and STTA. Using a fifth of the HRI I funds, it delivered on all three counts, with some discussions as to whether it could have done more on the third – as noted earlier under “missed opportunities.” The program carried out work on 52 government infrastructure grants for a total of $US 4.97 million, ranging from setting up temporary space for Taxes, Budget, National Port Authority, to the Parliament – the latter carried over to HRI II. The former Embassy and USAID buildings were also rehabilitated for the Office of the Prime Minister, the Ministry of Planning, and the courts. In effect, HRI supported all three branches of power in resuming core functions, in addition to municipalities in target areas, with support for temporary space, infrastructure 27 rehabilitation, and/or equipment and materials. There were also 42 grants for short-term technical assistance for approximately $US 2.17 million. The fit of the OTI model to the GOH post-earthquake context was optimal, according to the former Prime Minister Jean Max Bellerive: “I doubled as Minister of Planning and had lost 16 of my 20 Directors General. I was left with no technical or administrative structure to process regular work, much less the influx of donor proposals and funding. With HRI, all we had to do was meet two to three times until they felt sure they understood exactly what we needed and it was done.” The substantial work the GOH put into reestablishing critical basic services, like fuel provision, was not seen by the population, but the HRI-supported infrastructure gave the government something visible to show. For him, the missed opportunity was on his side: had he thought of delegating coordination with the program, the GOH would probably have benefitted more – the fact that he signed grant agreements or modifications to plans at times delayed work. The former head of the IHRC also had HRI’s timeliness in mind as he recounted all other donors’ substantial delays in coming forward with the support. A former HRI-supported consultant to the IHRC regretted not having pushed more, as the administration changed, for a transition plan for IHRC to phase out. Instead, it was allowed to disappear, with no entity formally taking over its role until a few months later. He did state, however, that some of the input program-supported sectoral experts provided for the IHRC continue to be used for the ongoing design of the Plan Stratégique de Développement d’Haïti at the Ministry of Planning. That is the case for the public health plan developed by the sector lead seconded by the Cambridge Health Alliance and funded by HRI. The OTI team conducted a case study of infrastructure grants, including those provided to the GOH, around mid-2013. The sample for this study included three quarters of government infrastructure supported and found that all of them are being used for the originally intended purposes. The evaluation team visited the former Embassy building and the temporary Parliament building. Both were in use and in a good state of repair. Despite the tensions around current parliamentarians’ dissatisfaction with the lack of private washrooms as the project closed, the Senate Secretary General who offered a short walk-through was very matter-of-fact about what had been provided by HRI and what had been added by Parliament. Although the additions and alterations voided the warranty – a major cause for concern on HRI’s side – the state of the buildings and the equipment within was such that these could also be viewed as a sign of ownership by the grantee. As one program engineer noted, “We had a very solid set up that allowed us to respond to demands at a rate that the government and grantees would be satisfied… [But] on a lot of the things that we did we rushed a bit and sometimes did not take the time to understand what the grantee really wanted. On Parliament, we did communicate, but it seems they did not really take time to review the detail. Once we got the initial okay, we moved forward very quickly, and did not stop again to check with them. Things change in construction all the time, but with the time and budget constraints we really don’t want to hear about any changes.” Beginning work in the corridors. Finally, in late 2010, HRI I started working in the development corridors identified by the upcoming new USG strategy; first, one implementing partner opened offices in Saint-Marc and then the other in Cap Haitien. Early activities were aimed at supporting people displaced by the earthquake to these areas and their host families and communities. Some grants were viewed as “entry grants,” cultural festivals, sports tournaments, and replications of successful Port au Prince activities to get to know the community and local government. As these relationships started to evolve, the transition to the second program phase happened. 28 After the decision to extend OTI activities in Haiti, there was a period of transition whereby a new solicitation for services was used to select a single implementing partner. This change in program management seemed to provide a more streamlined approach to the grant making, but not without the challenges of phasing out one partner and the personnel involved. Although many of the experienced local staff members were able to transition onto the new contract, the differences in management styles from one firm to the other required a bit of stock-taking and adjustment, during which time the number of new grants slowed until the transition was completed. A lesson learned from this period, state members of the management team, was that extra management support was needed to close out the two partners and start up the one without slowing the program down. In sum, did HRI I achieve its objectives? With regards to community stabilization, the program’s prioritization of short-term employment opportunities and its assistance on rubble removal, as well as small infrastructure repair was effective. Organizational learning gradually refined its approach to beneficiary selection and community engagement – and both processes were also maximized by other mission programs. The evolution from a dispersed emergency response mode to more targeted geographic areas also followed good practices. On increasing citizen engagement, all evidence points to the media interventions’ success in getting information out to affected populations – including on cholera and elections, both on the government and non-government sides. The more medium- and long-term investments on building local media and research organizations’ capacities for a better indigenous emergency response show mixed results. HRI I effectively allowed the GOH to resume core functions through support for infrastructure and equipment. While the program was responsive to GOH and USG by fulfilling GOH technical assistance requests and fielded a significant number of experts to different offices, the startup team’s initial expectations as to seizing the opportunity for reform were not upheld. In addition, the program allowed the USG to fill gaps as the earthquake caught most USAID offices without active programs and short on staff. As the Haitian colleague on the final evaluation team stressed, however, this point-by-point review of HRI I performance against its objectives and its fit to the OTI model, does not do full justice to its value. Methodological limitations inherent in evaluating numerous discrete completed activities did not allow the team to fully capture the program’s impact on improving beneficiaries’ quality of life and allowing communities to reconvene in the midst of wrenching devastation and despair. As the program moved away from short-term employment, which took the largest part of the funding, a stock-taking exercise at that point in time would have proven very useful. 29 IV. FINDINGS: HRI II HRI II, launched in March, 2011, operated with lower funding levels and focused more on addressing the new USAID strategy objectives in the North around Cap Haitien. Work continued in Saint-Marc, as well as in target neighborhoods of Port au Prince (Martissant and Bel Air). In Saint-Marc, program activities focused on both the urban and surrounding rural areas, providing support to priorities identified by municipal authorities and surrounding communities. Part of the rationale for working in both rural and urban areas was that the program wanted to avoid increasing urbanization if rural dwellers were to see the benefits concentrated in the city of Saint-Marc. The evaluation team met with the current mayor, as well as grantees carrying out infrastructure activities such as road and canal improvements and water system rehabilitation for both urban and agricultural uses. There was a clear appreciation of the road improvements that allowed faster transport of goods within the region. Regarding the irrigation work, although relationships between those sharing water resources in rural and urban areas seemed to be improving, there were still issues with users adhering to ration usage agreements, which still led to minor sabotage on water infrastructure from time to time. It was not apparent how HRI could have improved this relationship, other than reassessing the community engagement strategy at play in Saint-Marc, or working to provide a better legal framework whereby water usage agreements would be better adhered to. It was also clear to the evaluation team that although HRI could have built a good working relationship with the former mayor, the new mayor was skeptical about the program and seemed a more difficult partner with whom to engage. In the North, the multilateral investment in the Caracol Industrial Park (CIP) is intended to create a development pole for jobs and economic growth in Haiti. It is also seen as a potential incentive to alleviate some of the overcrowding in Port au Prince by offering economic opportunities in the region. There was widespread consensus among other USAID offices, all of which had contributed funding to the HRI extension, that the program’s role in the Northern corridor was critical and allowed them to implement key activities in a collaborative, timely, and knowledgeable manner. As the focus on the North increased, the program gradually expanded to cover six communes: Cap Haitien, Quartier Morin, Caracol (initial targets), Limonade, Trou du Nord, Ft. Liberté, Terrier Rouge, and Ouanaminthe. This will be discussed in more detail later in the findings section. RELEVANCE OF MISSION STATEMENTS An additional challenge, as well as an opportunity for HRI was that funds were being provided to them from other USAID operating units so that HRI could be used to fill gaps in their planned activities until longer term programs could get up and running. The opportunity of funding more grants to new partners was countered by the challenge of being strategic with programmatic choices. For example, funds from the health unit required HRI to rehabilitate selected clinics and improve electrical wiring in the public hospital in Cap Haitien, among others. These are not typical activities for OTI, but they were well received, executed and appreciated by partners, beneficiaries and USG health staff. The mixture of funds with multiple purposes as well as the need to align HRI activities in relation to the larger USAID longer-term development goals meant that at times implementation seemed fractured or at least difficult to articulate as a clear and focused strategy. This challenge was met through the development of region-specific mission statements after the HRI staff determined that they needed a better way to focus their efforts and ensure that their strategy was 30 understood by everyone, including their own teams, who were engaging the communities and developing activities. In the north, the HRI II Mission Statement was: "To prepare the Northern Corridor for significant investment by increasing access to and communication between the community and government institutions, increasing perceptions that collective action is an effective way to solve problems, and increasing awareness of how to access economic opportunity." In Port au Prince, the HRI office determined that their mission was oriented, “to promote stabilization in Martissant and Bel Air by improving the perception of security in the community; facilitating an improved relationship between communities and government; and empowering communities to play an active role in responding to community-identified needs.” At the core of both of these statements is the recognition for improving ties within communities and between communities and for government to provide more opportunities for community participation in solving their own problems and setting priorities. This does not mean that there was no community input or collaboration before, but the latter phase of HRI II increased the focus on it, with greater coverage given to community groups in the target areas, and more staff involvement with said groups. The implementation of these missions served as the foundation for the community engagement strategies that the evaluation team focused on as an important operational aspect of HRI II. The evaluation team used these mission statements as lenses through which the focus group data was analyzed. The following section provides a comparison of these engagement strategies and an analysis of positive and negative aspects of each experience. COMMUNITY ENGAGEMENT AND THE VALUE OF COLLECTIVE ACTION While the program had initially relied on local authorities for their knowledge of the communities’ internal dynamics, during HRI II the approach was in constant evolution, maintaining links to government – which continued to be the main grantee – and facilitating its responsiveness in the target areas, promoting more collaborative frameworks for local organizations, and enabling the longer-term programs to make headway in the chosen corridors. At the same time, there was an acknowledgement that the program could contribute the OTI community-driven model to support the shift from emergency response to longer-term development. The greater intensity of direct involvement by the program with the communities gave rise to numerous challenges, highlighting the complexities of participatory processes, particularly in a context like Haiti, with up to 70 per cent of the population living in poverty, weak local governments, and low confidence in institutions. This section first examines three different models of community engagement employed by the program during the last two years of implementation and provides lessons learned and recommendations based on those experiences. It then describes the findings on increasing access to and communication between government and communities. Finally, with regards to the North, it describes the outcomes of activities aimed at increasing awareness of economic opportunity. In Port au Prince. The program had been working in the Martissant and Bel Air neighborhoods early on through rubble removal and other short-term employment activities. Initially, these activities were organized by coordinating with the mayor’s office. Once the program moved to more direct engagement with CBOs, the multiple challenges of operating in these areas became increasingly evident. CBOs were constantly vying with each other for funding and were widely perceived as looking out for their members, not necessarily the community at large. To a certain extent these challenges resulted in the program teams prioritizing work with what they considered a few “good partners,” which in turn 31 led to further difficulties as other organizations disengaged or became openly antagonistic. The HRI team realized this was an opportunity to work differently, adjusting their engagement model according to the varying circumstances in these sections of Port au Prince. Community Context. Neighborhoods in Port-au-Prince can be distinguished according to many criteria, among others: their social, cultural, and political patrimony; the actual types and orientation of associations one found there; the socioeconomic status of its inhabitants; and the presence of NGOs. Although Bel Air and Martissant are very similar in social status, they are quite different on the other criteria. Bel Air was known as an old colonial bourgeois and middle class political urban neighborhood before ordinary people started settling in around the mid of the 19th century (Rouzier, p. 99; Corvington, 1987, pp. 203-204). Still, the presence of a few well known intellectuals and political figureheads maintained the tradition of the neighborhood as a political bastion. Bel Air residents tended to consider the area as more open and even superior to other “shantytowns” partly because of the many landmark institutions that exist in the area and the many political associations and movements – which one does not find in Martissant. Thus, political clientelism seems to be more present in Bel Air than in Martissant. Bel Air residents claimed that the area had no gangs although it was declared a red zone after Aristide’s departure in 2004. Contrary to Bel Air, Martissant was traditionally a rural area where some urban families would spend their leisure time. Over the last thirty years, Martissant has received an influx of rural people and the social topography has changed considerably into a slum particularly in the hills. Many “abitasyon” of Martissant such as Nan Ti Bois and Grande Ravine (in the hills) have experienced armed conflict between gangs. MINUSTAH has contributed to the reduction of violence in the neighborhood by disarming many of the gangs. After the earthquake, there seems to have been some reemergence of gang activities, particularly after the National Penitentiary collapsed. Bel Air and Martissant both share characteristics that are associated with shantytowns and amenable to gang related violence (Qfid, 2012), among others:  Over population and severely overcrowded conditions;  Poor access to basic services;  Deplorable sanitation and public health conditions;  Low literacy levels and substandard educational and training opportunities;  Lack of economic opportunities;  Sense of despair about the future;  Disenfranchised and marginalized youth;  Lack of consistent law enforcement by the HNP and stabilization forces, especially in well known gang-controlled neighborhoods. Both areas have benefited from NGOs presence in an attempt to curb violence. In addition to supporting professional training for youngsters and anti-violence activities, Viva Rio was also involved in rubble removal. CONCERN works on anti-violence via dialogues and among diverse sectors of the population. DINEPA was doing cash-for-work and sanitation. In Martissant, IOM was the prime partner for community building via a small grant mechanism initially started as a previous OTI program. IDEJEN supported vocational training for youth. USAID Projustice provided training on civic legal education and dispute resolution techniques. CONCERN gave training on local governance through town hall meetings and forums with community members. Initiative Development provided loans to small business owners 32 Martissant Platform. With hundreds of community based organizations on record for Martissant, it was clearly a challenge to identify which leaders to work with while avoiding what could be perceived as preferential treatment. There was a marked slowdown in program activities during this engagement process in Martissant, but the end result was the establishment of a community based platform structured around 100 CBOs. Every 10 CBOs elected one to represent them on a panel of 10 that served as the leadership HRI interacted with. The evaluation team held a series of focus groups with members of the Platform leadership, other Platform members as well as community members and CBOs outside of the Platform in order to better understand the tactic and result of the work in Martissant. As the Platform and the program staff engaged with Martissant, the absolute absence of the state gave the CBOs a leading role they were not equipped to fulfill were they to continue operating in competition. The program did reap the benefits of the more direct community engagement via the Platform. Although there was a vast array of opinions regarding the equitable distribution of project benefits as well as the prioritization of activities, HRI II was able to access locations and carry out work that most probably would not have been possible without the Platform. Most strikingly, the Platform seemed to have made inroads in an increased sense of community engagement that even reached the gangs in these neighborhoods – several interviewees proudly noted that the activities under implementation did not run security risks nor were the materials stolen. In addition, the program provided important capacity building for a group of 100 CBOs with the intent of leaving behind a stable partner for future community engagement efforts. Throughout the multiple meetings with the Platform leadership, there was a clear need for more trust building and open communication with program staff; for example, surrounding the final disposition plan for program materials such as computers and furniture. Members of the platform do not appear to trust that its leadership is working equitably on this issue, while the leadership feels like HRI staff could help them be more transparent through more open and frequent communication. Given the internal problems the Platform has already encountered and the prevailing perception that, without projects to implement, there is no point to having a platform, it is hard to tell whether it will continue beyond HRI. Already, as the evaluation team prepared to leave the country, there appear to be fissures between the leadership and its members, as they began vying for the resources that project close-out will leave behind. On the other hand, program staff reported that the Platform had registered as such, with very sound internal statutes, with the Ministry of Social Affairs and was working with UNOPS, as well. The amount of effort and time spent on standing up this collaborative framework and the numerous difficulties encountered attest to the program’s relevance and persistence in finding ways to work in the target areas. If USAID intends to continue any work in this section of Port au Prince, the Platform has a good foundation to engage in further activities. Petionville Platforms. Through cooperation with the 16/6 initiative set up by the Martelly government to address the IDP situation in Port au Prince, several other community-based platforms were engaged as OTI partners. Platforms in areas around Petionville such as Nerette, Morne Hercule, Jalousie, and Canapé Vert group CBOs focused on religious, community, youth, and social issues with members including working professionals, students, and self-employed people in the informal sectors. The platforms have worked with HRI staff to identify and prioritize locations and people for short-term employment activities, solar lamp installation, rehabilitation of buildings, and trainings. It is clear from interviews that they highly valued the improved security brought about by HRI activities, namely the solar lamp installation and pedestrian bridge rehabilitation. These platforms are much smaller than the 33 one in Martissant and continue to work with UNOPS through their relationship with 16/6. This may give them a greater chance at longer term viability. Bel Air partnerships. The platform model for community engagement did not take hold in Bel Air. Due in part to strong differing opinions and some political concerns, local leaders viewed the platform model with skepticism. Instead, HRI was able to build partnerships with a few local entities and individual community leaders and work directly through them. These partners seemed to identify appropriate priorities for HRI activities to focus on with a modest amount of visible success for the program. Many of the activities seemed to be valued by the community, with benefits perceived as broadly impacting a variety of residents. For example, a number of building and sports field rehabilitations provided CfW opportunities that seemed relatively well managed and equitable, despite an isolated incident of kickbacks for job selection. In addition, the focus on artists in the community seemed valued, as this area had a strong tradition of arts and culture. There did however appear to be some issues between some of the individual partners in Bel Air. For example, a library was rehabilitated inside of a renowned vocational training center. While the intent was to provide the community with a resource, particularly for the youth, the library remains closed to the community, with no real plan in place for how it will be managed and how the training center will allow public access to its private space. The program’s flexibility in adapting to the target area’s preferences as to how to engage the wider community is to be commended. The final evaluation team was struck, however, by the marked similarities between the approaches in Bel Air and Martissant, despite the fact that both sides emphasized the differences, not so subtly highlighting the superiority of each of their models. In both cases, the narrative behind the CBO alliances seems to have been dominated by few individuals who would choose the “legitimate” CBOs to work with. Martissant Platform members expressed concern about the reduction in the number of jobs created by the direct implementation grants – the jobs being one of the main incentives for the CBOs. Bel Air leadership criticized that approach directly, saying they did not want to create a platform to apportion jobs. This latter position was belied by a focus group discussion with former short-term workers in Bel Air, who all reported they had been recruited by members of the very same CBO that did not want to apportion jobs. The dearth of employment opportunities for the population in target areas clearly posed a significant challenge for the program and its local partners. Finally, on a more abstract note, the evaluation team took these overall indications of community engagement as possible signs of a deeper transformation whereby communities, in Durkheim’s terms, move beyond “mechanical solidarity” based on relations among “a system of homogeneous segments and similar to each other” to “organic solidarity,” a system of different organs each of which has a special role (1933, p. 181). In Haiti, there is a lack of an institutional basis for the emergence of organized collective action (organic solidarity) as such. Most labor transactions take place around immediate individual interests, including the traditional exchange of labor, the “konbit.” CBOs are not alternative means to constructing organic solidarity as most do not have the means to engage in collective actions that are beyond their immediate members’ interests – and this limitation is widely acknowledged by all. As pertains to this report, community needs were prioritized by various groups of individuals through a participatory process, which led not only to the valorization of the symbolic goods that the project had implemented, but also to their appropriation. The evaluation team believes that the appropriation of common goods such as a public square, a rehabilitated school, or a soccer field, has contributed to foster a collective spirit among various residents and reinforced their sense of community engagement. Such sense goes beyond individual interest and is linked to preparedness to future development, as mentioned in the mission statement for the Northern corridor. Whether this 34 community engagement is sustainable depends on the capacity of external entities and internal institutions to bolster such value. INCREASING COMMUNICATION BETWEEN COMMUNITIES AND GOVERNMENT Community liaisons in the North. In the northern corridor, part of the HRI portfolio was also in support of local government to improve its connection with the community and its responsiveness. As the weakness of the local governments surrounding the Caracol Industrial Park became evident, early in 2012, using a tactic from the OTI Afghanistan program, HRI provided mayors with administrative assistants who could provide them with guidance as to their roles and responsibilities, as well as help them organize to handle the expected increase in workloads as the region grew. After their contracts ended, nearly 9 months later, these administrative assistants identified municipal authorities’ lack of commitment to engaging all of the community as one of the main challenges and recommended short￾term staff be allowed to help ensure government conducted this outreach. Out of this initiative, the program’s “community liaisons” were born. As such they served as the program’s eyes and ears in several of the communities where grants were being implemented. They engaged directly and on a daily basis with the mayors, and the CASECS and ASECS from the targeted areas, ensuring information flows to and from the HRI program staff and the community on it priorities and activity plans. In general, this strategy was an effective way for the program to quickly gain a footing in several communes stretching from Cap Haitien to Ouanaminthe and then to ensure progress towards improving communications between local governments and their constituencies. It is important to note that a longstanding “culture of secrecy” amongst public officials resulted in very little interaction between them and community members, with the ensuing mistrust and lack of transparency feeding off each other and hindering open communication and participation. While the opinions of the mayors regarding the value or importance of these liaisons varied between communities, generally, liaisons seemed to increase communications not only between the local leadership and community members, but also between different levels of municipal leaders, such as between the mayors, ASECs and CASECs. The liaisons also served as mobilizers of community participation which resulted in a high level of awareness of the changes brought about by the grants, which in turn increased the level of community pride among the citizenry. One instance that indicates the high degree of engagement that these liaisons brought about is the fact that the former project community liaison in Terrier Rouge has gone on to become the interim mayor, and appears to be highly regarded, even though he is not an elected official. One of the communication gaps the liaisons could not help bridge was on implementation difficulties and unilateral changes to implementation plans. Against time and budget constraints, when infrastructure projects ran into problems, program engineers would have to make decisions quickly to avoid delays in implementation. The problem was that these changes were often not consulted with the relevant program officer nor communicated to the liaison. As the changes became apparent in the infrastructure work and local authorities complained, the liaison had no information to provide. It also seems that, given the speed of implementation because of the imminent end of the program, these frustrations coming from the communities were not adequately sized by the Cap Haitien team and it wasn’t until the final evaluation visits and discussions that the real magnitude of the discontent with some activities in some communes became evident. 35 Overall, in its second phase, the program dealt differently with local leaders in Port au Prince and those in the North. In fact, greater mistrust and disengagement from local authorities in Port au Prince led the program staff to build more direct relationships with CBOs, their leaders and other well-respected community members. Their focus was to build trust and cooperation within the communities and deal much less with connecting communities and local authorities. INCREASING AWARENESS OF ACCESS TO ECONOMIC OPPORTUNITY In general, most program stakeholders felt that HRI II accurately identified and seized windows of opportunity that were aligned with the USG pillars. Furthermore, their efforts contributed to strengthening an overall sense of increased readiness for longer term development in the region. For example, in an interview with Mr. Fritz Jean, former head of the National Bank in Haiti, it was proposed that the Northern Corridor strategy anchored to the Caracol Industrial Park should be bolstered by other economic strengthening activities such as support to small businesses. Starting late 2011, HRI started building the capacity of CBOs and other non-governmental entities to support grassroots economic initiatives. One such grant in Limonade involved training and capacity building for a group of thirty CBOs, providing 150 trainees with skills in business planning, organizing and grant writing. Despite repeated clarifications, expectations were raised within this group that USAID was then prepared to fund their grant ideas, which led to some frustration that this was not the case. There were, however, clear indications that participants of these trainings took the information they learned and began to apply it to their organizations to be more efficient and better prepared for opportunities. In fact, some of the focus group participants coming out of this training commented their organization broke into two groups: the original one, with all the old leadership that refused to allow for internal democracy, and the off-shoot made up of a younger generation. In the fertile regions of the North, HRI directed several grants towards small agricultural enterprises. Training and provision of light equipment for a federation of associations of peanut farmers to increase their production and its quality was enthusiastically acknowledged by beneficiaries, who have already seen substantial increases in production as well as decreased times for preparing the land. The regional offices of the Ministry of Agriculture (DDANE) were also actively engaged in providing local farmers with access to equipment through program-supported activities. Another local NGO referred by the DDANE had requested very specific program support to improve a local women’s organization in its rice production. The ten parboilers provided had so increased women’s incomes that they had started conversations with the USAID follow-on program, Feed the Future, for replication to the whole region with 500 parboilers. Other agricultural enterprises supported in the north involved planting coconut seedlings, irrigation of small plots, training in business management and providing supplies to a small wholesaler. One enterprise had received training and irrigation pumps; however, the equipment was not functioning at the time of our site visit, and the enterprise was not sure if or when a replacement would be provided, since they had voided the warranty in attempting to repair it. Regardless of these issues, the evaluation team overall was struck by the clear indications that new skills and technologies acquired were being applied to strengthen the overall capability of local small enterprises. More directly related to the CIP, HRI invested significant amounts of funding and effort in training people from the surrounding communities on industrial sewing. In mid-2013, the HRI M&E team conducted a study of these grants. Overall, the study found the training effective in preparing large groups with no prior formal work experience in technical skills and work readiness – work ethics, behavior, regulations in the workplace. The study also found a remarkably positive feedback on the training itself, which would explain there being no attrition. With regards to placement, over one third 36 of the 800 trainees were hired at the Korean factory – although the M&E team found some unexplained differences between the lists of trainees hired provided by the grantee and the employee lists provided by the company. According to company representatives, the superior performance of these trainees at their jobs would increase their placement rates in the future. Former trainees currently employed cited by the M&E team expressed their serious concern about managers’ poor treatment of workers. All of these findings were then corroborated during the final evaluation visit, in a focus group discussion with currently employed graduates of the training program, as well as unemployed graduates. All were highly appreciative of the training opportunity, citing even personal benefits accruing from the life skills acquired. As for the work itself, the views were invariably negative, the main cause being the poor treatment of workers by managers and the non-compliance with labor regulations – such as sick leave and maternity leave. The success of the training model led to a last grant aimed at training trainers from vocational schools in the area. The discussion with these trainers was also surprisingly positive as to the value of the curriculum, the approaches, and the emphasis on planning and structure. They had all gone back to their institutions ready to make wide-sweeping changes. Unexpected effects, positive and negative on economic and political situation. Although the cash-for-work activities did positively affect communities as planned through most of HRI I, replication of these activities later sometimes reinforced expectations that these types of programs would continue long-term and be a steady source of employment and income. When these programs didn’t always continue or were shortened or more strategically used under HRI II, it generated frustrations within the communities. When these activities were replicated outside Port au Prince, as in Saint-Marc for irrigation, curing a river, and road repair work, the management committees set up to organize maintenance after the program left did not do their job. Staff wondered whether the cash-for-work model had contributed to this lack of ownership or whether it simply had to do with the HRI office closing down somewhat suddenly, with no time for the follow-up that probably would have been required with the management committees. A more positive unintended outcome identified by program staff was a greater level of direct engagement and involvement by community members than expected. For example, in Bel Air the community pitched in to help sweep and clean up after the library rehabilitation activity. They have also seen one firm there fixing solar lamps that they weren’t even responsible for, with community members providing them with lunch and water while they worked. Although the communities had prioritized the revitalization of their recreational spaces, the great sense of pride felt by community members at seeing them done and improved showed much more impact of these activities than expected. The evaluation team feels that this gets at the core of the strategy in the Northern Corridor for paving the way for development and providing visible examples of the benefits to come. Most valued activity: increased perception of security. As in the earlier program, the solar lamp installation was likely the most valued activity from the standpoint of community safety. Countless interviewees and focus group participants cited a perceived decrease in violence, thefts and sexual assaults in areas where lamps were put in, although the dearth of official, specific information records make verifying whether there was an actual decrease impossible. They were particularly well received when community members had a strong hand in determining their placement. There did however appear to be a recognition that crime ticked up slightly in areas not covered by solar lamps, presumably 37 as criminals shift their activities to locations with more favorable conditions. These findings verified those of an externally-conducted “public safety cluster evaluation.” SEIZING WINDOWS OF OPPORTUNITY: IPD RESETTLEMENT STRATEGY As thousands of Haitians languished in temporary and semi-permanent settlements as internally displaced persons, the international donor community and the Haitian government worked hard to find a solution to clearing camps on both public and private land. There were a variety of incentives offered with varying degrees of success, but whatever resources that may be provided to IDPs in camps to entice them to move out, whether it be cash or even the promise of jobs, there was always the fear that these programs would actually attract more people to the camps seeking to take advantage of these opportunities. Former PM Jean Max Bellerive stated that one of the most politically unpopular decisions he had to make was to not support a plan to provide financial resources to people living in camps because he believed it would only swell the numbers, instead of reduce them. The mayor of Petionville attempted to distribute $500 in cash in camps occupying public plazas, but those that took the money and left only returned soon thereafter or were replaced by others who had gotten wind of the rumors of handouts. Attempts were also made to resettle thousands of people on officially designated spaces set up with semi-permanent structures away from urban areas, creating whole new communities. However, this created a highly dependent relationship with the donor supporting the settlement, meaning they would be held accountable for any problems with structures, infrastructure, security, and access to services. The International Organization for Migration (IOM), together with other UN agencies and other donors had been struggling to come up with viable solutions to IDP resettlement. OTI engaged directly with a counterpart at IOM in order to think through creative options that might be more viable than those tried previously. There was mounting pressure to clear public spaces after nearly two years of occupation and the international community was looking for results. An innovative rental subsidy model devised through the OTI- IOM collaboration turned out to be very successful, although it was implemented on a relatively small scale. Instead of just handing out cash for leaving a camp, as had been tried, the model involved approaching families in camps individually and offering options to motivate them to move: (1) if they found a rental property, the grant program would provide funds directly to the landlord, as well as a stipend for moving, or (2) if they had owned a home prior to the earthquake but could not return because it was damaged, the program would pay for repairs or reconstruction of the home, or (3) if the home was destroyed, they could receive a newly built transitional shelter. Nearly all IDPs selected the first option, as the vast majority of people in camps did not own homes previously. Although the landlord was paid directly, it was up to the future tenant to negotiate as good a price as he/she could get and keep the difference between the ceiling the year’s rent – for school supplies, health issues, or other needs. The approach was piloted in two public plazas in Petionville, Place Boyer and Place St. Pierre. The program was extremely effective and replicated in other locations soon thereafter by several relief organizations. Ironically, the USG took the decision not to replicate the model through additional OTI grants because it was already being replicated by others. Towards the end of the program, however, this position was revised and OTI asked to set up a cooperative agreement to explore resettlement alternatives. The findings of a rigorous independent evaluation of this pilot resettlement model support its effectiveness: after one year, no one who received assistance had returned to a camp, nor was there 38 evidence that the movement from the camps created any additional informal settlements. A little less than half of beneficiaries had remained in the same rental agreement for a second year and the rest found other rental arrangements on their own. For the first year, beneficiaries enjoyed having extra funds to pay down debt, pay for school, start businesses or to help family members. In addition, over three quarters of the landlords reported being able to make improvements to make improvements and upgrades to their properties. The evaluation team conducted a focus group interview with former camp residents of Place Boyer and Place St. Pierre. All but one had continued renting in the same location after the first year, and only the one stated that she couldn’t afford to maintain the rent. All reported feeling much more secure after spending two years in the camps in some cases. Many described to us the experiences of the camps, which often included violent and degrading treatment by authorities, and witnessing a large amount of criminal activities in the camps. 39 V. M&E GOOD PRACTICES, LESSONS LEARNED, AND RECOMMENDATIONS ACTIVITY LEVEL The M&E system as a whole was remarkably thorough: it required an M&E plan for each activity; followed up on the pre-established indicators; gave the M&E team responsibility for all Final Evaluation Reports (FER) as a way to ensure objectivity; complemented activity outputs with interviews with direct and indirect beneficiaries; included a “grantee satisfaction” section in the FER; and, from early on, grouped related grants for a single FER. The latter, for example, included cases of rubble removal or watershed management activities in contiguous areas. Monitoring of activities during implementation was mostly the responsibility of program officers and their assistants, who tracked their observations in the database’s “activity notes.” This thoroughness, however, had its downsides: the organizational distance between the M&E and program teams frequently gave rise to miscommunication and tension. Additionally, the program’s implementation speed and spikes in activity completion ended up creating significant backlogs for close￾outs as the M&E team had difficulty keeping up. The feedback loop from M&E for course correction was thus somewhat delayed, although monitoring by other program staff compensated to some extent. The high number of ongoing activities also meant there was very little time available for follow-up of the closed grants where outcomes would take a while to show. Despite these challenges, there were notable instances of timely organizational learning from implementation, including the beneficiary selection process, guidelines for which started being adjusted as early as April, 2010, and the regulations for supervision of the short-term employment activities. In line with its emphasis on communications grounded in an understanding of people’s information needs, the program took care to research target communities’ priorities and beliefs. In late 2011, focus group studies in Martissant and Bel Air showed the communities’ main concerns were security, employment opportunities, and recovering the recreational spaces lost in the earthquake. Aside from informing programming decisions, these studies allowed HRI to test its communication campaign messages (for example, around the Caracol Industrial Park). PROGRAM LEVEL HRI’s approach to evaluation beyond the activity level varied by phase. The emergency response was measured with a greater focus on quantitative indicators (number of people employed, cubic meters of rubble removed, number of schools cleared, number of canals treated, etc.) easily aggregated across grants. To the extent that HRI I aimed at providing a timely response to emerging needs on the three components, program outputs were the outcomes. During the latter part of the second phase, the team identified five topics of interest: public safety, two geographic clusters, industrial sewing training, and infrastructure. HRI outsourced the first three of these research endeavors, leaving the last two for internal work. OTI country programs regularly face challenges when attempting to assess program outcomes given changes in objectives and a wide range of interventions. The degree to which these Haiti studies achieved their objective of aggregating outcomes 40 across multiple grants varied mostly depending on the understanding of the program rationale, the precision of the research questions and/or hypotheses, and the flexibility to come up with a design that accommodated different types of activities while tracking commonalities. The more straightforward design was the infrastructure case study conducted by the OTI M&E specialist. It drew a purposive sample of 40 infrastructure activities (above $100,000), around one fourth of which were in support of the GOH, to explore to what extent infrastructure investments made throughout HRI I and II are being utilized by beneficiaries for their intended purpose. All but one were being used as intended – the only exception had made slight adjustments. The study also found unexpected effects in the North, where significant investments in public spaces had led to expectations that more would be done and heightened rivalries between communes. However, the study took place at the very end of the program life so no adjustments to design or implementation came out of it. From a technical perspective, the more rigorous design and informative analysis were found in the study of industrial sewing training (4 grants) conducted by the HRI M&E team. The methodology included a desk review, discussion with program staff, field visits, interviews, focus group discussions, observation, and a survey. The findings were presented in the immediately preceding section and were handed over to the USAID livelihoods office for their use in the longer-term program still in procurement. With regards to the value for the feedback loop, the more timely study was the one on public safety. As the survey proceeded, the program team built in improvements to the new solar lamps grants. A cross￾cutting challenge for all research endeavors was the dearth of baseline data collected by the relevant government entities. Citizen’s perceptions of the road safety campaigns were very positive: they reported feeling more secure, witnessing fewer accidents, and greater trust in the campaign-identified moto-taxi drivers. Hospital managers, for their part, reported perceiving a decrease in the number of patients from moto-taxi accidents, but the official records do not stipulate causes of the accidents, so the precise difference cannot be established. The more complex studies focused on geographic target areas: Saint-Marc and the North (two intervention communes and a control commune). The first ended up being a collection of very in-depth grant-by-grant studies as the third-party evaluation team struggled to understand program rationale for Saint-Marc, one of the development corridors, with no attempt at aggregating outcomes across activities. To be fair, it could have been the case that there was no way to aggregate, the program had not yet achieved critical mass or alternative explanations could apply, but the external evaluators did not offer any. The second geographic cluster evaluation also required significant involvement from the program team as the external evaluators struggled to arrive at a meaningful questionnaire for the parallel surveys (of beneficiaries and the general population) and then, again, at a coherent analysis and interpretation of the results. In brief, the external evaluators found that the Northern regional objective of increasing access to and communication between local government and the community had been achieved among beneficiaries, less so among the general population. The comparison with the control commune further highlighted this achievement showcasing what happens when donor support strengthens CBOs, sidelining local authorities. On the second objective, that of increasing perceptions of collective action as the best means to address community issues, the results show the influence of external factors in the environment that are beyond program control. Simply put, while the general population does acknowledge the value of collective action, they see its truer embodiment in the traditional “konbit” – the Haitian equivalent of barn-raising, which has been gradually disappearing as CBO mobilization takes its place. The disinterested and non-manipulative collective action of konbit contrasts, in their view, with the CBOs’ prioritization of economic benefits for their members and their linkages to politics. On the 41 third objective, that of increasing awareness of access to economic opportunities, beneficiaries reported high levels of awareness and positive expectations for the future, while the general population does not. RECOMMENDATIONS All of these research efforts significantly contribute to documenting program outcomes, unexpected effects, and fed into the final evaluation team’s work. They demonstrate the value of taking time to spell out program rationale undergirding the different types of interventions, assess outcomes, and note limitations. The HRI team is to be commended for its evaluation efforts. The following points build on these efforts: 1. Implementation speed and urgent issues, as Farmer (2011) notes, kept everyone focused on the immediate term. For HRI I specifically, while outputs captured program impact to a large extent, more systematic research of what these outputs meant for the beneficiaries at the time would have been very useful as the relevant qualitative information found in snippets throughout the database gets lost. Recommendation: when a sizable portion of program investment deals with a single approach (in this case, short-term employment opportunities) or issue, an external research endeavor should assess outcomes before the program focus shifts away. 2. Both external and internal cluster evaluations generated useful information, only some of which was in time for program adjustments. Recommendation: start assessing program outcomes earlier in the program life. 3. Clustering poses great challenges when the portfolio is as diverse as HRI’s. The struggles in designing the evaluation tools illustrate how difficult it is to aggregate across grants unless there is a very precise conceptualization of the beliefs, attitudes, and behaviors the program seeks to change –and how these vary by target geographic area and target population groups. Recommendation: set aside time in internal team discussions to focus specifically on the critical perceptions prevalent among target populations and how the program message strategy is addressing them. Use this input to design M&E tools and test them. 4. With the M&E team in charge of all FERs, there is not enough time for follow-up on outcomes if these take longer, nor is there time to focus on figuring out how the program outcomes come together. The high value of the HRI internal studies challenge the assumption that only external perspectives bring in the desired objectivity – while they also saved the substantial leadership time that the external studies required. Recommendation: as cluster evaluations progress, take stock of the value of information collected and adjust the balance between internal and external M&E resources. 42 VI. CONCLUSIONS, PROGRAM CHALLENGES, AND RECOMMENDATIONS Overall, the final evaluation team found HRI implemented a remarkably fast response in an extremely challenging environment, proving its flexibility in adapting to changes in said context and to requests stemming from USG policy imperatives. The weight and operationalization of the latter, however, at times impacted the program’s ability to be purposive in setting a coherent and consistent strategic direction. In the immediate aftermath of the earthquake, OTI responded to the critical needs by prioritizing community stabilization through short-term employment for rubble removal and other small infrastructure work, communications and outreach to the affected populations, and support for the GOH. HRI then transitioned to “laying the foundation for longer-term development” as the regular programs took much longer than expected to become operational. While responding to the new USG strategy, the team cast a wide net in terms of variety of activities, identifying good local partners, and putting forth an intensive and committed effort to help build collaborative frameworks in the target areas. At that point, the HRI team brought to bear its local knowledge and the OTI bottom-up approach to craft regional mission statements that spoke to local perceptions, concerns, and priorities. Although fraught with difficulties, this approach produced results that were widely acknowledged as positive, contributing to the communities’ sense of well-being and inclusion. In line with one of the main principles undergirding both the “whole of government response” and the new USG country strategy, HRI sought to actively involve government in its activities. During the first phase, given the nature of its interventions and the context, government protagonism was achieved, despite occasional difficulties. During the second phase, results are mixed: where willing government partners were found, program effects on governance were evident in terms of increased access and communication. Where partners were less engaged, beneficiaries and members of the communities attribute the benefits of the activities to the donor and the CBOs. Government officials’ presence only at inaugurations did not seem to count as engagement in the communities’ view. For the North, in a narrow sense, HRI II succeeded in laying the foundations for long-term development more evidently in those sectors where clearer guidance could be had – more concrete plans for the follow-on programs. In a broader sense, however, the program’s impact on communities in terms of “readiness for development” was amply proven throughout the final evaluation team’s field work. Solar lamps allowed women greater freedom of movement, children the opportunity to improve their academic performance without having to migrate to better-serviced towns, and the creation of micro￾enterprises that took advantage of the nighttime socializing. Rehabilitation of recreational spaces increased people’s pride in their town, influx of visitors, and a perception of greater regional integration. Rehabilitation of public offices motivated officials to work longer hours and provided their constituencies with a clear reference point of government presence. Vocational training provided young women with the necessary technical skills, but also a highly valued opportunity to change their attitudes and behavior – as references to “structure,” “responsibility,” “discipline,” and “planning” abounded in their conversation. As staff described, the program brought about a sense of a “nouvelle ère” accompanied by a perception of inclusion in development, that is, tangible signs of incremental progress. 43 PROGRAM CHALLENGES AND RECOMMENDATIONS Preceding sections each note lessons learned and signs of organizational learning. This section reiterates the main challenges the program faced and outlines recommendations (bullets) that could serve for the future. On the gap-filling function Normally for OTI the gap-filling function applies to discrete activities or, at a stretch, a cluster of grants, not the main thrust of a country program. The protracted delay of the longer-term USG programs changed this, placing the country team in a difficult position, facing intense pressure for rapid implementation, often conflicting directives, and a lack of concrete plans as to what would follow, in addition to having to program an uncommonly diverse portfolio.  As one OTI team member suggested after reviewing the development process for the mission statements, it would have been useful earlier to intentionally create breathing space to review the strategy in depth. This would allow field teams in similar situations to explicitly lay out the rationale for each programming stream, the assumptions, adjustments and whatever components or designs are meant to be tested and then reviewed.  Three of the four OTI start-up team members commented that, as a result of the Haiti program, they had asked themselves whether a program could exist to serve a gap-filling function exclusively. This suggest OTI/DC consider developing guidance for field teams in similar situations where gap-filling is not for a small portion of the programming.  The number of directives and information queries for the field team could have possibly been reduced by formalizing the information flows. For most OTI programs, the 4-corner model helps manage information flows between the field and headquarters for both OTI and implementing partners. In this case, informal communications protocols between OTI, IP, and DOS taxed the field team.  Despite the team’s initiative in attempting to ensure follow-on programs would pick up as much as possible of ongoing activities that needed further support, it was not clear which activities would gain that extra push. Given the dearth of specific guidance as to what foundations to lay, OTI programs could attempt to negotiate in exchange for their gap-filling function a role in developing follow-on initiatives, thereby consolidating USAID investments in recovery to date. As the program reduced its short-term employment component Given that two-thirds of the HRI I grants were for community stabilization, most of which used short￾term employment, the levels of effort and funding devoted are captured, to a large extent, by the outputs. In that context, attempting to further document the psychosocial impact of short-term employment by comparing communities that received this type of support and those that did not would have been very difficult and insensitive, but perhaps secondary information – in the media, for instance￾could have been gathered. Additionally, there was a 2011 PPR recommendation that the program find a way to measure secondary impact of these activities that was not followed – how did the benefits impact the larger community? What did people do with their income? Timely research on this would have buttressed program claims to success, particularly in view of the 2010 RIG audit results.  OTI should routinely consider conducting “after-action reviews” or evaluations of large investments in similar activities for timely documentation as to program results.  Setting targets at start-up requires monitoring and documenting changes regular review and update. Reconstruction needs As field leadership commented, the post-disaster context made it only logical that the program would be asked to do infrastructure, which not only stretched the small-grants model, but also required technical 44 expertise from the outset. Although the implementing partners rapidly evolved their teams to respond to this need, alternative models could be explored.  Whether resulting from armed conflict or disaster, contexts where reconstruction work will be unavoidable merit consideration and discussion of alternative ways to implement: including infrastructure work in the initial contracts, using an engineering firm as a sub-contractor, or others. Technical assistance to the GOH The originally envisioned HRI role for STTA to the GOH in support of key reforms did not coalesce. Initial documents lay out the names for some of the entities, but the consulting arrangements involved numerous difficulties, including those mentioned earlier – donors vying to offer in-house expertise and weak absorptive capacity by the government. Beyond the initial stretch, the final evaluation team considers there could also have been a problem with arriving at a strategic definition of TA opportunities and how these came together to what ends. Nonetheless, the program was able to respond to GOH and USG requests and field a substantial number of consultants, some of whom played key policy roles, while others fulfilled administrative functions.  Identification and in-depth discussion of opportunities for TA support to the host government should be part of the “strategy review breathing space.” Should TDY support be needed – as was the case for Haiti, given the limited number of OTI staff that could be in country - the areas of expertise should be made explicit. This is particularly important given that identification of TA opportunities should be driven by the analysis of the political context.  In hindsight, from the GOH perspective, aside from the high-level technical experts, they were sorely lacking in mid-level officials, staffers to help manage the overwhelming amounts of work – including dealing with donor offers and proposals.  One of the difficulties faced was the set-up whereby consultants would at times behave as though they responded to the implementing partner that hired them. STTA for government support should formalize into the contracts that, on the substantive issues (not administrative) the consultant responds to the government entity he/she supports. Community expectations HRI II implemented intensive community consultations, with government participation, to prioritize activities and then, for infrastructure, to inform on design and plans for what was prioritized. Due to time constraints, there were no more consultative processes organized by the program. For the most part, neither resolution of implementation difficulties nor decisions on substantial changes were communicated to the communities.  In keeping with the participatory dynamics set in motion, substantial changes and difficulties should be consulted or, at the very least, communicated.  More generally, OTI can mitigate expectations by periodic communications regarding the time and sectoral constraints for its interventions to various communities’ leaders and organizations.  Training on project design and management inevitably raises expectations as to future funding of the initiatives developed, regardless of the disclaimers.  Planning ahead to build in a “very small grants” mechanism in the training grants can help manage these expectations and create incentives for participants. Clear, professional communications with community organizations The final evaluation team met with a couple of grantees who expressed their puzzlement as to the frequent unilateral changes to the work previously agreed upon. One agricultural cooperative had requested a cooling room for its sales point and was offered that and much more. Upon further analysis of grantee capacity after the grant agreement was signed, the program decided to continue, but providing other types of support. The organization was told the program had not been able to procure 45 the equipment. The fact that the reasons offered the grantee differed substantially from those the final evaluation team received reflects, at least, an unclear communications protocol. On its part, the Martissant Platform recounted multiple changes in the directives received from program staff as to the number of grants that would be funded and the number of short-term employment opportunities that would be created. The initial numbers had been communicated in a public gathering with the whole Platform – the 100 organizations – but when the numbers were hugely reduced there was no public communications and the leadership was left alone trying to explain to the other 90 organizations there would not be funding or jobs – and hardly anyone believed them.  Establishing a clear communications protocol facilitates relationships with grantees and ensures accountability. Organizational learning for a “toolbox” As noted earlier, HRI I and II experiences on outreach, beneficiary selection, and community engagement have been used by some USAID/Haiti offices. Were these processes to be documented for use by other programs in Haiti and other OTI programs, they could prove very useful. HRI’s gradual realization of intensive community engagement as the key to ensuring ownership, improving local governance, and more equitable distribution of aid is, in the Haitian context, of paramount importance, along with the operational mechanisms it developed to that effect. Follow-up on labor conditions for graduates of industrial sewing training Although this point is not as general as the preceding recommendations, the consensus among graduates of the training programs working at the CIP was unequivocal as to the poor treatment of workers and non-compliance with national labor regulations. When asked what they thought of the CIP, other members of the communities visited uniformly expressed very negative views based on the rumors they had heard to the same effect. Since the final evaluation team’s visit, at least one very unfavorable news piece on this topic has come to light. The OTI field team encountered several difficulties in attempting to support the Ministry of Social Affairs and Labor to set up an office in the vicinity and construction work was still ongoing at the time of the final evaluation team’s visit. Likewise, the IDB partner commented there was to be a labor affairs office with an anonymous complaints mechanism within the CIP itself, but it had not been implemented yet. 46 ANNEXES 47 ANNEX A - OTI/HAITI PROGRAM PERFORMANCE EVALUATION SCOPE OF WORK BACKGROUND The Haiti Recovery Initiative (HRI) is a U.S. Agency for International Development, Office of Transition Initiatives (USAID/OTI) funded project designed to support short- and medium-term activities aimed at stabilizing Haiti through support for community revitalization, improved governance, and increasing citizen engagement. USAID/OTI launched the HRI program in January 2010 to respond to immediate, post-earthquake stabilization needs. Initially, the program focused on quick-impact, rapid response interventions intended to reduce risks for instability and enable the Government of Haiti to resume critical functions. In March 2011, the program entered its second phase (HRI-II) with a focus on transitional assistance and providing a foundation for longer-term development investments. The HRI program is expected to end in September 2013. The overarching objectives of the HRI program in both phases have focused on three key areas:  Community stabilization  Support for the Government of Haiti  Increased citizen engagement As the local context and transition environment has evolved, a variety of functional approaches have been utilized in an effort to achieve these overarching objectives. A summary of functional approaches to each phase of the program, relevant time frames and geographic areas of focus are listed below: Timeframe Functional Approach Geographic Focus Area Haiti Recovery Initiative - I 1/2010 to 3/2011  Reduce risks for instability in disaster￾affected communities with large-scale temporary employment and visible progress of disaster relief efforts  Support the restoration of core government functions by providing material support, temporary office space and technical assistance for key functions  Encourage peaceful, democratic elections through information dissemination and voter education  Enhance citizen participation in relief and recovery through dissemination of humanitarian assistance and cholera prevention information  Greater Port-au-Prince  Saint Marc  Cap Haitien and surrounding towns 48 Haiti Recovery Initiative - II 3/2011 to 9/2012  Reduce risks for instability in target communities through IDP resettlement, increased access to economic opportunity, improved perceptions of public safety, and increased access to services  Strengthen Government of Haiti presence and capacity by supporting critical government functions and facilitating an investment-ready environment in the North  Engage citizens through strategic communications campaigns and community events  Martissant and Bel Air neighborhoods of Port￾au-Prince  Saint Marc  Cap Haitien, Limonade, Caracol, Trou du Nord, Fort-Liberté 9/2012 to 9/2013  Reduce risks for instability in target communities through increased access to economic opportunity, improved perceptions of public safety, and increased access to services  Strengthen Government of Haiti presence and capacity by supporting critical government functions and facilitating an investment-ready environment in the North  Engage citizens by increasing opportunities for dialogue within and between community and local/national leaders  Martissant and Bel Air neighborhoods of Port￾au-Prince  Cap Haitien, Limonade, Caracol, Trou du Nord, Fort-Liberté A summary of activities initiated between January 2010 and January 2013 are shown in the table below (this includes cleared, completed, closed and cancelled activities): Office GUC Value Non￾GUCs Value Total Total Value Haiti Recovery Initiative - I Port-au-Prince 512 $38,381,139 94 $8,142,626 606 $46,523,765 Saint Marc 30 $1,735,955 4 $29,649 34 $1,765,604 Cap Haitien 25 $1,840,969 5 $260,202 30 $2,101,171 Haiti Recovery Initiative – II Port-au-Prince 88 $16,757,465 15 $980,211 103 $17,737,676 Saint Marc 45 $4,574,055 9 $250,713 54 $4,824,768 Cap Haitien 77 $8,069,561 18 $1,529,021 95 $9,598,582 Total 777 $71,359,144 145 $11,192,422 922 $82,551,566 Note: GUC is Grant Under Contract. Non-GUCs consist of Short-Term Technical Assistance (STTA) and Direct Distribution of Goods and Services (DDGS). 49 The Monitoring and Evaluation (M&E) Strategy for the HRI program relies on three interrelated and mutually-reinforcing levels of analysis at the strategic, programmatic and activity levels. The role of M&E at the strategic level was to routinely assess whether HRI’s goal, objectives and assumptions remained relevant to the country’s transitional context, as well to USG strategic and foreign policy imperatives. At the program level, the focus of HRI’s M&E tools and practices narrowed from the overall post￾disaster context in Haiti to HRI’s three program objectives. This enabled HRI to focus in particular on whether the program’s activities were in line with these objectives, taking into consideration the key issues, geographic areas, actors and effective approaches. At the activity level, HRI intensively monitored activities during implementation and evaluated activities upon completion. EVALUATION QUESTIONS The independent, program performance evaluation should document accomplishments, outcomes and lessons learned, with the primary audiences being both OTI and the USAID/Haiti Mission. Although there is no formal handover of OTI activities to the Mission, the knowledge gained from HRI’s activities and operations can provide valuable assistance to the Mission’s planned and future programming. The evaluation should discuss and analyze program performance and success, but will also address opportunities missed or accomplishments that fell short of potential or expectations, as this information can be a useful tool in informing future USAID/Haiti programming after the HRI program closes. Specifically, the evaluation should be guided by the provisional questions below:  Did OTI programming through HRI reflect a fast, flexible and purposeful response to the complex dynamics of the transition in target areas of Haiti and seize key windows of opportunity to achieve the program goal and objectives? Were there issues that should have been addressed but weren’t?  How and how well were HRI’s activities coordinated at both the strategic and objective levels with USAID, Department of State, Government of Haiti, and other donor programs in Haiti? What factors helped or hindered coordination?  To what extent did HRI effectively lay the groundwork for other, longer-term investments by USAID, Government of Haiti and other donors?  Were there any unexpected effects of the program’s activities or implementation methodology, either positive or negative, on the economic and political situation in target areas of Haiti?  What has the program accomplished at the program goal and objective levels?  Did the results of HRI activities differ between regional focus areas? If so, what were the distinguishing factors between regions or between the approaches that HRI took?  What were the program outcomes and effects (intended and unintended) on the various beneficiary groups? Beneficiary groups should include: grantees, public officials and citizens.  What activities undertaken by the program were most and least valued by local partners (community, associations, government, etc.)? Why? 50  What lessons can be derived from the HRI experience to inform USAID/Haiti on strategic approach and methodology after OTI departs Haiti? To what extent are these lessons contingent upon OTI-specific operations and financial mechanisms, and are there aspects of the strategic approach that cannot be replicated for this reason? METHODOLOGY This developmental evaluation will be largely qualitative in nature and include mixed methods to increase results validity and diversity. Methodological specifics will be worked out between the evaluators, OTI/Haiti and OTI/DC and the evaluators are encouraged to suggest new methods. The evaluation should NOT focus on questions that often concern more traditional development programs. For example, as a small grants program that modifies its direction depending on the evolving situation in￾country, long-term sustainability should not be the primary area of focus. Incremental progress in stability, democratic trends taking hold, community involvement, and an abeyance of recent violence or significant disruptive events are targeted to buy time for longer-term development initiatives to gain a footing. Key differences between short-term stabilization programming and long-term development programming need to be acknowledged by the evaluators, and incorporated into the evaluation’s design and analysis. While some monitoring and evaluation work has been done over the program’s history, and the program database is a rich source of information on individual projects, there are no existing, comprehensive baselines of HRI programming. This is typical for an OTI program that is launched based on overall political and contextual analysis, but without protracted pre-program gathering of baseline data. Thus, a certain degree of reconstruction will be necessary. Similarly, the program is not set up with randomized control groups or other experimental or quasi-experimental designs. Where possible, the evaluation will gather third-party baseline data, comparison group data, or other data to complement existing information. For example, comparisons of results in intervention areas versus non￾intervention areas (of similar size, demographic composition, development, etc.) may be appropriate in some contexts or geographic areas and not others. The evaluation team should be prepared to modify its approach based on the unique characteristics and cultural nuances of HRI’s various geographic focus areas. Possible methods for the evaluation include:  Facilitated workshop with key program staff to reflect on program implementation, challenges and successes;  Field visits to the implementation areas;  Interviews with key program stakeholders, including US Embassy and USAID staff, community leaders, government officials and beneficiaries;  Focus group discussions and interviews with beneficiaries, non-beneficiaries and with grantees;  Observation;  Surveys for beneficiaries, non-beneficiaries and other stakeholders; and  Documentation review e.g. quarterly and annual reports, SRS and PPR reports, existing data and review of HRI’s activity database. 51 TIMELINE Preparation – April 2013 (3 weeks)  Consultants attend a facilitated meeting with OTI Washington and Haiti staff to discuss SOW, evaluation questions, methodology, travel and logistics planning.  Develop and submit outline of methodological approach, data analysis plan and timeline of key milestones.  Conduct literature review of program documentation and external sources.  Database analysis – the team must understand and use the Activity Database. Relevant documentation and database access will be provided electronically by OTI; OTI will also provide database trainings as part of this preparation phase.  Conduct interviews in Washington, DC with current and former OTI staff, as well as staff from other USAID or USG offices, DAI and Chemonics. Conduct other phone/in-person interviews with relevant US Embassy personnel no longer in Haiti (Ambassadors, USAID Mission Director, US Department of State personnel, etc.).  Develop data collection and evaluation tools. Field Work Trip 1 – May 2013 (3-4 weeks)  Conduct field interviews with OTI/Haiti staff, HRI program staff, USAID/US Embassy staff, Government of Haiti representatives, grantees, community members, program beneficiaries and other key informants.  Collect data and conduct site visits.  Provide an out-briefing to OTI/Haiti team and via teleconference upon return to OTI/Washington.  Preliminary analysis and further research, as needed. Field Work Trip 2 – mid-July to mid-August 2013 (3-4 weeks)  Conduct follow-up or additional interviews in Washington.  Conduct field interviews, collect additional data, conduct site visits in Haiti.  Analysis of results.  Begin writing report.  Provide an out-briefing to OTI/Haiti team and written debrief upon return to OTI/Washington. Report Writing and Outbrief – August/September 2013 (3 weeks)  Complete report writing.  Final report and summary presentation.  Briefing to OTI, and others as requested, in Washington. Milestones Time frame Brief outline of the methodological approach for the evaluation and rough timeline for accomplishing key milestones At least one week prior to field work trip 1 A proposed itinerary for field study and list of interviewees At least one week prior to departure Final versions of data collection tools (surveys, focus group questionnaires, etc.) As developed Draft program performance evaluation report September 15, 2013 Provide a teleconference or in-person briefing to OTI’s Senior Leadership, Haiti Management Team and other USG/external briefings (possible) in Washington prior to completing final version mid-September 2013 52 Milestones Time frame of the report Final report Due no more than 2 weeks after receipt of all comments from OTI on the draft report (NLT October 5, 2013) Power Point (or similar format) presentation summarizing key findings, conclusions and recommendations Submitted electronically at the same time as the final report Data collected during course of evaluation Submitted electronically at the same time as the final report 53 ANNEX B – LIST OF INTERVIEWEES AND FOCUS GROUPS INDIVIDUAL INTERVIEWS Name Organization HRI Position/Title Category Nicole Widdersheim USAID/OTI Former Country Rep OTI Team Elizabeth Winger USAID/OTI Former Deputy Team Lead OTI Team Anita Sachariah USAID/OTI Former PM OTI Team Elizabeth Martin Chemonics Former Deputy Country Rep OTI Team Stephanie Broughton DAI Former DAI DCOP Program staff Tom Adams U.S. Department of State Haiti Special Coordinator USAID/USG Eileen Wickstrom Smith U.S. Department of State Deputy Coordinator of Assistance USAID/USG Johanna Mendelson Forman CSIS Senior Associate Independent Resource Patrick FN Pierre USAID/OTI Governance Expert Start up Team Fritz Weden USAID/OTI Sr. Field Advisor Start up Team John Rigby USAID/OTI Sr. Transition Advisor Start up Team Bridgette Burke Chemonics Former DCOP Start up Team Nancy Ruppel USAID/OTI Field Program Manager OTI Team Austan Mogharabi USAID/OTI Deputy Country Rep. OTI Team Lewis Tatem USAID/EGAD EGAD USAID/USG Steve Olive USAID Deputy Mission Dir. USAID/USG Carl Siebentritt DoS/INL Chief USAID/USG Marcia Glenn USAID Transition Advisor USAID/USG Rhett Gurian Chemonics COP Program staff Mario Nicoleau USAID OIEE Team USAID/USG John Robinson DoS Econ Chief USAID/USG Charlotte Eddis USAID Health Coordinator USAID/USG Jessica Reitz USAID/OTI Country Rep OTI Team Fritz Longchamp GOH Préval Chief of Staff GOH Bernard Jacques Chemonics Project Officer Program staff Patricia Dyer Chemonics DCOP Program staff Lee-Siane Jaeger Chemonics Project Officer Program staff Josh Kunin-Goldsmith Chemonics Former COP/DCOP, CfW Support Program staff Amy Patanasinth Chemonics Senior Grants & Subcontracts Manager Program staff Kedner Pierre Chemonics Cash-for-Work Specialist Program staff Melissa Rosser USAID Director of GJD Office USAID/USG 54 Name Organization HRI Position/Title Category Stanley Fardin Chemonics Senior Engineer Program staff Leopold Cine Fondation Maurice Sixto Acting Coordinator CBO/Partner Lionel Benjamin Ti Souf CBO/Partner Rosny Desroches Civil Society Initiative CBO/Partner Christine Jacques DAGMAR CBO/Partner Mecure Homere MTPTC Director, Saint-Marc Office GOH Jean Rony Eugene GOH Mayor, Saint-Marc GOH Jean Michel Renon SESAM Director CBO/Partner Ambassador Merten DoS Former U.S. Ambassador to Haiti USAID/USG Jean Junior Lalis Petit Journal Director Beneficiary Rose-May Guignard CIAT Senior Urban Planner GOH Michael DeLandsheer UTE Executive Director GOH Emmett Fitzgerald IOM Former Strategy and Policy Advisor and IDP Camp Manager CBO/Partner Julia Weinstock Chemonics Sr. M&E Officer Program staff Natalie Brunet Chemonics Former Program Officer Program staff Fritz-Gerald Pierre USAID/OTI M&E Officer OTI Team Elie Lafortune Chemonics Sr. M&E Officer Program staff Laurence Estime Chemonics Program staff Sinclair Cornell USAID/OTI Communications Specialist Start up Team Tom Stukel USAID/OTI Former Country Rep Start up Team Eric Rolex Joseph Chemonics M&E Officer Program staff Tarseline Bien-Aime Chemonics Program Officer Program staff Marie Baptiste Chemonics Senior Program Officer Program staff Jess Bryant USAID/OTI Field Program Manager OTI Team Vasty Guerrier Chemonics Program Officer Program staff Justin Kosoris DAI/Chemonics Senior Grants & Subcontracts Manager Program staff Regine Zamour DAI/Chemonics Program Officer Program staff Serge Guilleme RECOPAN Director CBO/Partner Fritz Jean CYAT Former Chamber of Commerce President CBO/Partner Batuel Bertrand Mayor of Limonade GOH Violet St Preux INDEPCO Office Manager CBO/Partner Marie-Laurence Lassegue MoCC Former Min. of Communication GOH Rhett Gurian Chemonics COP Program staff Gabriel Verret IHRC Former Dir. IHRC GOH Jamil Simon Consultant Communications Specialist Program staff Max Goldensohn DAI Start up Team Lead Start up Team Frank So USAID/OTI PM Start up Team Corrie Drummond USAID/OTI Former Field Program Manager, Haiti OTI Team Rebecca Coolidge DAI Former COP Program staff 55 Name Organization HRI Position/Title Category Clair Sartiaux Chemonics Regional Program Manager Program staff Albert Pierre Paul Joseph CLES Executive Director CBO/Partner Jean-Marie Théodat University in Limonade Geography Professor Independent Resource Sylvain Côté IHRC Former Administrator Consultant Geneviève Olivier Platform Nerrette Committee member Castel Célestin Platform Morne Hercule Secretary Clifford Toby Bel Air en Action Committee member Jocelyn Privert GOH Senator LIST OF FOCUS GROUPS Focus Group Name # Participants Organization Category Bel Air CBO Leaders 4 Bel Air CBOs (ex Bel Air en Action) CBO/Partner Martissant Platform Leaders 8 Martissant Platform CBO/Partner Program Officers and M&E Staff 5 Chemonics Program Staff Community Liaisons 4 Chemonics Program Staff MFK Peanut Farmers 15 Meds and Food for Kids Beneficiary Limonade Direct Beneficiaries 9 Beneficiary Limonade Non-Beneficiaries 9 Other Community Members Limonade Local Leaders GOH Trou du Nord Beneficiaries 5 Beneficiary Trou du Nord Beneficiaries 15 Beneficiary TdN Mayor Group GOH TdN CASECs ASECs GOH INDEPCO Trainees 11 Beneficiary INDEPCO Trainers 12 Beneficiary OpenStreet Mappers 9 Beneficiary Computer Lab Students 10 Beneficiary Martissant Platform 1 - CSL Trainees 8 CBO/Partner Martissant Platform 2 9 CBO/Partner Bel Air Artists 17 Beneficiary 56 GROUP INTERVIEWS Group Name Number of Participants Participant Name Organization Position Category OTI Senior Leadership Team 6 Stephen Lennon OTI Acting Deputy Dir. OTI Team Nealin Parker OTI Acting Dir. OTI Team Bill Hyde OTI Chief, Field Programs OTI Team Karma Lively OTI Chief, OMD OTI Team Rick Swanson OTI Chief, PLI OTI Team Katherine Donohue OTI LAC Team Lead OTI Team OTI Program Manager 3 Leslie Maccracken OTI PM OTI Team Lauren Lewis OTI PM OTI Team Dan Tirrell OTI PM OTI Team USAID Haiti Task Team 7 Russell Porter USAID Director, Programs, USAID/OAPA USAID/USG Beth Hogan USAID Sr. DAA, LAC USAID/USG Rosalie Fanale USAID Sr. Advisor, HTT USAID/USG Gene George USAID Sr. Advisor, HTT USAID/USG Jacques Bernard USAID Sr. Advisor, HTT USAID/USG Belinda Bernard USAID Desk Officer USAID/USG Julie Denham USAID Former Sr. Advisor, HTT USAID/USG Chemonics Program Management Unit 6 Renee Lewis Chemonics Program staff Lindsay Madson Chemonics Manager Program staff Michael Geerston Chemonics Home Office Director Program staff Hayley Edwards Chemonics Manager Program staff Dylan Butler Chemonics Associate Program staff Amy Trill Chemonics Associate Program staff Syndicat des Chauffeurs 4 CBO/Partner L'Hôpital Justinien 2 CBO/Partner Radiovision Nordesienne 4 Beneficiary Union Force Caracol 5 Beneficiary Scagitech CBO Trainees 45 CBO/Partner 57 ANNEX C – INTERVIEW QUESTIONS AND DISCUSSION GUIDES KEY INFORMANT INTERVIEW QUESTIONS Local and National Government Stakeholders 1. In what ways did your Ministry/Agency/Office collaborate on any strategy to respond to post￾earthquake needs, as well as any OTI grants in which you were involved? Who would you say led the process? 2. What measures have been taken by your Ministry/Agency/Office to better prepare for and mitigate the effects of future disasters and/or issues of instability? 3. What form of assistance did your Ministry/Agency/Office receive from OTI grants? a. STTA b. Training and/or capacity building? c. Materials? d. Financial Support? 4. What was the outcome of the support? 5. What improvements in public services have occurred since the earthquake? What services, if any have declined? In what way has USAID/OTI contributed to those outcomes? 6. Have there been any reform measures undertaken to improve efficiencies and services? If so, in what way has USAID/OTI contributed to those efforts? 7. Do you feel that USAID/OTI has contributed positively or negatively to citizen participation? How and why? 8. FOLLOW UP: Does your Ministry/Agency/Office have a process to capture public/community input or feedback on decisions to prioritize or improve services? If so, what is the process and how is it different now than previously (before earthquake? Immediately after earthquake?) If not, why not? How has USAID/OTI contributed? 9. Do you feel that USAID/OTI has contributed positively or negatively to community stabilization and/or security? How and why? 10. Have there been explicit attempts within the public sector to “build back better” in terms of transparency and efficiencies? If so, has it been successful and how and do you feel that USAID/OTI contributed to this? 11. Were there any key opportunities missed or issues that should have been addressed but weren’t when working with OTI? 12. How well did the OTI grants address the long term strategic needs identified by the Government of Haiti/local government and how well did they lay the ground work for long term development in general? 13. Were there any unexpected results or consequences of OTI activities? If so, what were they? 14. What activities did you value the most? Which did you value the least? Why? 58 15. Did your Ministry/Agency/Office learn any lessons for future disaster response or other strategic processes? If so, has there been an attempt to formally collect and implement those lessons? OTI Team Members 1. How long were you involved with the program? 2. What made OTI unique in Haiti post-earthquake? 3. What was the coordination like with other USG actors? Other donors? 4. What worked well in the program? What didn’t work as well as expected? Why? 5. In hindsight, would you say the program missed any key opportunities? Made the most of those that came up? 6. Any lessons learned? For OTI? Unexpected outcomes (positive or negative)? 7. How did community engagement & participation fit in? Variations by geographic region? By type of activity? Proportion of grants responding to community-identified priorities? 8. Other? Questions for BRESI 1. What were the main achievements for the Internews activities on: a. production, b. research, c. training? 2. What remains? For example, what did Bresi do at the time and how has that changed? 3. What were the main challenges? How did ENDK shift from emergency mode to recovery/reconstruction? 4. From your perspective now, is there anything you would have done differently? Was there anything you thought was relevant at the time, but could not work on? 5. Was there an opportunity to work with ENDK in the provinces? 6. Were you familiar with other activities USAID/HRI was supporting at the time? a. What were the coordination levels? 7. Lessons learned? Questions for Grantees and Beneficiaries Grantee 1. Since when does the organization exist? 2. What are the main objectives of the organization? 3. How many members does the organization have? Men……………Women…………… 4. Brief description of the structure of the organization. Levels and Decision making process? 5. Why did the organization come up with this particular project? 6. How did the organization reach agreement with its members about the project? 7. What did the organization do specifically with the help they received? 59 8. Did the organization target any specific zone, sector of the population or groups? 9. What were the main challenges the organization faced in implementing the project? 10. What were the main effects of the activities undertaken? 11. Were there any unexpected results of the activities undertaken? At what levels? 12. What is left of what you have done? How do you try to maintain any inroads you have made? 13. Looking back now, are there things you would have done differently? 14. Did you have to collaborate with any other organizations or people to implement the project? 15. What criteria did you use to recruit participants to the project? 16. What this the first grant you ever received? Did you ever receive any grant from another source? Beneficiary 1. In what way do you think the project was beneficial to the community? 2. What does the project mean to you? 3. Are there things that you have realized because of the project? 4. What satisfaction/dissatisfaction do people have with the project? 5. What do you think would need to be corrected if another project like that were to be implemented? DISCUSSION GUIDES Approaches to community engagement in HRI Introduction:  The program approached community engagement differently in Martissant, Bel Air, and the North. The different ways seem to have evolved naturally in the program’s quest for greater community buy-in, as it sought to carve out room for a more strategic approach.  The RIG audit emphasized the program should prioritize community engagement processes and a more bottom-up approach.  The program used other ways of engaging the community since the beginning, especially in the CfW component that reached the community through local government officials. Apparently, at the time, the program could not approach the community directly, only through the mayors (to confirm).  A more systematic approach to community engagement started later in the program life, as strategy statements specific to each office were developed. The main objective for this research is to better understand how the program engaged with the community and how this affected its results. It is the final evaluation team’s understanding that both program staff and grantees consider this more systematic and context-grounded approach to community engagement more pertinent and efficacious than the earlier processes. Given how recent this approach is, discussions will focus on “community engagement,” “program benefits,” and expectations and plans for after the program ends as a way to bring up discussion of the processes involved. These discussions could also bring to light some of the higher-level outcomes alluded to in previous conversations with program staff and grantees: a renewed sense of identification with and pride in the community –including a restoration of the social fabric, a perception of greater inclusion beyond the areas traditionally benefitting from projects, and a contribution to conflict mitigation or resolution. It is possible, however, that these outcomes are not widely known or understood as such by the community 60 given how recent the shift was. If the latter is the case, are there any promising signs perceived by the larger group or community? Target area variations. The final evaluation team has thus far delved more into the community engagement process as related to the Martissant Platform. Program activities in Bel Air have proceeded differently, as stated by both program staff and grantees. Leaders from partner organizations in Bel Air were emphatic as to their community being more fractured than Martissant. For this reason and given prior experience with political platforms, Bel Air organizations preferred to work in coordination, rather than as a formal platform. Further research on the following topics is needed: How many organizations for work in Bel Air has the program engaged? Where are they based? How were they identified? How did grant implementation proceed? What evidence is there of community buy-in beyond the specific organizations’ constituencies? Has the last stretch of programming (since Nov.-Dec. 2012) marked a shift in how the program engages the community in Bel Air? Based on that information, the research can turn to outcomes –of which little was said in previous interviews. For the North, the shift to a more intensive community engagement process appears to have happened while drawing up the community profiles and mission statement –and the inclusion of the community liaison agents. Design Recruiting. Given the target area and study objectives, recruiting will be purposive, aiming for four focus groups of the types described below, each with an even gender mixture. This will ensure that the discussion can cover subjects relevant to all participants, rendering the results more useful. Group 1: representatives from organizations on the Platform that are not part of the 10. Group 2: representatives from Martissant organizations not on the Platform. Group 3: beneficiaries/participants in Platform activities (grants). Group 4: other members of the community (not beneficiaries or participants). We should know ahead of time who the participants for each group will be and how they were selected. In some cases, it is possible participants’ attitudes about the Platform and its workings are known ahead of time; a mix of positive and negative views is strongly encouraged. Should the facilitator consultant have contacts and prior experience in the area, he could help with recruiting. Composition. Ideally each discussion would be comprised of a group between 6 and 8 participants. The number should be small enough so that there is ample opportunity for each person to provide input, but large enough to encompass a good range of opinions. Budget. Facilities, snacks, and a transportation allowance for participants. Focus group script Aside from the introduction, the following script focuses on the study of the Martissant Platform. The general gist of the questions applies to programming in Bel Air and the North, but the specific wording will be adapted. Introduction to the study, including the following points:  Program ends, this is part of the final evaluation.  We are interested in better understanding how the program engaged with the community and how this affected its results.  We are as interested in what worked as in what did not work. 61 1) Representatives from organizations on the Platform that are not part of the 10:  How do you view your role/relationship with the Martissant Platform and what are some examples of your involvement?  What benefits does the community gain from the establishment of the Platform? What has been overlooked or worsened by the Platform? o Have you perceived an increase in a sense of identity as a result of Platform activities? Of pride in the community among participants in the activities? Community cohesion? o Has the Platform intervened to mitigate/mediate conflicts within the community?  How has the establishment of the Platform changed the way that your organization engages with or interacts with the community? Has anything changed? o How representative do you think the Platform is? o Have areas beyond Grand Martissant been effectively included?  What would ensure your continued commitment to engaging your community via the Platform?  Do you see any potential threats down the line (that the Platform itself concentrates too much power, loses its contact with the community, others?)? o Have you discussed these threats? What can be/is being done to prevent them?  Aside from the USAID-supported activities, has the Platform mobilized to solve other community problems, either using community resources or requesting support from other donors, the national or the local government?  What are your expectations for the future of the Platform? 2) Representatives from Martissant organizations not on the Platform.  Have your heard about the Martissant Platform? What do you know of it?  Why is your organization not involved in the Platform as a member? o What benefits do you see from not engaging as a member of the Platform? o What benefits would you see from engaging as a member? As a partner?  What are the main challenges organizations working in Martissant face? What do you think the main challenges facing the Platform are? o What is your organizations’ relationship with the local government? The national government?  What are your expectations for the future of the Platform? What would you recommend? 3) Beneficiaries/participants in Platform activities (grants)  What do you think are the main problems Martissant faces at present? Do these vary by areas or population groups?  What programs/activities have you been involved with related to the Martissant Platform? What was your involvement?  What worked well, what didn’t work well?  What do you see as the greatest benefit of the activity(ies)? o Has the Platform addressed a priority need?  Has the Platform changed the way you interact with your community to make decisions or solve problems? In what ways?  What are your expectations for the future of the Platform? Your fears? What would you recommend? 62 Analysis. The evaluation team (including the consultant) will analyze the focus group transcripts and notes, noting points of agreement and disagreement. Analysis will also include observation of the group discussions and interaction, as well as information on the context. Additional interviews In addition, for Martissant, either before or during our second trip, it will be important to seek out individual interviews with members of the community to gain complementary or more in-depth information along the same lines. We need to define what areas of Martissant can be covered. Interviews with the government officials who interact with the community organizations are also critical. A final point for this study is assessing program-supported activities aimed at strengthening community organizations to sustain project gains. This includes training for community leaders (Martissant) and support for maintenance committees (in the North), as well as conducting further research to determine what Bel Air organization are planning for along these lines. 63 Focus Group Discussion Guides Community Focus Groups – Local Government Questions 1. What was your role in project implementation? 2. How would you have improved the process of prioritizing and selecting projects? 3. What is your opinion of the local government relationship with the program? 4. Has the relationship changed among local authorities and how? 5. What would you say are the most significant changes in the community? Why? 6. Has there been a change in the way the community mobilizes to make requests? 7. How do you feel the industrial park in Caracol will affect your community? 8. Is there anything that you have seen in the community that you haven’t seen before? Community Focus Groups - Beneficiary/Non-Beneficiary Questions 1. How were you engaged in the activities? (Beneficiaries only) 2. Some people feel that there have been changes in the community, some say not. What would you say? 3. IF NO CHANGES: Why not? 4. IF CHANGES: What are the challenges you face in maintaining positive momentum/overcoming negative changes? Governance: 5. What is your current relationship with community leadership? 6. Do you feel that you have a say in what is being decided for your community? 7. What are your opportunities for communication with local leaders? Examples. 8. Would people in your community say that there is more confidence in/credibility of local leaders? 9. How were the projects prioritized and selected? Examples. 10. If discussion on communication doesn’t lead to specific discussion of projects, we can probe further on specific activities. Access to Economic Opportunities 11. What do you know about the industrial park in Caracol? 12. What have the projects allowed you to do that you couldn’t do before? Bel Air Focus Groups 1. How were you engaged in the activities? How did you become involved and how long did you participate? Did you participate in any other activities under this program? If so, what? 2. What did these opportunities allow you to do that you couldn’t do before? 3. What worked well? What could be improved about the activities? 64 4. Some people feel that there have been changes in the community, some say not. What would you say? IF NO CHANGES: Why not? IF CHANGES: What are the challenges you face in maintaining positive momentum/overcoming negative changes? 5. What do you think are the main problems your community faces at present? 6. Do you feel that you have a say in what is being decided for your community or how problems are addressed? 7. What are your opportunities for communication with local leaders? Examples. 8. What recommendations (if any) would you have for improving communication in Bel Air for improved problem solving? If time permits, or we decide we want to ask for a comparison with Martissant: 9. Did you know that in Martissant, the residents formed a platform to make decisions and prioritize activities for the neighborhood? What are your opinions of the Platform there? Would something like that help or hinder decision making and conflict resolution in Bel Air? Martissant Focus Groups: Questions for Platform Members 1. When did you become involved in the Martissant Platform? 2. How has the establishment of the Platform changed the way that you engage with or interact with others in your community? What are the main challenges that the Platform is facing now? 3. How do you think the Platform can address them now? 4. What in your perspective on the role of the Platform in the community? 5. What are the strengths and weaknesses of the Platform? 6. What incentives are important for you to continue to engage with your community via the Platform? Questions for Non-Members (trained by the Platform) 1. Date of creation and Objective of the organization? 2. How did you all get in touch with the Platform? 3. What kind of training you received from the Platform? 4. Have you had the chance to use the training? 5. What in your perspective the major role of the Platform in the community? 6. What are the strengths and weaknesses of the Platform? Questions for Non-Members 1. Date of creation and Objective of the organization? 2. What in your perspective the major role of the Platform in the community? 3. What are the strengths and weaknesses of the Platform? 4. Why are you not involved in the Platform as a member? 5. What benefits do you see from not engaging as a member of the Platform? 6. What are your expectations for the future of the Platform? 65 Questions for Beneficiaries 1. What programs/activities have you been involved with related to the Martissant Platform? What was your involvement? 2. What worked well, what didn’t work well? 3. What in your perspective the major role of the Platform in the community? 4. What are the strengths and weaknesses of the Platform? 5. What do you see as the greatest benefit of the activity(ies)? 6. In what ways has the Platform changes the way you interact with your community to make decisions or solve problems? 7. What did these opportunities allow you to do that you couldn’t do before? 8. Some people feel that there have been changes in the community, some say not. What would you say? IF NO CHANGES: Why not? IF CHANGES: What are the challenges you face in maintaining positive momentum/overcoming negative changes? 9. What do you think are the main problems your community faces at present? 10. Do you feel that you have a say in what is being decided for your community or how problems are addressed? 11. What are your opportunities for communication with local leaders? Examples. 12. What recommendations (if any) would you have for improving communication in your community for improved problem solving? Questions for Petionville IDPs 1. Where are you from originally and how did you end up in the camp? 2. What was life like in the camp for your family? What kind of pressures did you experience? 3. What was your experience with donor organizations and local authorities while living in the camp? After moving out of the camp? Was there any follow up with you? 4. What motivated you to remain in the camp? What motivated you to leave the camp? 5. Was your previous dwelling classified as a red or yellow house? Did you return to where you were from? If not, where do you move to and where do you live now? 6. Were you provided with a rental subsidy? If so, for how many months? Was the rent given directly to the landlord or to you? Were you able to negotiate a lower rent with the landlord? How have rent amounts changed since you left the camp? 7. Was or is your previous home under repair as part of the program? 8. Was everyone in the camp selected for the program? If not, how were families selected? 9. What satisfaction/dissatisfaction do you have with the camp clearance project? 10. How has your life changed since you moved from the camp? 11. What do you think would need to be corrected if another project like that were to be implemented? 66 ANNEX D – EVALUATION QUESTIONS MATRIX: INFORMATION COLLECTED IN INTERVIEWS AND FOCUS GROUP DISCUSSIONS Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned USAID/USG Context: -Mission in transition to new strategy; move to USAID Forward had ground its ability to procure to a halt. High value of OTI nimbleness. -Unusually high level of pressure to implement & of oversight; not as much freedom to define own strategy. -IEEE: OTI ability to adapt & work at local levels -DG: OTI was really valuable in regaining credibility for the Mission’s ability to deliver -Amb: “ability to -Urban Planning Advisor: helped mitigate/prevent conflict; “stellar activities that fit very well in infrastructure, enhancing what the program had been asked to do” -TA to GOH after the earthquake was to be done by other DG partners; turn￾over before & after elections made it difficult -STM housing for IDP women -“Would have liked to have seen more initiatives bolstering the private sector, helping small businesses get reestablished.” -OTI procurement nimbleness, role vital -Maybe in haste did lot of projs that did not have a lot of impact (ex plazas art gallery) -Justice: coordinated use of funds (soccer fields, courts) -Infrastructure Office: very good coord with OFDA -Approval mechanism by Amb & Mission Director early on to ensure fit of grants within 4 pillars & geographic targets, “worked really well” -Big picture: starting with post-eq activities, lay groundwork for other programs to follow on: did rubble removal, could follow with housing, also explained what USAID was about. Started conversation with govt officials for all of us, to go beyond transition to more permanent programming. -On early arts & sports activities: “a respite from the earthquake was incredibly important.” -Health: got started on long￾term plan before program is online. -CFW: more than just injection of funds, brought people together, prioritize, organize for equitable distribution, renew activity -Informative to know that some messages we understand are not understood by the population at large (in ref to CIP info campaign) -GOH constant changes in authorities at national & local levels (ex. St Marc new mayor upset at program because too late to respond to his requests) -Difficulties procuring STTA to GOH -INL: hard to measure outcomes of work on security (road safety & public safety) -New programs online much slower than expected -Office of Sp -USG plans re￾visited, ex.STM, time wasted. -Coordination with other donors facilitated targeting (ex. Canadians in south) -GOH buildings, including courts in the North -Press Center & training -Clearing rubble -Cholera response, -Information on voting centers -Resettlement -Solar lamps -Stabilization overall (seemed like it reduced political instability) -“Fast, immediate impact, design wasn’t aimed at long-lasting impact.” -National Budget transparency replicated later at much bigger scale -Communications campaign in N counteracted: “govt communication is -Very critical role, other progs had no provision for infrastructure, also as new programs delayed, OTI filled gaps “some of these gaps were pretty large” -Ex: elections call center: voting centers had changed -Helped stabilize communities that were traditionally restive -Corridors strategy: - “Great value in corridors, was the glue that held things together, very strategic” -Critical interventions at -Program very successful at adapting to the local context -Built up internal engineering team Importance of outreach (CIP) & of monitoring information campaigns. Other progs learned not to have the USG at front communica ting about anything. -“We learned lessons on how OTI worked on outreach & communica tions. Them being on the ground early helped pave the way.” -Learned 67 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned USAID/USG contract & move quickly, wide variety of partners & tools, extremely useful;” grant ideas were “99% great, on target, & in support of what we wanted to do.” Also, “they knew the neighborhoods & the people,” “good humanitarian tool & good political tool.” “Made the USG look really good… fastest partner on the ground.” -“Impressed by the CR & DCR’s contact with key people in the community, it was really very good, something USAID officers used to do before, 20 years ago.” -“Plus speed, transparency, & paper trails required, without losing ability to move in a nimble & quick fashion in a situation like (Amb) -Prog responded to random requests from GOH -Prog stood out because had info handy, knew where had ongoing activities -EGAD: overall good coord, ex. CFI support “OTI really responsive,” also coord on support to GOH Tax Offices -Health: “Very collaborative, un￾bureaucratic;” able to change its team & hire engineers for infrastructure work -Backed up OTI when DC couldn’t understand how certain projects fit within the strategy -INL: collaboration on use of INL $ was great, engaged regularly -“I’ve never -INL: court rehabilitation in the North -CFI TA support for Prime Min’s office (strategic plan being used now) -Training facilities in North will transition to larger program EGAD: “helping to build human & social capital is part of laying the found for long￾term dev” -Housing: prog work on rubble removal & coordination with CBOs entry point for housing projects. -Housing: previously no housing agency or housing strategy. Now there is. -DG: “good bridge between immediate needs & main programs to come” C: what could have OTI done better to capture info on impact or verify their ideas or its experimental nature? Would have like them to review their projects from the perspective of the USG strategy. Not responsive enough. -Mission was very understaffed at the time of the earthquake & immediately following -“OTI was at times put in a very challenging position as policy makers in DC pivoted to seeing as very useful to implement” and had to spend a lot of time following political imperatives -“Some of the gaps to be filled were poor & people immediately suspect that everything is nefarious & will benefit particular interests.” -Rehabilitating public spaces “quick way to show community benefits from increased activity" in the North. the right time: “great work on resettlement.” -“Up N, seems like the work is exactly what we should be doing” -St Marc: OTI started conversations for all of us, opened doors & filled gaps -Health: the program is “the only thing we can show, everyone else is backlogged 3 years.” from OTI’s community engagemen t, not to directly select beneficiarie s for housing projects, but have community leaders of different population groups do that. These orgs then held a press conference, with the GOH, to explain selection criteria. -Learned to give municipal authorities key role. “Our engineers thought it was about finding land & just building.” -At time of 2011 PPR, OFDA considering including a communica tions 68 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned USAID/USG that was extremely valuable.” seen a place as micro-managed as H.” -Econ & Commercial Office: coordinated on Cap airport pretty large;” gap-filling function was an integral part of a strategy component to its response package as part of DART -6-mo STTA requires clearly defined goals, hard to get traction otherwise because term was too short. (EGAD) -Consider more purposefull y transitionin g out embedded consultants Start-up team (OTI/IPs) -Very quick start-up; OTI DC prioritized Haiti -Id of entry points on target -Coordination w/ other USG entities, including military on communications timely & effective -“We are a tool, part of the larger USG strategy. We’re -Greater engagement on TA for the GOH, taking advantage of the window of opportunity for public sector reform in key areas -Others think that expectation may not have been realistic -Still others that the window of opportunity for public sector -Coord with military comms program -Relations with other USG entities at start-up & co￾location with IP -Hi level of coord with other CFW efforts, set up a database including geographic info. -Coord with -Acknowledge￾ment that gap￾filling role was a necessity: “sometimes we just don’t have a choice.” -In the first few weeks drew up a rough estimate of CFW employment target, which was off mostly because of rapidly increasing prices for fuel and rental of heavy machinery. The number, -Public scrutiny & info flows. USG reactive to media attention. -Audited on CFW target that was an initial estimate -Unclear how much analysis & vetting of mayors to work with -Struggled to meaningfully address wide -TA for housing policy, Unit for Damage Assessment -Were able to find willing partners in some mayors for CFW -Models for CFW: training, security equipment, coord w/ mayors, payroll mgmt. -By virtue of staff hired, each IP rapidly developed expertise on infrastructure & cabling/electricity. Chemonics then built this up as more and more infrastructure grants were approved. -Setting targets at start-up requires monitoring and documenti ng changes -Push back (on CFW extra $, write up) -Post￾disaster context made it only logical that would 69 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Start-up team (OTI/IPs) a tactical tool, we can’t have our own strategy. If that strategy itself is flawed, we are pushing a rock up the hill and having it roll down on us; a tactical weapon can’t win the war. There may have been a little of that in Haiti.” -“My impression afterwards is that we didn’t really develop a strategy for Haiti; we became the ‘go￾to’ folks. I’m not sure in retrospect that that is totally bad, although at first I did not think it was good.” -Strat. docs evolution: recommend looking to temp employment for greater benefits to the community (irrigation rehab, ex), id: IHRC as opportunity, key GOH Min; id. “laying founds reform closed around Jun 2010 -Should have pushed back harder when awarded additional 25 million for CFW -Could have sequenced activities more intentionally (infrastructure/ CFW then community development), but the program was directed by DOS from DC -Working in other badly hit areas (Petit Goave, Leogane) other USAID progs difficult because also had high turnover of TDYs -Initially daily reports to HTT -Coordination between IPs and them and OTI was excellent however, stuck and the OIG CFW Audit focused on that. range of activities: humanitarian, political transition, long-term development - “Clearly when we started there was a big gap in the Mission’s ability to do its normal work, had a hard time getting people and doing activities.” get asked to do infrastructu re, which stretched the small￾grants model -Field team overburden ed by info requests from DC: “What have we done for Haiti in the past two hours?” -“Despite all that was going on operational ly in the first six months, OTI (field and Washingto n) should have begun sooner to develop a strategic vision for the program over the ensuing years. This could have been done without tying CR's hands on 70 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Start-up team (OTI/IPs) for l-t dev” would be a challenge. arrival in June. And could have accelerated the process whereby she was able, over time, to spearhead developme nt of a strategy.” -Could have benefited from help from the Mission preparing for the Audit. OTI Team -2 IPs to 1 slowed down -Last stretch were able to take a step back (Fall 2012) & reflect more on the context & programming decisions -Gender issues from the beginning -More TA for GOH, but without more OTI staff in the field that was not possible -Working in other eq￾affected areas -Could have started elections￾related work earlier, allowed to do it at the very last minute -Continuing with resettlement, including testing -Mayor of PaP wanted to control who was hired, so contacted Min of Social Affairs as a workaround -USG emphasis on branding increased last 2 years -After Mission clearance set up, learned not to put forth grant ideas that would not be approved. -Funds provided from other operating units -Program in Cap, clear we would support the CIP, but not much more guidance than that. -Aside from health, not a lot of clarity on other sectors, so hard to determine what will be picked up -expectation that local capacity building through grants will be advantage for future program implementation -At first, communities did not want a participatory process: “just get the rubble out.” -North: community priorities focused on small infrastructure -Turn-over in GoH, including at the Min of Social Affairs to help set up a labor rights office to deal w/ CIP. -Scant political will. -Significantly slowed down in transition from HRI 1 to 2, while pressure for implementatio n remained high. Shift from DAI’s approach (direct HRI 1 – missed geo targets affected by earthquake -Differences between PaP and Cap widely acknowledged, but not necessarily between communes in the North. Political & social context varied between communes –local government officials. So did their understanding of their own role in implementation. – North: difficulties with construction -AIA least -Yellow shirts for rubble removal – beginning of a sense of unity for reconstruction -Resettlement with IOM -Community engagement -Solar lamps -Shifts in strategy (nuances) not well documented, making it hard to track what should have changed in practice -Context very difficult to operate in PaP: gangs, absence of local government, unemployment, unclear how representative CBOs are of community interests -Comm engagement: Mart couldn’t continue as before -Built in more engineering positions to deal w/ increasing infrastructure work. -Staff reviewed CFW beneficiary selection procedures multiple times to prevent corruption -Having identified lack of ownership as one of the main problems, program went -Other￾directed grants inevitably took more time & effort to implement -Given post￾earthquake scenario, should have had infrastructu re work included in the initial contracts w/ IPs or bid for an engineering firm for 71 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned OTI Team alternative models -“If had started with community profiles, would have done a better job at community engagement. We addressed people’s demand for education & services, but as soon as we dug a little deeper, a main problem was really the fact that people don’t take ownership of their community.” to fill gaps until programs come on line requires close coordination and creative programming to fit OTI strategy and mission implementatio n) to Chemonics was also a big change. -Culture of dependency made community engagement a challenge in that a lot of people’s interest in participating was related to jobs, refreshments, stipends. firms. back to direct implementation, so as to engage community (via CFW) to promote sense of ownership. -Aug 2012 Rolling Assessment: teams worked on mission statements half the funding. - “Intentional ly creating breathing space for looking at the strategy – amount of funding & fast pace made this program different.” -Going down from 2 to 1 IP requires proper support (close down, transition, start-up concurrentl y) to ensure program does not slow down. - Information & communica tion flows btw State & OTI field were never formalized Program staff -Program ability to respond to GOH requests was unique at -Whole team involved in the nitty-gritty details, could -Coord between IPs on geographic distribution, -Where management committees fail to maintain -CFW improved communities’ perception of -Turn-over in GOH -Media grants weren’t -DCOP: STM activities because in rural area stood out much more. -CFW for rubble removal highly valued; some women employed -Prog more efficient due to community engagement: -Difficulty of closing out HRI 1 helped us organize better in -CFW, beneficiary selection -Everyone 72 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Program staff the time -North: shift in latest stretch: “70% used to be directed from the top, 30% community￾driven, now a big shift, our contact is with the larger community, before it was mostly just the mayors.” -With the bottom-up approach, the build up to the concrete project makes it all the more visible (meetings preceding the infrastructure) so that the inauguration is a response to a felt need. -North rapid shifts: from working w/ a few mayors on their priorities for their communes to CIP to expansion to more communes. -There was a period of decreased project development not take a step back to reflect on evolving context. -“Discussions were too fast. Had great field people but discussions were at micro and meso levels, not macro level. No time to analyze the outcomes, distinguish between those and outputs, between immediate needs and political analysis, seeing the activities and interacting with the community. I wrote so many of the ideas and grants, discussed them, reviewed budgets, but got to see so few of them. Spent a lot of my time at my computer, on email, the database, reviewing proposals. It was a question of literally physical time to do things.” worked out eventually -“What worked well was our ability to move in quickly and address issues at levels. We had a very solid set up that allowed us to respond to demands at a rate that the govt and grantees would be satisfied.” What did not work as well: “on a lot of the things that we did we rushed a bit and sometimes did not take the time to understand what the grantee really wanted. On Parliament, we did communicate, but it seems they did not really take time to review the detail. Once we got the initial okay, we moved forward very quickly, and did not stop again to improvements makes it difficult to see how what the program did lays foundations for longer term. -Perhaps if had designed grants with more cost￾sharing in STM would have had greater community investment for maintenance. -Community engagement processes made significant headway on bringing together PaP communities in target areas, but challenges posed by the specific contexts (gangs, politicization, weakness of the State, rivalry between CBOs) may prove too large -For the North, one of major shifts from HRI 1 to 2 was the focus on more medium-term type of grants: more infrastructure & training, less CFW & festivals. -In the North: USAID -CFW best practices of security training & equipment were later replicated by other donors -Perhaps replication of CFW for other regions did not allow for community buy-in, could have explored more cost￾sharing -Judges and other GOH officials work longer hours and more efficiently -Own professional development (ex. mayor of TR) - Communities engaged much more than expected – the program was used to doing so much for grantees that it was a surprise (ex., handed over to IP after start-up (no relation w/ Internews until 3rd grant). -Staff that migrated from DAI to Chem had difficulty adapting to differences in roles & responsibilitie s -In-kind nature got little buy-in from some local authorities -Underlying problem of widespread corruption: “allowed us to understand how corruption works on the ground, we’d adjust our regulations & they would find ways to circumvent them, especially the grassroots orgs” -CBOs mushroomed after the eq -North: local However, community did not show commitment to maintaining improvements – despite the fact that management committees were formed. -STM activities used CFW (ex. curing river), which may have then led to unwillingness to work on maintenance w/o pay -Approach in STM stronger on community engagement -Very visible change on infrastructure in STM -STM sub￾contractors mostly from PaP, which created difficulties used the pay to set up small informal business -Also CFW w/ visible works (watershed mgmt.) -Resettlement; debates & screening in camps -ENDK the radio show was excellent, not so sure the training & equipment went as well -Press Center: “govt had nowhere to address the press from, there were no HQ, journalists had nowhere set for access to govt” -GOH stepped up to a leadership role for cholera response -Festivals & similar activities not valued (N) -STM working on water & roads w/comm engagement process, stopped threats to infrastructure -Cap: rehabbing local govt offices (municipalities, courts) contributed to “Avant il n’y avait pas vraiment de participation communautaire. Maintenant on demande beaucoup plus de la communauté.” “La participation communautaire aide à aller plus rapide. “ -Trade-off btw implementation speed & community engagement -On MP, “people on the Platform are those who want to stay in M, they have their children & houses there, they want to make it livable.” “The state is very, very weak, almost non￾existent, that is why citizens must change & take responsibility for changing their area. “ Cap. -M&E at activity level & cluster level -Very closely monitored CFW: surprise visits, roll call twice daily, pre-marked timesheets, anti￾corruption training, hot line -Also on CFW, excellent decision to have used Unitransfer for payroll -Given amount of infrastructure work, gradually built up number of in-house engineers. -North: community profiles: “time spent on them made sense; we were able to make connections w/ the community, leaders, public sector, churches. They gave us an understanding of what the community’s way of life was.” -Avant on était un support administratif seulement mais dans notre dernier rapport underestim ated the size of ops staff required for the burn rate. -Duration of constructio n contracts (12-18 months) limited the amount of time for proper planning. “Wish we had known we would be doing so much constructio n work, would have prepared differently.” -Very good to have had an FPM for the North -North: geographic teams made more sense -agents de liaison: more frequent meetings with POs for planning. 73 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Program staff when DAI staff were adjusting to Chemonics management system. -Elections work (staff had developed many more grants, but couldn’t implement in the short time available) -Bottom-up approach in North much better than top￾down as it was at the beginning (when projects were identified by the PaP office, the Embassy or OTI) -Huge missed opport: housing for IDP women in STM (Mission could not come to agreement on the model to use) -Missed the opportunity to do things more efficiently, but “HRI 1 was asked to react non-stop, there was political pressure to rush.” -Solar lamps “would have been much more fruitful if they had been part of a national plan for check with them. Things change in construction all the time, but with the time & budget constraints we really don’t want to hear about any changes.” -During last year (HRI 2) GOH involvement limited to signing the grant agreement (infrastructure on public lands) linked up to alternatives for sustainability: for the CLACs, built up the relationship with the Ministry of Culture & National Library; collaboration with the Catholic Church; management committees -On HRI 1 ICE, while staff valued opportunity to be creative, found it “hard to build activities that are relevant & appropriate w/o a USG strategy.” Context: USAID strategy was not yet finalized, thus could not be shared w/ staff. “Because OTI is not needs-based, it is very important for everyone to understand the program rationale.” community provided lunch & help for workers setting up the solar lamps) -Team at its best right now, morale back up (everyone understands their roles) -USAID visibility greatly increased through community engagement processes & infrastructure in North construction firms did not have easy access to credit, as did foreign firms, which made it difficult for them to compete. -Significant difficulties in verifying legality of land titles -Community expectations (want everything to be free; expected CFW to last long term) -Big change in program staffing in 2012 was detrimental, plus added uncertainty as to extension -HRI 2 started w/ divisions among staff, team not as collaborative or cohesive -Staff turn￾over also affected relationships w/ grantees & local partners, some of which refused to greater commitment by authorities to their work & improved public opinion of them -Solar lamps (local authorities’ FG in Limonade): Les lampadaires diminuent l’insécurité, facilite les élèves d’étudier le soir, diminuent le banditisme, créer le loisir. Les lampes diminuent le taux de natalité, augmentent les ventes des petites marchandes. L’augmentation du taux de la réussite des enfants aux examens. -Solar lamps challenge: batteries die in 3- 5 years. -North: “Visible change meant a lot, plus the opportunity to come together around something tangible.” -HHA (clinic) very valuable partner -Communication campaign for on a suggéré de travailler avec la population aussi et on est devenu agent de liaison -greater emphasis on bottom-up approach during HRI 2 -“ pour éviter de perpétuer la culture de la dépendance la communauté doit prendre la relève” -Importance of communication outreach for the community became evident after serious delay on the Petit Goave road (community not informed about benefits accruing from it) -Revised beneficiary selection criteria, have a Manual, included in grant agreement -Teams out in the field from beginning to ensure CFW was not put to political use. -MP: “could be a double-edged sword,” concentrates too much power Also, meet other agents to exchange experience s. -closeout meeting with community & local authorities to explain implementa tion challenges & changes. Lack of information on changes led to increased community distrust with regards to local authorities. -Cap local pride & sensitivities regarding all things from PaP was not adequately acknowledg ed from the beginning. -project docs in English caused difficulties 74 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Program staff renewable energy.” work w/ newly assigned POs.” -“Once community knows USAID behind the projects, they become insatiable, want everything -Needed more time to do things better, for POs to provide technical support for partners cholera prevention in North very timely in explaining details to local authorities & communiti es - Communit y engagemen t processes will not change communiti es in 6 months -Despite pressure for speedy implementa tion, can plan better for maintenanc e & repair, particularly of equipment provided (ex: batteries in solar lamps, incinerator in HUJ, Terrier Rouge radio station antenna, UFC water pumps -M&E 75 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Program staff backlog because of peaks in grant completion -Start designing cluster evaluations sooner; develop M&E processes by activity type. -If M&E officers in charge of FERs, small team is not enough to cover monitoring & higher￾level analysis, as well GOH national & local (CIAT, UTE, PM, Privert, MOCC; Mayor Carrefour, Mayors North) PM: fast, efficient, timely, collaborative MOCC: TA, on target Allowed GOH to function PM: GOH stake in Northern corridor strategy -Chief of Staff to Pres: “Every govt agency that benefited from OTI was pleased.” -former IHRC: PM: difference with other donors -Missed: program could not provide support to military; working space for Council of Ministers (Australians supposed to do it, but cancelled w/o notice) -Missed: UTE would have like -In early stages, -Some mayors collaborated better than others. (ex. Carrefour) -PM Bellerive was impressed at the level of coordination with OTI and the access to resources they gave him: “The team’s availability was something to be valued; I -For GOH, CIP was in the works since 2009 (negotiations w/ Korean company started early 2009) HRI support improved GOH perceptions of USG & relations -TR: community liaison became mayor & is widely perceived as a hard worker, open, & very good at communicatin -HRI 1 infrastr: change in administration caused difficulties as temp bldgs’ users could not change plans -TdN Casecs & Asecs: program did not work in most sections, so unaware & -UTE – some messages crafted for billboards were not understood by all in different locations. CIAT: AIA “Big ‘F’ for the project, the people handling the funds in Chemonics didn’t understand its nature, urban planning. There was no room for Chemonics to tell the company what to do… I wasn’t allowed to be at the AIA debrief. The consultant viewed himself as doing HRI 1 -Chief of Staff to Pres on streamlining effects of temporary bldgs.: “I had a hard time convincing my people that the temp bldg. was not meant to functions as before… everybody was involved in thinking how the space would -CIAT: AIA grant generated massive amounts of data that would have required periodic meetings & a team, rather than 1 person -STTA: for UTE 1 consultant never really integrated 76 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned GOH national & local (CIAT, UTE, PM, Privert, MOCC; Mayor Carrefour, Mayors North “OTI’s were the only really flexible funds” “OTI & Chemonics were very amenable & looked for ways to make things work.” to have worked with video medium for communications strategy. knew I could call them on a Saturday night if I needed to. On a Sunday evening, I was visiting a friend and I ran into one of the OTI team looking at the remains of government buildings.”- UTE: held weekly meetings to review campaign messages for several months (w/ OTI, Chemonics, firm) g with other officials & the population (frequently uses the radio station). -TR: local authorities value community engagement on prioritization of projects, plan to do it in the future -TR: soccer field facilitated integration into regional tournaments -General: rehab of public offices improved work ethic among officials (TR mun, courts) - Competitiven ess between communes on the appearance & quality of infrastructure uninvolved. -TR: prioritized a beach, but weren’t told why it could not be done. Nor were they informed of the results of a study of water quality at the same beach. the work for Chemonics & USG, not as it was, a grant to the GOH…. Materials were not what was requested.” -GOH requested support for regional plan for the N via State, IDB promoted AIA -UTE:community feedback on CIP communication campaign -Most – resources that allowed gov’t to function, like office space and facilities, communication center. -Least: St. Marc mayor jatropha be used differently.” Group of less useful staff members went abroad after the eq. -TdN: mayor not very involved TR: mayors & C/Asecs engaged with program & municipality TR: local authorities perceive the community trusts them more as a result of the projects into the team, the other one did. Perhaps a function of UTE role and clear definition of expectation s. -UTE: communica tion campaign allowed us to understand the need to communica te what we do, not just do CBOs & other partner organizations CBOs: CBOs: implementation difficulties & changes: IOM: flexible, innovative, -Mart Platf: work with handicapped people -CSL training allowed 4/10 organizations in IOM: few N: negotiations with FTF for 500 parboilers CBO Limon: “if USAID does not come to your area, you are not going to get HRI 1: -Post-eq scenario complete chaos, social, political, & security Solar lamps, plazas CBOs: training TdN computer lab: summer courses for 120 youth & adults Outcomes: prog had expected that going through govt entities as grantees would engage officials -OTI used to be more bureaucratic, now are much closer, meet with them frequently -IOM collaborati on on camp clearance was success and 77 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned CBOs & other partner organizations actively engaged, allowed them to do something they could never have been able to do on their own IDB: were pushed to do things much faster than is normal for us, but may have been the only way to get them done. Must now patch up gaps. Training, comms campaign FG to write their own proposals (before had to go through a larger org because could not do it on their own). issues with IP on mgmt Excellent coordination development. ” -Ag associations organizational training: some were divided as a result of trained youth demanding a say, modernizing management. Became more open-minded, developed team spirit. systems completely collapsed -Training raised expectations -Activities addressed priority needs id. by the community, but problems in implementatio n impeded access to the benefits (ex., CEBA, vocational schools CEBA & Mart, library CEBA) - Implementatio n speed did not allow for changes on infrastructure projects -MP: insufficient info to members, perception that the 10 keep the best things for themselves. Quotas per organization were not respected. -MP: claims to have trained non-member for 3 months and build up relationships. From CBOs’ perspective, in PaP this did not happen. In the North, it varied by commune & personality of mayor. -Comites de Gestion: widespread participation & interest in North -Mart Platform: CSL trainees perceived MP as a political force, allowing a common vision, higher visibility in the community. -MP CSL trainees: w/o projects, there is no Platform On IDP returns, mid-2011 SRS sets focus on returns (+more responsive & effective governance) replicated by others after first collaborati on with OTI 78 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned CBOs & other partner organizations orgs did not hold up (only 1 of 6 orgs had received training from the MP). Beneficiaries -Many beneficiaries were impressed with speed of service and implementation. Some wondered why USAID didn’t adopt this model before. -CBO training was seen by participants as foundation for getting future development projects – need to manage expectations Training for organizations sometimes caused internal strife as youth demanded more say (ex. Scagitech) -Youth at computer lab: increased thirst for knowledge; school exam scores higher (87% passed) -Indepco training: life skills component greater impact than foreseen, both young trainees and adult trainers from other centers, stressed how training changed their approach to personal life, emphasizing “discipline,” “planning,” & “structure.” -Pervasiveness of unemploymen t: former trainees -Emphasis on soccer fields not meeting international standards. -Required greater supervision by program of construction firms (Cap) -Solar lamps not at sites identified with community leaders (TdN) -Installed equipment not working properly Ex. radio tower in Terrier Rouge delay in fixing - Moto safety seemed to catch on more in Cap than St. Marc. Ex. Lots of observed moto taxis with orange vests, nearly none in St. Marc. -Solar lamps effects on safety, schoolchildren’s academic performance, leisure, women, small merchants mentioned in all 3 communes Plazas CFW -Scagitech training, despite raised expectations. Valued business training, technical ag components, & legalization of organizations’ papers (including finalizing internal statutes & clarifying members’ roles & responsibilities). -INDEPCO training: behavioral training highly regarded, impact cited on professional & personal lives -Computer lab: summer sessions for 120, thirst for knowledge – Cap: road safety campaign decreased accidents (not quantified), confirmed by Justinian Hospital directors; high demand for training of more drivers in Cap & from other communes -TdN in general negative perception of relationship with mayor -TR: generally a very positive perception of local authorities, reports coincide on frequent meetings, prioritization CFW: -Some of the youth trained to use heavy equipment were employed in direct implementation grants in PaP -High level of involvement by Cap Drivers’ Union facilitated the more holistic road safety campaign (coord w/ PNH, identification, gathering points, TOT, limited number of helmets, radio messages) -Most CBOs expressed satisfaction in training received on improving organization and planning. (ex. Martissant Platform, Limonade CBOs Ensure communica tion system in place for full information flows throughout implementa tion (for participator y processes, not enough to participate at the beginning) Ex. location of solar lamps consulted - Unexpecte d emphasis on the jobs created. -CBOs recommen d the MP set up sub￾committees & put out more info 79 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Beneficiaries -TR: no one observed any negative effects -MP: platform brought together Fontamara & Martissant -In North, people are staying in their communities more now that they are seeing improvement s during school term, only 1 of 20 students had actually worked on a computer before. Least: RECOPAN Other members of the community Unanticipated rivalries between communities (N) Martissant: exclusion initial phase Generally a rosier picture, saw outputs. Others (former consultant to IHRC, former consultant to MOCC, IDB representativ e for CIP) -Needed more advocacy to avoid the unfortunate gap when the IHRC fell apart (w/o the gradual transition to GOH responsibility) -MOCC as a result of training: “this was one of the more successful campaigns I’ve seen in saving lives through -Coordination w/ IDB on AIA grant -Former IHRC consultant: “at a certain level the work plans for some sectors developed by IHRC consultants were taken up later.” -former IHRC consultant: “the understanding of the IHRC mission among the general public & the Executive. We didn’t do enough by way of explanations of the IHRC’s mission & structure to support 80 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Others (former consultant to IHRC, former consultant to MOCC, IDB representativ e for CIP) information. We used TOT, trucks with speakers, SMS, posters. There was no other way of combating a cholera crisis other than through information. As lethal as cholera is, it is curable if you do the right things early on.” recovery in the short & long terms.” Structure was “meant to establish good communicatio n channels between donors & GOH (Executive) so projects were implemented quickly, following a strategy and a plan. It worked at a certain level, but it could have been better. There was a lot of pressure to do something quickly to show the govt was doing something.” Eval team observations - Turnover on OTI & IP sides -Fast: CFW January to June 2010: 61 grants for $ 3,613,425. Total cubic meters of rubble removed: 11,495,016. -Widespread perception among staff that before Mission statement in -After-action review of emergency response, including CFW -DB includes little information on lessons learned and on difficulties throughout implementation. -Aside from support for CIP, not much guidance for program strategy in the North corridor. Using bottom-up approach carved out a niche in the 6 communes: “promoting the perception that the whole region can benefit from -Types of activities raised the standards -HRI 1 infrastr: program views GOH changes to temp bldgs. as detrimental in that these voided the warranty -Engaging GOH, though prog followed “principles of eng” defined at beginning (focus on key Min, support GOH outreach, promote GOH lea role) -Short-term nature of interventions -HRI 2 PaP: although the mission statement refers to improved relationship between the community & govt, signing the grant agreement was the extent of govt involvement for the most part. -In the North, -In the North, program staff concurred on identifying the most valuable activities as infrastructure grants and the least as arts & festivals. The former are also perceived as the most appropriate for “laying foundation for -Enabling GOH to function: all three branches -Initially “employ y most likely to turn to violence,” comm selection of participants). Emergency media & comms support.- Increased sense of inclusion in the region, -Beneficiary selection: devolve to community -Coupling training w/ CFW allowed program, better build up to jobs RECOPAN case as ex. of the need for full disclosure on implementa tion challenges and changes, same for Mart Platf -Project design & 81 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Eval team observations North, their role was to just implement projects, not develop them -Resettlement alternatives -Community liaison, platforms both critical tools in the different contexts -Gap-filling function of OTI programs in places where a political transition is not the main focus: consider what type of gap, how, to what ends. Avoid trying to make it look like a traditional OTI￾like program. -Self-censorship by OTI team during the time of Mission approval -Program flexibility in that community engagement took 3 different approaches: M platform, Bel Air, and the community liaisons... And there was an evolution on -TA to GOH & window of opportunity for governance reform TA grants in HRI 2: PAP029, 038, 043, 044, 093 - Team noted: Needed more OTI staff if wanted to do more TA to GOH. -No physical time to take a step back (related to high funding levels & pressure for speedy implementation) . development.” That the process was community￾driven re￾energized staff who facilitated it. - INL, Livelihoods & ag -Health was different, had a clearer understanding of where they would be -CIP: training, (esp. Parliament), but it was at least partially GOH taking charge and making changes it saw as necessary, currently responsibly maintaining - Competitiven ess among CBOs, communes challenged by absence of follow-on by host government and CBOs, difficult in PaP to achieve full community ownership mixed results on improving the relationship between local authorities & constituencies. TR sharp contrast w/ TdN. Limonade in between. However, staff & particularly the community liaisons perceived much more by way of governance outcomes than was actually the case. longer-term development. -For the period immediately after the eq, in PaP, staff, USAID, & GOH found arts & festivals of critical important as a respite from the horror and as a return to a sense of normalcy. -It would appear that some in the SMT would have wanted to do more of these “more OTI-like” activities, but the mix appears to have worked out well & was appropriate. -Partners’ meeting in Cap -Ag projects: tiller, warehouse, credit for seeds for peanut farmers; steamers for women rice farmers (about to be replicated by FTF); pride in their community. Thirst for knowledge (ex. comp lab) Integration into modernity (Indepco) -pockets of freedom to define what OTI wants to do were some of the high value (ex IOM relocation) - mgmt. training inevitably raises expectation s -M&E: definition of “direct” & “indirect” beneficiarie s must be agreed upon by IPs & communica ted to staff (ex. DAI256 direct beneficiarie s are CHF staff, as well as the 786 families receiving host family direct assistance) -LL on communica tions, informing the communiti es on what will happen, get their input, test materials for comms campaigns; role of 82 Evaluation Questions Stakeholders Fast, Flexible, purposeful Opportunities (missed/taken advantage of) Coordination (USG, GOH, other donors) Lay groundwork for long-term investments Unexpected effects Challenges Geographic targets (different results) Activities most/least valued Goals & objectives Organizational learning Lessons Learned Eval team observations community engagement in the last stretch, no? Starting with the community profiles and discussions around the Mission statement, the program carved out room for a more strategic approach “Purposeful” poses questions as to the program strategy. Although no one disputes that the entry points on target, most interviewees mentioned the absence of a clear strategy as a main challenge. community liaisons -Gap-filling function -Internews: not handed over from start-up team to IP, understand logic, objectives, approach, and have decision￾making authority. Eval: scant integration w/ other program activities. -Staff in the North evaluated more positively the comm. engagemen t and infrastructu re grants than activities with which the program started, which were more top￾down. 83 ANNEX E - REFERENCES Publications Corvington, G. (1987). “Port-au-Prince au cours des ans. La Capitale d’Haiti sous l’Occupation. 1922- 1934.” Port-au-Prince: Imprimerie Henri Deschamps. Deshommes, F. (2004). Décentralisation et Collectivités Territoriales en Haïti. Un Etat des Lieux. Cahier No 1. Port au Prince : Editions Cahiers Universitaires. Durkheim, E. (1933). The Division of Labor in Society. Glencoe, IL: The Free Press. Farmer, P. Haiti after the Earthquake. New York: Public Affairs, 2011. Jadotte, H. and Pierre, Y. F. (2008). “Local Governance and Sustainable Peace: The Haitian Case.” In The Paradoxes of Peacebuilding post -9/11, ed. Baranyi, S., pp. 85-93. Vancouver, British Columbia: UBC Press. Katz, J. The Big Truck that Went By. How the World Came to Save Haiti and Left Behind a Disaster. New York: Palgrave MacMillan, 2013. Latin American Public Opinion Project (2006). Political Culture of Democracy in Haiti. Latin American Public Opinion Project (2012). The Political Culture of Democracy in Haiti and in the Americas: Towards Equality of Opportunity. Maguire, R. “Goodbye to ‘pixie dust’? A new US development strategy in Haiti.” Journal of Peacebuilding and Development, Vol. 6, No. 3, 2011. Mandel, J. and Sommerfeldt, E. “Closing the Loop – Responding to People’s Information Needs from Crisis Response to Recovery to Development. A Case Study of Post-earthquake Haiti.” Internews, 2013. Marcelin, L. H. (2011). “Cooperation, Peace and (Re-)construction? A Tale from the Shanties.” Journal of Peacebuilding and Development. Vol 6, No. 3. Qfid M.G. (2012). “Evaluation Plan HIS Martissant”. Port-au-Prince, Haiti: USAID. Rouzier, S. (Year not indicated). “Dictionnaire Géographique et Administrative Universel d’Haiti. Tomes 1 et 2.”. Paris: Imprimerie Charles Blot. Shamsie, Y. (2011). “Pro-Poor Economic Development Aid to Haiti: Unintended effects arising from the conflict-development nexus.” Journal of Peacebuilding and Development. Vol 6, No. 3. Théodat, JM (2003). Haiti République Dominicaine. Une île pour deux. Paris: Editions Karthala. Other documents Government Accountability Office. “Haiti Reconstruction. USAID Infrastructure Projects Have Had Mixed Results and Face Sustainability Challenges. Report to Congressional Requesters.” June, 2013. “Haiti: the Slow Road to Reconstruction.” Oxfam Briefing Note, January, 2012. “Independent Review of the US Government Response to the Haiti Earthquake. Final Report.” March, 2011. Kolbe, A. (2013). “Revisiting Haiti’s Gangs and Organized Violence.” Discussion Paper 5. Humanitarian Action in Situations Other than War (HASOW). Office of Inspector General (2010). “Audit of USAID’s cash-for-work activities in Haiti.” Audit Report No. 1-521-10-009-P. Office of Inspector General (2012). “Audit of USAID’s Haiti Recovery Initiative activities managed by Office of Transition Initiatives.” Audit Report No. 1-521-12-009-P. Smucker. G., et al. (2000). “Political Will for Decentralization in Haiti” (Project Report for Development Alternatives 84 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523