Promoting Stabilization and Community Reintegration Project (PSCRP) - DRC PERFORMANCE EVALUATION REPORT This document is the final version and is dated 22 October, 2011. The contents of this publication may be reproduced in whole or in part without permission provided: (a) the material is distributed free of charge; (b) credit is given to MSI and MSIs funding source (where mentioned), and; (c) this restrictive legend is included in full. The findings, opinions and views expressed in this publication do not necessarily represent those of Management Systems International (MSI) or any MSI Client. 2 | Page Table of Contents Table of Contents .......................................................................................................................................... 2 Acronyms ...................................................................................................................................................... 3 Executive Summary ...................................................................................................................................... 4 Introduction and background ........................................................................................................................ 6 Purpose of evaluation and evaluation’s methods .......................................................................................... 8 Findings, Lessons Learnt, Conclusions, and Recommendations ................................................................ 10 Overall analysis on key normative evaluation dimensions ..................................................................... 10 Contribution to community stabilization and recovery ........................................................................... 11 Contribution to the eastern DRC stabilization agenda ............................................................................ 26 Lessons Learnt / Good Practices ............................................................................................................. 30 Annexes ...................................................................................................................................................... 35 1. Scope of Work of the evaluation ..................................................................................................... 35 2. Changes to SoW and rationale ........................................................................................................ 42 3. Annex to evaluation of Impact ........................................................................................................ 43 4. Annex to Question 1: Comparison of select key strategic documents ............................................ 44 5. Annex for Question 3: Evolution of schooling levels of marginalized groups ............................... 48 6. Annex to Question 4: Effectiveness per Result............................................................................... 49 7. Annex to Question 6: Changes in social dynamics and community cohesion ................................ 53 8. Annex to Question 8 ....................................................................................................................... 54 9. Annex to Question 14: Good practices & lessons learnt on community recovery.......................... 61 10. Evaluation design matrix ............................................................................................................ 65 11. Meetings held .............................................................................................................................. 70 12. Bibliography/sources of information .......................................................................................... 71 13. Glossary of terms ........................................................................................................................ 74 14. Suggestions for future success story write-ups ........................................................................... 75 15. Copy of evaluation tools ............................................................................................................. 76 Table of illustrations Figure 1: Theory of change/Results Framework of the project, adapted from PMP .................................... 6 Figure 2: Map of PSCRP community groups in North Kivu ...................................................................... 41 Figure 3: Map of PSCRP community groups in South Kivu ...................................................................... 41 Table 1: List of beneficiaries and stakeholders according to Task Order ................................................... 14 Table 2: Proportional contribution of key community processes to Results' achievement ......................... 19 3 | Page Acronyms ARD Associates in Rural Development Care Cooperative for Aid and Relief Everywhere CDC community Development Committee CDD Community-Driven Development CLD Comité Local de Développement CLPC Comité Local Permanent de Conciliation CMM Conflict Management and Mitigation CODESA COmité de DEveloppement de l’aire de SAnté CoP Chief of Party COPA COmité des PArents CRLP Community Recovery and Livelihoods Project CRS Catholic Relief Services DCoP Deputy CoP DDR Disarmament, Demobilization, Reintegration DFID (UK) Department For International Development DRC Democratic Republic of Congo ECZ Equipe Cadre de la Zone de santé FARDC Forces Armées de la RDC FY Fiscal Year GBV Gender-Based Violence HIV/AIDS Human Immunodeficiency Virus / Acquired Immune Deficiency Syndrome HQ Headquarters I4S/ I-SSSS International Security and Stabilization Support Strategy IA International Alert IDP Internally Displaced Person IGA Income Generating Activity INGO International Non-governmental Organization IRC International Rescue Committee KAP Knowledge, Attitudes, Practices LNGO Local Non-governmental Organization MnE Monitoring and Evaluation MONUC Mission de l’Organisation des Nations Unies au Congo MONUSCO Mission de l’Organisation des Nations Unies pour la Stabilisation en République démocratique du Congo MSI Management Systems International NGO Non-Governmental organization NRC Norwegian Refugee Council NSP National Solidarity Program OTI Office of Transition Initiatives PAP Plan d’Actions Prioritaires PEAR+ Program of Expanded Assistance to Returns PLD Pain Pour les Déshérités PMP Performance Management Plan PSCRP Promoting Stabilization and Community Reintegration Project RFA Request For Application SoW Scope of Work STAREC Programme de STAbilisation et de REConstruction des zones sortant des conflits armés UN United Nations UNDP United Nations Development Program UNICEF United Nations Children’s Fund UNOPS United Nations Office for Project Services USAID United States Agency for International Development USD United States Dollar WFP World Food Program 4 | Page Executive Summary Purpose and background of the project PSCRP’s purpose was to “Enhance reconciliation and recovery in targeted communities in Eastern DRC”. The project is implemented by MSI in partnership with International Alert over 26 months from October 2009 until November 2011. PSCRP is a community reconciliation and recovery project using a community-driven development (CDD) 1 methodology, with a cross-cutting conflict-sensitive approach. It supports 302 Main evaluation questions communities in Eastern DRC, totaling an estimated 280,000 inhabitants. The project is fully funded by USAID. The project was designed and implemented in the context of a new paradigm of “stabilization” in Eastern DRC, initiated by the UN, and to which USAID wanted to contribute through, amongst other, this project. 3 1. To what extent were the four intended results achieved, in comparison to project indicators? 2. Did the project contribute overall to reconciliation and conflict transformation in communities? 3. Was the project coherent with the community-related components of the Eastern DRC stabilization frameworks? 4. Was the combination of reconciliation with recovery an effective strategy for stabilization? 5. Are there areas where PSCRP can be deemed to have reached good practices? 6. Identify practices drawn from the project’s experience of community processes that contribute to reconciliation and reintegration, and make recommendations for future programming. Evaluation methods The evaluation was conducted from September 19, 2011 to October 8, 2011. Given the very limited resources (time and people) available to conduct the evaluation, secondary sources were prioritized whenever possible, including document review and secondary stakeholders’ interviews as well as key project staff. Key findings and conclusions 1. Based on project indicators, PSCRP has globally achieved expected targets under each of its Results. More nuance in indicators and more attention on data collection could however have been ensured during the design and implementation of the project; 2. Evaluating the contribution of the project to reconciliation and conflict transformation is highly dependent on complex qualitative data and on what conceptual framework is chosen to perform this analysis. This being said, PSCRP appears to have address successfully some dimensions of reconciliation and conflict transformation, and to show potential to address other. The short duration of the project renders both the achievement of changes in this field as well their measurement difficult; 3. The Eastern DRC stabilization frameworks remain very informal, fluid processes when it comes to programming approaches insofar that there are no clear, formal, detailed official documents that describe what the programming approaches of the I-SSSS or STAREC are or should be, and as their own staff recognize that their approaches have evolved significantly over time. Based on the information obtained by the evaluators which is mostly based on the perceptions of senior UN and 1 See Glossary in Annex 13. 2 Out of those 30 communities, the full scope of the project was implemented in 29. The 30th community received a more limited support under Result 1 only. This was agreed upon in coordination with USAID. IA was not involved in this 29th community. For this reason, the report refers alternatively to 29 or 30 communities as relevant. 3 Questions’ language in this executive summary has been slightly adapted for ease of reading. 5 | Page government stabilization personnel, PSCRP appears to have been highly coherent with those frameworks, and to have positively supported their advancement and evolution; 4. There is a general sense throughout the evaluation questions that combining socio-economic recovery with reconciliation components has a mutually reinforcing advantage. Given that those two dimensions are key to notion of stabilization as described by USAID in the project Task Order, one can conclude that combination of those two components is probably more effective for stabilization than the implementation of more traditional, separate projects; 5. PSCRP has followed or generated a number of good practices in terms of socio-economic recovery, of conflict prevention and management, and when it comes to combining the two. Much however remains fairly informal insofar that these practices are not necessarily documented in writing in an official, easily accessible project implementation manual or any such document. Key recommendations 1. Monitoring frameworks of such project as well as their implementation could be improved in a way that allows to better track – and then evaluate – achievements; 2. The way USAID expects projects to contribute to broader frameworks (the UN/Government stabilization frameworks in this instance) should probably be better formalized. For example, the Task Order of such projects could specify what contributions USAID expects given projects to deliver over its course. In PSCRP, the robust contribution delivered seems to rely essentially on individual leadership by MSI’s CoP; 3. PSCRP should produce a comprehensive project implementation manual compiling all of the tools and processes used during implementation, as well as recommendations for improvements, so as to maximize the chances that its good practices and lessons learned can be passed along to future similar projects. Key lessons learned 1. A 2-year duration is insufficient for such project to deliver sustainable socio-economic recovery and conflict prevention and management outcomes. The project has achieved its expected Results but more time would be needed to solidify and institutionalize those; 2. Sustainability of outputs and outcomes appears to be determined primarily by ownership of community process by internal and external stakeholders, as well by external factors. This substantiates the need for truly comprehensive approaches to stabilization, including in the context of Eastern DRC, where (a) socio-economic recovery, governance and conflict prevention and management interventions need to be foreseen on the long-run and where (b) security/military and political/diplomatic efforts need to be maintain along development actions. 6 | Page Introduction and background Description of the project Funded by USAID, PSCRP is implemented by MSI in partnership with International Alert over 26 months (extended from 24 months) from October 2009 until November 2011. PSCRP is a community reconciliation and recovery project using a community-driven development (CDD) methodology, with a cross-cutting conflict-sensitive approach. It supports 304 communities in Eastern DRC, totaling an estimated 280,000 inhabitants at the time of project start. PSCRP’s results framework is presented below. The Task Order and the other reference documents provided to the evaluators explicitly indicate neither the problem (or opportunity) the project was trying to address nor the assumptions underlying the project. The evaluators’ understanding is however that the project was mainly trying to seize the opportunity of the at-the-time recent political, security and institutional progresses in Eastern DRC to contribute to sustaining stability. The assumptions underlying the strategy is that delivering socio-economic outcomes combined with conflict mediation services in a way that seeks to enhance local governance and social cohesion will increase the likelihood of sustained stability (adapted from Task Order language). The project is fully funded by USAID; the implementing partners are not providing any match funds. USAID’s involvement in the project is not determined in the Task Order as provided to the evaluators. This involvement has been described as follows by the CoP: (1) request for “hand-in-glove” coordination with USAID; (2) approval by USAID of key personnel, Workplan, PMP, targeted communities, of all grants and sub-grants and of communications and training materials; and (3) approval of or requirements for grants under the rapid response mechanism. Theory of change of the project The Project is intended to contribute to USAID/DRC’s goal of “Increasing stability by mitigating the causes and consequences of conflict in the country”, including its focal area of “Promote conflict prevention, mitigation and resolution, as well as stabilization and recovery, with an initial focus on the on-going tensions in Eastern DRC”. The project is also expected by USAID to contribute to I-SSSS’s component 4 (Facilitating return, reintegration & recovery) and STAREC’s component 25 (Humanitarian and social component). The project also included a side additional Result on ‘rapid response mechanism’. Although this mechanism was used twice during the course of the project, it was not mentioned noted in the SoW of this evaluation and is thus not covered in this report. 4 Out of those 30 communities, the full scope of the project was implemented in 29. The 30th community received a more limited support under Result 1 only. This was agreed upon in coordination with USAID. IA was not involved in this 29th community. For this reason, the report refers alternatively to 29 or 30 communities as relevant. 5 The project falls under Component 2, contrary to what is stated in the evaluation SoW. Purpose: Enhance reconciliation and recovery in targeted communities in Eastern DRC Result 1: Increased social and economic peace dividends in targeted communities Result 2: Strengthened social mechanisms for community reconciliation, conflict mediation and prevention capacity Result 3: Enhanced participatory local governance processes with community and civil society organizations Result 4: Increased reintegration of marginalized people into targeted communities Figure 1: Theory of change/Results Framework of the project, adapted from PMP 7 | Page Project implementation location The project is implemented in a total of 30 communities in 5 areas of North and South Kivu in the Democratic Republic of the Congo. Those communities are clustered in a total of 15 groups. Maps displaying those groups are presented within the SoW in Annex 1. See footnote on previous page for clarifications on the fact that this report sometimes makes reference to 30 communities and sometimes to 29 communities. Key global and local contextual elements Key global trends The project was initiated and has been implemented in a context where the notion of “stabilization”, although understood differently from one context to another, has been increasingly referenced in security and aid environments. Furthermore, the project uses a CDD approach, a modality that has been increasingly used in post-conflict and other fragile settings in developing countries. Key contextual elements in Eastern DRC The Task Order makes reference to the following key contextual elements: the protracted insecurity in Eastern DRC from armed groups, the complex governance framework and gap, the challenges faced by civil society, the new (at the time) peace process for Eastern DRC, population movements, DDR, the new stabilization framework comprising the I-SSSS and STAREC, the then expected 2010 local elections. As the project is nearing its end, the same key contextual elements certainly remain relevant, with the addition of upcoming presidential elections expected at the time of the evaluation to take place on November 28, 2011, and the evolution of the national and international regulatory framework around mining. Local elections did not take place; neither did the increasing of the number of provinces or an effective delegation of budgets and power to provinces and chefferies/secteurs despite the fact that those evolutions are mandated by the Constitution. In addition, refugee return over 2009-2011 was significantly slower than anticipated. Critical other projects and programs International Rescue Committee’s (IRC) “Tuungane” project corresponds closely with the scope of PSCRP’s Result 1. This project funded by DFID since 2007 builds upon earlier IRC projects funded by USAID (OTI and CMM) and uses a CDD approach. Amongst others, it supports communities in North and South Kivu. Tuungane started first in South Kivu, while PSCRP started first in North Kivu. MSI and IRC coordinated community selection. IRC was also implementing another similar project working through local partners, under the name ‘Tushiriki’. USAID was also financing until late FY11 CRS for a project in South Kivu comprising activities comparable to Result 2 of PSCRP. The Task Order makes reference to a number of other initiatives, including the USAID-funded, ARD-implemented Durable peace and Stability in Eastern DRC Project, that started before PSCRP but was terminated early by USAID, as well as UNICEF’s program for assistance to returnees (PEAR+). Priority evaluation questions The evaluation SoW did not differentiate between priority and secondary questions. The project’s CoP and DCoP agreed on the following priority questions (formulation summarized below): 1. To what extent were the four intended results achieved, in comparison to project indicators? 2. Did the project contribute overall to reconciliation and conflict transformation in communities? 3. Was the project coherent with the community-related components of I-SSSS and STAREC? 4. Was the combination of reconciliation with recovery an effective strategy for stabilization? 5. Are there areas where PSCRP can be deemed to have reached good practices? 6. Identify practices drawn from the project’s experience of community processes that contribute to reconciliation and reintegration, and make recommendations for future programming. 8 | Page Purpose of evaluation and evaluation’s methods Intended use of the evaluation report and intended users The purpose of the evaluation as stated by USAID to MSI was to inform USAID for upcoming programming. Specifically, for each result USAID wanted to see how well it was achieved, the specific approaches that worked particularly well, and the challenges and difficulties that were encountered in the course of pursuing each result. Specific lessons learned and key recommendations surrounding each result were also expected. MSI’s understanding of intended users is that USAID DRC will be the primary user of the evaluation report and may be shared with other USG staff as designated by USAID. Evaluation Methods The SoW of the evaluation is presented as Annex 1. The evaluation was conducted from September 19, 2011 to October 8, 2011. This was followed by a presentation of initial findings to USAID on October 10, 2011 and by a part-time period of report writing, including several feedback phases, through October 24. The evaluation covers the project time span from its origin, October 1, 2009 until September 30, 2011. USAID-funded budget for the evaluation was estimated by MSI at USD 60,000. No other sources provided funding for the evaluation. The detailed evaluation methodology and key evaluation tools are presented in Annex 15. Data collection methods Given the very limited resources (time and people) available to conduct the evaluation, secondary sources were prioritized whenever possible, including document review and secondary stakeholders’ interviews as well as key project staff. A purposive sample of four communities6 Description of the evaluation’s design was selected for field visits, during which semi-directive group interviews were held. A 1-day workshop was held with key project staff to fine-tune analysis from the above processes and to attempt to elicit whether field findings could be extrapolated to the entire project. Sound social science methods were nevertheless used to the best extent possible, especially in terms of how communities were sampled, interviews prepared and held, and data extrapolated. Given the limited time and resources available for the evaluation, the evaluators were not able to engage in-country partners and beneficiaries or national counterparts in the design and conduct of the evaluation. All those stakeholders were however interviewed by the evaluators over the course of the evaluation. The project CoP and DCoP were extensively involved in reviewing and approving the detailed methodology proposed by the evaluators, together with the MSI home office conflict evaluation specialist. MSI approved the methodology before data collection started. Extensive data was made available to the evaluators by the project’s team, however a lot of data and documents were in a row form that was not easily analyzable by the evaluators given the limited time and resources available. The SoW was slightly adjusted early on in the process to accommodate delays in deploying one of the evaluators due to flight issues. The wording of evaluation question number 1 was slightly modified due to discrepancies between various documents. No other modifications were made to the SoW. Evaluation limitations The main limitations that the evaluation faced were the following: 1. Evaluators accepted documents from the project team until October 5, 2011. Unfortunately, up until that date some of the project wrap-up documents were not yet finalized; 6 See annex 15 for explanation of selection process. 9 | Page 2. Limited time available: Excluding initial preparation, international travel, preparation and delivery of draft reports and presentations, and final report writing, the consultants were allowed only 8 days each (16 days total) for evaluation fieldwork; 3. Due to the above time constraint, evaluators were able to visit only a very limited number of communities and thus had to rely primarily on secondary sources and extrapolation; 4. The incumbent of the DCoP changed in August 2011; the new incumbent at the time of the evaluation had thus only limited direct knowledge of the project implementation; 5. Due to the absence of randomized control and treatment groups in the project, none of the findings of the evaluation can strictly speaking be attributed. This is however not a formal impact evaluation and thus such methodology was not expected, as per USAID’s evaluation guidelines. Evaluation team The evaluation team was composed of two external consultants, Alexandra Vasseur contracted by International Alert with a remit to look specifically at the community reconciliation and integration components of the project and the peace-building lessons its experience suggests, and Xavier Bardou contracted by MSI as team leader. No conflicts of interest were identified; it is however important to note that the consultant contracted by MSI had been separately contracted in 2009 by MSI to produce three guidance notes that MSI later used as reference material to write its technical offer for the project. This same consultant also provided advice to MSI to feed into the design of its CRLP offer. The team of evaluation consultants combines extensive experience of each of the project components’ subject matters (6 years of direct CDD experience, 5 years of community reconciliation experience, 11 years of local governance experience and 5 years of reintegration programming experience) with a total of 9 years of full-time and part-time work on and in DRC. Furthermore, both evaluators have previously been involved in the evaluation of similar projects, and both have prior experience working with USAID. There are no significant unresolved differences of opinion on the part of the stakeholders involved in this evaluation. 10 | Page Findings, Lessons Learnt, Conclusions, and Recommendations As required in the SoW, this section is organized around the TORs key issues and to the extent possible includes findings, conclusions and recommendations for each. Overall analysis on key normative evaluation dimensions The 3 normative questions listed in the evaluation SoW very much cut across a number of the specific questions that are further discussed below. This section offers an overview of the evaluators’ conclusions that are relevant to each of those normative dimensions. Relevance As further described below, the project appears to be broadly relevant from three intertwined perspectives: 1. To the Task Order and the stabilization agenda in Eastern DRC, a critical expectation of USAID (see questions 1, 10 and 11); 2. To the security and policy environment (see question 7); 3. To good practices in similar projects (see questions 14 to 16). Given the above, the evaluators deem the project design and implementation highly relevant. Impact As stated above in the methodology considerations, there is no control group in this project, so strictly speaking changes towards the project’s indicators cannot be attributed to the project’s treatment. This being said, baseline and end-line results can be compared. The chart in Annex 3 suggests mixed changes in terms of impact. Under the first indicator, three clusters of communities showed improvement: one doubled its baseline figures, another increased by more than 25% and the third showed an increase of 7%. Clusters of communities where a decrease occurred were affected by a margin of no more than 15%. Under the second indicator all clusters of communities but one showed advancement; three of which were over a 20% increase. Further research would be needed to understand the dynamics that might explain this situation7 . Unfortunately, the evaluators were provided the impact-level data of the end-line only at the end of their data gathering and analysis process, and it was therefore not possible to incorporate questions on this in interview protocols, community visits and in the staff workshop8 . However, using the definitions and analyses suggested in the Task Order along with responses to the set of evaluation questions below, there are a number of clear pointers indicating that the project is contributing to USAID’s higher goal in DRC of ‘increasing stability by mitigating the causes and consequences of conflict in the country’ (see questions 10 and 11 below, on project contribution to the stabilization agenda in Eastern DRC). While this can be estimated through observation of changes at Results’ level and based on stabilization stakeholders’ feedback, the below data does not verify the causal link between the Results and Purpose, nor does it allow to measure the extent of changes at a purpose level, if any. Effectiveness Please refer to question 4 below which fully covers the project’s effectiveness. 7 See limitations stated in Annex 15 about the quality of data collection during baseline, where people’s perceptions might have been overstated based on their expectations of the project. 8 The evaluation was carried in months 23-24 of the project, which lasts 26 months in total, thus it is not abnormal that the project team processed its end-line data only at the time the evaluation took place. 11 | Page Contribution to community stabilization and recovery 1. Did the strategies in the original design meet the requirements of the Task Order? Did they need revising during the course of PSCRP? If yes, why? Analysis The evaluators reviewed key strategic documents (project Workplan in particular) as well as the Task Order. The former were compared with the latter, and then with implementation practices. See Annex 4. Findings 1. The project Workplan, approved by USAID, covers only a subset of the items presented in the Task Order. All core items present in the Task Order (community recovery and community reconciliation) are absolutely covered in the Workplan, however other secondary elements in the Task Order (linkages with local authorities and their capacity building, addressing a root cause of conflict such as land tenure, working with GBV survivors and other vulnerable groups) are not directly covered in the Workplan; 2. Explanations that justify this situation are as follows: o In between contract award and issuance of the Workplan, clarifications were discussed between MSI and USAID on specific items of the Task Order, especially the level of and approach for engagement with ex-combatants, and this was reflected in the Workplan; o The Workplan covered only Year 1 of the project, as required. Given the linear nature of the implementation of the core components of the project, it appears reasonable that most secondary activities would have been scheduled for Year 2 only. This idea is, however, not referenced in the Workplan and it does not seem that secondary activities were all significantly implemented in practice, even in Year 2; 3. For the core components of the Task Order, the Workplan is consistent. The Workplan provides a much deeper, detailed and justified description of project implementation, as expected. Both documents are globally consistent with what is generally considered good practices in community￾driven development projects; 4. The actual implementation of the project closely followed the Workplan, with only minor, non￾strategic changes; 5. It appears that secondary activities presented in the Task Order were perhaps too ambitious. Some have not been incorporated in the Workplan and have not been fully implemented in practice, even during Year 2. Some actions were hwoever undertaken in Year 2 against each broad secondary activity presented in the Task Order. The CoP indicated that USAID was regularly briefed on the project’s intended efforts towards each secondary activity and that what was implemented matched USAID’s expectations as discussed through those briefings. For example, the Task Order includes a broad activity of linking communities with local authorities and building the capacity of the latter. What the project planned and implemented to this end is a limited support to CLPCs, and this was agreed upon with USAID over the course of the project according to communications reviewed by the evaluators; 6. At the community level, the competencies attributed to committees have eventually questioned the ones of customary leaders (for example the handling of dowry conflicts). They did not manage to connect with the responsibilities and roles of provincial authorities in Priority Action Plans designing processes, and vice versa. 7. The Task Order provided for an expansion of the project to Ituri, should the project obligated amount be increased. While project funds increased, it was decided in conjunction with USAID that PSCRP would not move forward with geographic expansion. It is the evaluators’ opinion that an expansion to 12 | Page Ituri at the late time where project funds were increased would in any case not have been realistic given that the remaining time available was not sufficient to ensure appropriate implementation; 8. The Task Order includes very specific language about what the project is expected to undertake for very specific types of ex-combatants. This was not achieved as such by the project. The evaluators however reviewed communications between MSI and USAID held before the signature of the Task Order that clearly narrow and scale down USAID’s actual expectations of what the project should achieve towards ex-combatants. The implementation of the project appears to have been consistent with USAID’s expectations as stated in those communications. Conclusions Globally, the project as implemented is consistent with what was planned as per the Workplan, as well as with what was provided for in the Task Order and communications with USAID. Most secondary activities presented in the Task Order were translated into only very minor actions during project implementation, and minor variations were observed in between the Workplan and actual implementation. None of those changes affect the effectiveness or impact of the project to a significant level, to the extent that this can be estimated. Recommendations 1. Implementing agencies and USAID could strive to agree on more realistic offers/Tasks Orders. The number and extent of secondary activities in the offer and Task Order were not necessarily realistic given the time and resources available for the project. USAID should seek to plan for longer durations for projects with such a wide number of activities; 2. In multi-year projects, implementing agencies should be required to produce Workplans beyond the first year only; 3. The approach developed at community level to enhance the capacity of local authorities to consider and respond to community-identified recovery needs should now be extended up to the provincial level, in order to contribute to PAP as well as to institutionalization of peace building processes. This has been developed by PSCRP in coordination with UN-Habitat at the level of traditional local leaders on land conflicts management and could be extended to higher levels of the administrative pyramid in some areas. 2. Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? Analysis The evaluators reviewed relevant project documents and interviewed key MSI and IA staff. The evaluators agreed with MSI and IA to understand the word ‘objectives’ in the question as meaning ‘results’, given language used in the Results Framework and data available. Findings There is general consensus amongst internal stakeholders within the project team that there was a disconnect or lack of coherence between the sharing of roles and responsibilities and the project architecture. Key issues were as follows: 1. MSI was in charge of the recovery component and IA of the reconciliation component, which both require work at the community level, yet only IA had community-level staff. While IA was nominally in charge of all community-level facilitation, this architecture was ill-suited to the project and generated tension and frustration between staff at all levels; 2. The LNGO partners engaged in the project were in charge of the delivery of most of the reconciliation component, the component IA was in charge of. Yet, they were contracted by MSI. This created a host of accountability issues, where LNGO partners did not easily accept oversight by IA and where MSI and IA struggled to ensure the appropriate level of technical and quality control. In 13 | Page addition, this created at times duplications and at time gaps in the implementation of the reconciliation component between IA staff and LNGOs; 3. The perception of senior project staff is that recourse to LNGOs for the implementation of the reconciliation component was based on the understanding that those had the technical capacity to deliver the same. While most of the LNGO partners had a track record of delivering similar activities, including in some cases in the same communities as PSCRP and as former IA partners, their technical capacity was not at the initially expected level. The perceived added value of using those partners was thus at least changed, if not scaled back, over the course of the project. Additional activities and resources had to be mobilized so as to address this situation; 4. MSI and IA identities and cultures appear to have somewhat diverged during the implementation phase and their somewhat separate visions on how to best combine socio-economic recovery and peace building activities may have been a source of tension and frustration. In addition, the original staffing levels were insufficient. There was not enough mid-management in South Kivu and not enough oversight on partners. Some functions were expected to be covered by a single individual across both North and South Kivu (MnE, construction, grants) but this did not work out and had to be changed. In addition to quantities, the seniority/capacity level of some positions or functions was also underestimated. This was especially the case with MnE, where a relatively junior person was initially put in charge, who proved unable to deliver any MnE systems, a situation that hampered the current project for its first year. Conclusions The sharing of roles and responsibilities, and the overall project architecture were not well suited to the project’s results, as they did not allow for the best division of labor and the best support to communities under the various streams or results of the project. Efforts were made to address the shortcomings, but some challenges remained unaddressed until the end of the period covered by the evaluation. As per question 4 below, the project was nevertheless able to deliver against its expected Results. Recommendations In future projects, more coherence should be sought between division of labor and organizational structure. The actual level of technical and organizational capacity of LNGO partners should be assessed more realistically from the onset and partnerships should always be understood as capacity building partnerships first in contexts such as Eastern DRC. 3. Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? In particular, how effective was the project in responding to the needs of ex-combatants and other marginalized groups? Analysis In agreement with MSI, the evaluators treated this question at the level of the entire PSCRP project, and not only at the level of the individual community development subprojects. Considering quantitative data available, the issue of effectiveness of the project on marginalized groups has been assessed through the evolution of: 1. Social integration: The presence of girls, refugees, and ex-combatants in schools, in 5 communities from South Kivu9 2. Economic integration: The participation of women, Pygmies and ex-combatants in the implementation process of infrastructure community subprojects; (see Annex 5); 3. The potential impact on women of health related activities. 9 The 5 communities are Sange I, Sange II, Sange III, Mutarule I and Bwegera 14 | Page Further information on related activities and their impact on social cohesion and marginalized groups integration is presented below under Question 6. Findings List of beneficiaries from the Task Order List of stakeholders from the Task Order 1. Communities 2. Returnees 3. IDPs 4. ex-combatants, including child soldiers 5. marginalized 6. survivors of GBV 7. vulnerable 1. UNDP - MONUC – UNOPS – WFP 2. other agencies (ARD, NRC, IRC, Care International, Caritas, Save the Children) in Eastern DRC 3. other US-government agencies in Eastern DRC 4. provincially based local NGOs 5. local authorities (district or province) 6. committees Table 1: List of beneficiaries and stakeholders according to Task Order “Marginalized” has been mainly referring in PSCRP to Pygmies and other ethnic minorities. The latter differ from a community to another. 1. The enrollment of schoolgirls has increased in a higher proportion than the one of boys over the course of the project; 2. Although the Pygmies and children of ex-combatants did not see major change in their level of enrollment, their number has increased in absolute value; 3. The proportion of schooled returnees has been stable, with an increase of returnee girls enrolled; 4. One has to consider that enrollment of children is a progressive process throughout the first quarter of the school year; the figures available for school year 2011 were thus not definitive at the time this evaluation was conducted; 5. The participation of marginalized groups within peace committees shows very poor participation: from 0 to 13% in Uvira axis for example; 6. The composition of peace committees has eventually changed over the project period10: the participation of Pygmies has increased in some communities between March and July 2011 (generally from 0 to 2, with exceptions such as in Bwegera where 6 Pygmies were part of the peace committee early on – they did not leave the committee later but their important involvement from the onset means that there were no further increase in their number later). Considering specific needs of women, data made available allowed to measure the effect of sensitization and equipment of the health center of Kalunja (Fizi, South Kivu): July 2009 – October 2009 May 2011 – August 2011 Number of pre-natal consultations 98 593 7. One can estimate that the increase in pre-natal consultations is a direct result of the community subproject mainly because of poor equipment and conditions in the health center before: the population had previously to refer to Baraka, whose access was problematic both in terms of time and security; 8. One LNGO partner organized sensitization activities related to HIV/AIDS; 9. Peace committees have supported the process of referring survivors of sexual violence to medical and judiciary services; 10. GBV in the domain of economic integration of women has been targeted through sensitization of communities on inheritance law. 10 Available data did not allow any comparison over time for most of the targeted communities. 15 | Page Key stakeholders of the project are LNGO partners and, at community level, peace committees and CLDs. Other stakeholders can be identified: local authorities at community level, including traditional leaders, the administration, the police and the military. Conclusions11 1. The Pygmies have seen a positive evolution in the determination of the communities to interact with them; in some places, this process had positive effect on their economic integration (allocation of fields, access to employment within the administration); 2. However, the understanding of integration has been limited to this group; as an example, there is no data on other minority groups’ participation within committees to conclude on an improved consideration of their needs in decision-making processes; 3. Participation at schools, in committees, on construction sites may have increased, however some negative perceptions remain strongly anchored in the population with regards to ex-combatants, some ethnic groups (such as Banyamulenge) and specific social identities (such as pastoralists); 4. Survivors of GBV have been eventually targeted through sensitization activities (see Sange I and II). However the consideration given to the specific needs of those individuals has been linked, and limited, to competencies and skills available within LNGO partners; 5. Whereas the project addressed extensively social and economic integration/assimilation of Pygmies (see Question 6 below), the reintegration of ex-combatants, and the social and economic empowerment of women, it seems to have had difficulties to target GBV survivors as well as social stigmatization of unmarried mothers; 6. A constraint lies in the limit in competencies delegated to peace committees to address penal cases; this limit does not allow them to support survivors of rapes for the recognition of their status of survivors, the recognition and punishment of the crime, and compensation;12 7. As this evaluation is neither an institutional nor organizational evaluation, information related to stakeholders will be found throughout the report. Recommendations 1. Data collection should remain consistent over the course of a given project in order to allow calculation of trends and to highlight processes; 2. Activities related to health and education could be an entry point to access GBV survivors; 3. Sensitization should address stigmatization of rapes survivors and unmarried mothers; in particular, the difficulties for unmarried mothers to access schooling should be closely monitored and addressed. 4. To what extent were the four intended results achieved, in comparison to project indicators? (Detail per result) Analysis The evaluators compared targets with achievements for all indicators, per Result. In addition, they provided a judgment as to the relevance of the indicators to measure the given Result and as to the relevance of the data collected under each indicator. Data integrity/reliability was not verified by the evaluators given the recent data quality audit conducted by USAID that was extremely positive. Annex 6 presents for each Result a table with the agreed indicators, set targets and achieved figures. 11 To conclude on integration processes it would be necessary to compare such figures with population data – not available – as increases and decreases may also come from changes in the whole population. 12 One case has been reported by project staff about a rape survivor who had been referred to the judiciary after asking the peace committee for mediation (expected to lead to compensation). After 6 months in custody (illegal), the perpetrator was released and came back to the community. The survivor did not obtain information on the conclusion of the legal proceedings; neither did she obtain any compensation. 16 | Page Result 1: Increased social and economic peace dividends in targeted communities Findings 1. The achievements of the project under Result 1 are above what was committed, based on the agreed indicators and targets; 2. The project was overall able to report against set indicators in an appropriate manner. However the PMP indicates that for indicator 2, reporting will be disaggregated by sex, while this has not been done to date; 3. The set of indicators is somewhat narrow and limited compared to the Result, given that it measures community subprojects implemented and individuals living in targeted communities, but does not capture information on the services provided by the outputs of the said community subprojects (e.g. in the case of a community subproject that was a school, the service would be the education/schooling) and on the segment of the population using the service. It is however understood that the overall duration of the project, compared to the time needed to deliver the community subprojects’ outputs, would not have easily allowed for a meaningful measurement of social services; 4. Indicator 1 assumes by convention that any community project is by nature the “impact of a peace process”, a logic that is questionable. The Indicator was however set as such in the Task Order; 5. Indicator 2 assumes by convention that population estimates by community at project’s start remain valid at project’s end, and no new estimation has been conducted. Given potential for populations’ movements in Eastern DRC, this is somewhat simplistic13 Conclusions . 1. Based on the indicators approved by USAID, the project overachieved under Result 1; 2. Those indicators are however quite narrow, and sex-disaggregation was not implemented as planned. Recommendations 1. USAID and its contractors might want to consider more nuanced indicators for such Results; 2. More attention should be paid by MSI to disaggregating data by sex. Result 2: Strengthened social mechanisms for community reconciliation, conflict mediation … Regarding results 2 and 4, qualitative analysis has been added to quantitative data provided by the Indicator Performance Tracking table. As for social mechanisms for community reconciliation, conflict mediation and prevention capacity, the evaluators understand: 1. Community reconciliation has been linked to social cohesion: the degree of social diversity; 2. Conflict prevention : the quality of governance in the management of resources (land and water); 3. Conflict mediation/management : the degree of control of violence. Qualitative analysis has been documented through the set Result indicators (see Annex 6). Findings 1. Indicator 2.A: the impact of training sessions has affected positively the understanding of the targeted population regarding conflict mitigation/resolution, but in a less extent in the second year of the project. These sessions have given better results in the South (76% of target) than in the North (66%); 2. Indicator 2.B: the number of peace-building structures demonstrating active engagement in peace and/or reconciliation processes has improved and now shows active engagement in all of them; 13 There has been significant displacement of population in eastern DRC during the lifetime of PSCRP but there was not any significant displacement in the communities where PSCRP intervened, nor significant returns. The population values at project end are thus most probably somewhat accurate, but the system remains simplistic. 17 | Page 3. Indicator 2.C: the increase in attendants has increased over the second year and reached better results than expected; 4. Indicator 2.D: coordination meetings occurred regularly and extended to all communities successfully. Conclusions 1. The success of dialogues and coordination meetings can be emphasized as it has not been affected by people regular and essential occupations over the year; 2. Considering the impact of training sessions on people interested in conflict mitigation/resolution, we conclude that interest for such activities comes mainly after a period of implementation and support to mediation structures, from the actual interest they gain from implementing the conflict mediation activities themselves, rather than from further formal training. Initial formal training early on remains however critical to the ability of the committees to perform their core functions. Recommendations 1. One can expect a self learning process when it comes to understanding issues related to conflict mitigation/resolution; once committee members receive initial training, the monitoring of their successes in mediation activities will be more efficient than training sessions; 2. Information on whether engagements taken in coordination meetings have been followed by actions from leaders and Local Authorities would have been interesting as an additional indicator. (The evaluators were told that PSCRP is preparing such analysis as part of project’s wrap up). Result 3: Enhanced participatory local governance processes with community & civil society org. Findings 1. The achievements of the project under Result 3 are above what was committed, based on the agreed indicators and targets. This statement takes into account the fact that measurement of Indicator 3 had to be adapted to take into account the positive trend of communities preparing joint community development plans; 2. The project was overall able to report against set indicators in an appropriate manner. However indicator 2 should have been disaggregated by sex according to the PMP. It also appears that there might have been a change in the intent of the measurement of Indicator 1: PMP target’s low figure suggests that it might not have been initially planned to include CDCs under this indicator, while they have been included in project’s reporting documents14 3. The topics and scope of this Result are obviously very wide and could invite many more indicators. The set of indicators used by the project is however reasonable in number and captures well a sample of relevant critical dimensions of the project. ; Conclusions 1. Based on the indicators approved by USAID, the project slightly overachieved under Result 3; 2. The indicators capture well the scope of the Result, however where data should have been sex￾disaggregated, it was not reported that way. Recommendations More attention should be paid by MSI to disaggregating data by sex. 14 MSI noted that it allowed to ‘report them’ somewhere as no other indicator captures their number. 18 | Page Result 4: Increased reintegration of marginalized people into targeted communities Findings 1. The target has not been reached, but evolution of ex-combatants in project activities has had a regular positive evolution; 2. The figure for other marginalized groups has evolved positively and surpassed the target; 3. The evolution of the participation of marginalized groups has decreased over the last months of the project. Conclusions 1. The project has had positive effects on marginalized groups involvement in community processes supported by the project; 2. Although their participation in community development projects has increased over the period, their representation in community organizations has shown signs of decline in the last months; 3. One cannot conclude to a strong tendency, as this last evolution may come from other occupations they have at that period of the year; by comparing this evolution with the one of the whole population (indicator 2C), this tendency cannot be considered as a reverse of the integration of these groups. Recommendations Other indicators may be considered on social cohesion, such as the evolution of ethnic diversity and gender equity in schools and the evolution of the number of inter-ethnic marriages. However within the short timeframe of this project, such evolutions might hardly be attributable to the project, and changes in practices may be driven by the project presence rather than by real changes in attitudes. 5. Were any community interventions more successful than others? If so, what appears to have contributed to disparate levels of effectiveness? Were there unforeseen negative impact? Analysis The evaluators collected staff perception on the variability of success in two different ways: 1. In each community, what was the variability in the performance of the community in using the various key community processes and for each key community process, what was the variability in the performance of the various communities; 2. What was the variability in the contribution of each key community process to each of the Results. Unfortunately, the evaluators were able to collect only a limited number of staff responses for the 1st analysis listed above, so the value of the data for this analysis is limited. Findings 1. According to staff responses to the evaluators’ questions, there is very limited variability in the performance of the various communities in using the different key community processes. This suggests that there is very little disparity in the way the project was implemented from one community to another and in the way the communities were able to engage with the project’s processes. The evaluators were surprised of this finding, but unfortunately time was insufficient to research this topic further through triangulating staff responses with other data; 2. However, still according to staff, there is a significant variation in how various key community processes respectively contribute to each of the 4 Results (see chart below). This appears to the evaluators a logical and expected finding that resonates with experiences of other similar projects; 3. What’s striking and most interesting in the chart below is that it shows that community processes that were predominantly put in place for the purpose of the community recovery component (processes 1- 2-3) appear to have contributed also to the conflict resolution Result, and the other way around. 19 | Page The mark in each cell indicates the degree of contribution of the given community process to the given Result. 1 indicates a very high contribution; 7 indicates an absence of contribution. Key community processes contribution per Result R 1 R 2 R 3 R 4 1. Production of the community development plans 1 2 1 2 2. Implementation of the community recovery subprojects 1 3 2 2 3. Design of the maintenance plans 2 3.5 3 5 4. Decisions taken in accountability meetings (community-committees) 3 2 1 2 5. Coordination meetings between development and peace committees 2 2 1 2 6. Final meetings on community dialogues (community-leaders) 2 2 2 2 7. Conflict mediation and resolution services 3 1 3 2 Table 2: Proportional contribution of key community processes to Results' achievement Conclusions 1. Community processes that primarily have a community-recovery function appear to be critical for the success of the conflict resolution Result, and the other way around. This is further analyzed under Question 12 below; 2. Especially, and as it pertains to unforeseen effects of the project, conflict mediation and resolution services have had a potential in settling dispute arising from the socio-economic recovery component of the project (further discussed under Question 12 below as well). Recommendations 1. USAID could consider further developing the approach of combining community recovery and conflict resolution Results in programming, given the above findings that suggest that they were able to support each other in PSCRP; 2. Furthermore, as part of the conflict sensitivity approach, the conflict transformation component could generate risk analysis with regards to the implementation of productive infrastructure and/or other activities in the specific context of each community. 6. Were changes in social dynamics or community cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? Analysis The results from focus groups with project staff have been added to data collected by the evaluators during community visits. Other qualitative data from the project monitoring tools has been compiled: “Rapports de mission de suivi des noyaux de paix”and “Tableau de suivi des indicateurs paix/violence”. The analysis here focuses on the integration capacity within heterogeneous communities and social cohesion factors. Findings Changes in social cohesion as collected in communities can be measured through (See Annex 7): 1. The impregnation of human rights principles in beliefs and practices; 2. The evolution of ethnic diversity and gender equity in schools; 3. The evolution of interethnic marriages; 4. The degree of inclusivity of all segments of the population in decision making processes for planning and implementation of community development projects; 5. The degree of ethnic diversity in religious and traditional celebrations. 20 | Page 1. Whereas pygmies have benefited from training and sensitization activities developed in communities, the reintegration of ex-combatants continues to suffer from stigmatization from the community members based on stereotypes; 2. Improvement in the cohesion among ethnic groups has also benefited from the project; nevertheless, depending on availability of natural resources (mainly land and water), stigmatization of Banyamulenge and/or pastoralists continues to take place; 3. Beliefs and myths are persistent and affect women survivors of rapes and their children. The KAP survey (August 2011) shows that very few people look at survivors of rapes as equal human beings: categories coming first are Pygmies, ex-IDPs, widows, ex-combatants, returnees (Macha Mubambiro) / Pygmies (Karuba) / ex-combatants, Pygmies, IDPs, returnees (Bwegera) / ex-combatants and Pygmies (Sange). Conclusions 1. Looking at changes in perceptions, attitudes, behaviors after only 2 years of activity is inappropriate; 2. However, Annex 7 provides some insights on positive evolutions, to which the project may have contributed as suggested by the involvement of peace committees and CDCs; 3. School rehabilitation/construction has been a supportive opportunity to affect the equity in between gender and ethnic groups; 4. So far, the project has not had an effect on the general approach to marginalization processes and integration capacity. One may question the penetration of this concept: why did it benefit Pygmies and not to unmarried women? 5. It becomes even more sensitive when it relates to the recent history of cycles of massive violence, as the perception of ex-combatants can suggest; 6. Sustainability may come after further support from committees, focus on economic integration and learning by experience through community work. Recommendations Focusing on ex-combatants reintegration, and more generally on identities perceived to be conflict￾drivers, calls for more consideration to the psychological dimension of the process, i.e. on the issue on how to deal with past massive violence and dialogue about the crimes of the past. This does not necessarily mean that future projects such as PSCRP should take on such issues, but rather that government and donors should attempt to cover those additional dimensions as well in their global strategies and priorities, which can then be taken on through separate – but ideally coordinated – projects. 7. Was the theory of change, as expressed in International Alert’s global conflict resolution strategy, adapted and relevant to the challenges of the security and policy environment? Analysis It was clarified at the start of the evaluation that the question did not accurately quote the relevant reference document. The reference document’s correct title is ‘PSCRP Overall conflict resolution strategy’. From a normative perspective, the referenced document does not really present a ‘theory of change’ per se, to the extent that it does not offer a layered, logical set of sequential hypotheses for change. The document itself does not present itself as a theory of change and does not use that language. IA agreed to this finding and thus the evaluators handled this question not from the perspective of a theory of change, but from the perspective of a programmatic approach. Findings Main principles related to a concept of theory of change in the PSCRP strategy are: 1. Peace building should be cross-cutting in programming and implementation of socioeconomic recovery activities; 21 | Page 2. Conflict analysis emerges from the community level: this way, the process of generating a conflict analysis leads to a dialogue which carries the foundation for conflict transformation; 3. Community-level support should be approached through distinctive committees in order to avoid tensions resulting respectively from responsibilities in conflict management and the ones in resources management; 4. Coordination between both committees and coordination between project staff and Local Partners should be harmonized. Conclusions These principles have supported the implementation of socio-economic recovery especially at critical moments. The project staff mentioned several elements from the security and policy environment: 1. The legal ban on artisanal mining and the arrival of miners in the region increased the competition to access to labor; communities, with input from the peace committees, have incorporated this extra labor force in the organization of the building sites, avoiding struggle between workers, including the new arrivals; 2. The coordination between peace committees and local development committees has also developed advocacy capacities of communities at the level of the local administration: in Mulamba, military forces had seized trucks from the community subproject construction site, after which the construction contractor refused to resume the construction; the peace committee and the local development committee coordinated to advocate towards the Chef de groupement and managed to retrieve the vehicles from the military; 3. The peace committee mediated a conflict between CDC members and their president over lack of transparency and evidence of misappropriation of funds; the impartiality of the peace committee helped settle the conflict peacefully. Recommendations Considering the fragile environment in which activities take place, it would be recommended to further articulate these principles: 1. As mentioned earlier, and in the light of difficulties that committees have faced in dealing with contractual terms of the community subprojects contracts with construction contractors, carry out risk analysis with regards to implementation of productive activities; 2. Match LNGO partners capacities with needs emerging from the conflict analysis to the possible extent and ensure close quality control; 3. Maintaining a position of impartiality within the peace committee, outside the CDC, might be necessary when the latter is party to the conflict to settle. This has occurred in 11 cases out of 21 sub￾projects reviewed. Some peace committees members are also members of the CDC in South Kivu, which may put them in a difficult position when they have to settle impartially a case where they would be an involved party; 4. The coordination between project staff and LNGO partners suggested by the 4th principle (see findings above) could be further formalized to facilitate convergence between transversal and vertical communication channels (see PSCRP “Schéma des interactions sur terrain”): the suggested platform would integrate together LNGO partners, facilitators, the community development coordinator and the grants coordinator, thus acting as a support to the coordination process at the community level. 22 | Page 8. Did the project contribute overall to reconciliation and conflict transformation in targeted communities? How effective were its strategies and achievements? What was the impact (positive or negative, short or long term) of the project’s various components on peace￾building at community level? Analysis 1. Impact and outcome indicators as presented in the PMP refer more to the integration process and social cohesion than to reconciliation and conflict transformation; 2. Additional analysis of qualitative information collected in the 4 communities visited by the evaluators provides some insight on the reconciliation and conflict transformation process; 3. The analysis framework has been supported by the Reflecting on Peace Practice efficiency criteria15 Findings (those are the questions highlighted in bold in the findings below): strategies that contribute to the higher number of these criteria are considered more efficient than others. 1. Does the project target main determinants and actors at stakes in community social cohesion, conflict prevention and conflict management? Referring to conflict analyses (“Tableaux de sensibilité au conflit”), answers collected in the 4 communities visited by the evaluation and from the project staff, peace committees deal with land conflicts, conflicts between cultivators and pastoralists, inheritance disputes and other conflicts regularly met at community level. However, some limits have to be considered in the capacities of peace committees to address specific conflicts involving high level authorities but also in some cases conflicts between cultivators and pastoralists. Given this limitation, the project presents good results in terms of conflicts settlement (See Annex 8.1) – from 50 to 100% of cases are settled within a 3-month period – and has found ways to coordinate with local authorities (see unexpected results in Annex 8) in most communities; 2. Did the project lead to create or reform institutions or political mechanisms aiming at answering to people demands and addressing injustice? (See Annex 8.2) Committees are perceived by the community as answering their needs. When they pre-existed prior to project implementation, they have been reformed to ensure participatory and inclusive approaches. In particular, the peace committees extended access to conflict management to the most vulnerable who cannot afford to bring their case to the judiciary. The functioning of the CDCs improved the consideration of the needs of all segments of population, including women and ethnic minorities; (See also Question 3 above) 3. Did the project stimulate communities to take upon themselves initiatives to prevent and settle conflicts? See Annex 8.3. As for sustainability, experience from other projects suggests that after 2 years only, and given that most of socio-economic recovery community subprojects were just completed, the sustainability would be fragile, especially in a pre-electoral period; (See also Question 9 below); 4. Did the project encourage resistance to violence and provocations? Violent conflicts are not documented and the referral to peace committees or police or the judiciary does not give information on whether the proportion of conflicts managed in a non-violent manner has increased. The KAP survey (August 2011) provides some insights which would have been worth to compare with data from 2010 to ascertain the potential encouragement from the project however the evaluators did not have the resources to produce that analysis. Nevertheless, some situations have shown the capacity of communities to better respond to violence and threats by using the peace committees and mediation processes (see Question 7 above) and advocating at groupement level to solicit the police and the 15 Those efficiency criteria are linked to (i) determinants, populations and dynamics of conflicts, (ii) institutions and mechanisms for good governance and justice, (iii) the appropriation and the sustainability of initiatives mitigating divisions and strengthening cohesion, (iv) the capacity to oppose to violence, (v) security, real and perceived, (vi) cohesion among antagonists. 23 | Page military when threatened by rebel groups (Rutshuru 02/2011), or the local administration to protect the population against arbitrary custody by the police (Kibumba); 5. Did the project contribute to improvement of the security situation? In Sange I and II, comity members mentioned a decrease in the number of rapes and violence due to the occupation of inactive young people and ex-combatants on community subproject construction sites. Annex 8.4 presents some findings from the monitoring of peace indicators. However as the monitoring only focused on positive stories, it is not possible to have a comprehensive answer to this question based on the data available; 6. Did the project improve relationship between antagonist parties within the community? The issue related to land occupation and sharing between cultivators and pastoralists remains an issue of concern for most communities. Annex 8.5 provides some indications for the 4 communities visited by the evaluators (source: “tableaux de changement communautaire”). However as the monitoring only focused on positive stories, it is not possible to have a comprehensive answer to this question based on the data available. Conclusions Based on the criteria presented above, we can conclude that the project addresses successfully two of these criteria, and presents positive potential regarding the other. Recommendations 1. Monitoring and evaluation could be improved in a way that allows comparison from a period to another. Annex 8 presents the limits of the tools that have been used so far. Some recommendations are provided as well; 2. When it comes to civic participation and conflict management, a more comprehensive approach covering both civil society capacity building and public-sector institutional support would be highly recommended in a post-conflict environment. It would allow addressing conflict management and socio-economic recovery on an integrated and harmonized manner. 9. Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? Analysis Given the data available at the time of the evaluation, the evaluators were only able to analyze results’ sustainability, and were not able to address outcomes’ sustainability. The evaluators held a workshop with key projects staff. Using scenario planning methods, staff identified likely sustainability of results, including determinants where possible. The subsections below make extensive reference to project indicators. They are all presented above under Question 4 for reference. Findings Result 1: Increased social and economic peace dividends in targeted communities The project’s indicators for Result 1 are not very relevant to help estimate likely sustainability of achievements. Sustainability could instead be estimated through forecasting evolution of utilization of social and economic infrastructures delivered by the project, as well as the level of involvement of the mid-local levels of the technical line ministries (such as the ECZ in the case of health or the sous-proved in the case of education, for example) in the operating of the said infrastructures. Staff perceive the sustainability of Result 1, based on the above alternative indicators as likely, and think it can be foreseen to last for several years (assuming a stable security and policy context). Staff estimate that the trend is likely to be one of continuous, slow gradual increase. The reasons why staff offer this estimation are the following: 24 | Page 1. New/nicer social and economic infrastructure attracts utilization; 2. New/nicer social and economic infrastructure plays a pull factor for populations who are more likely to relocate within the catchment area, thus increasing users’ numbers; 3. Increased utilization of social services functions as a snow-ball effect within communities; 4. As an external factor, population return (refugees, IDPs) is broadly expected in PSCRP areas; 5. Operations and maintenance plans were put in place to increase infrastructures’ sustainability, including with considerations given to easing access, including financial accessibility. In the opinion of evaluators, staff perceptions match with the experiences of other projects in Eastern DRC. The sustainability of social services is however significantly challenged in such contexts by financial access inequity issues when service delivery units are not externally supported. Result 2: Strengthened social mechanisms for community reconciliation, conflict mediation and prevention capacity 1. According to the experience of the project’s staff, the community-based approach will support the continuous interest of the population in the medium term. Peace-building structures provide mediation services for free, mediators have deep understandings of the root causes of the conflicts, and local authorities have so far had positive interactions with them (with some exceptions as already stated, but surmountable with strengthened cooperation). Some communities, not targeted by the project, have already replicated the experience. In the opinion of the evaluators, demand for such services might be a key driver of sustainability of the institution providing them, however the free nature of the services might not be sustained if the committees are no longer externally supported; 2. Knowledge about conflict mitigation/resolution will remain with the beneficiaries of the trainings and the project staff expect them to continue using the training tools; 3. More concerns are expressed on the average number of people attending the meetings of community committees, which is expected to vary in accordance to the interest each meeting’s topic represents for the community; 4. The same concern applies to the number of dialogue and coordination meetings, which has already shown signs of decline. “Transportation fees”, provided by the project on a monthly basis to community members, will stop and might limit the willingness of interest of participants to attend. The project staff don’t expect to see these dialogue and coordination meetings last over a year, which actually could still be quite optimistic in the opinion of the evaluators. There is a tension in between genuine and financial motivations on the part of the members; while the former might exist, the fact that the latter was put in place during project implementation and is then stopping might undermine the former. Result 3: Enhanced participatory local governance processes with community and civil society organizations The project’s indicators for Result 3 were deemed relevant to estimate likely sustainability of Result’s achievements. Staff perceive that achievements under this Result are sustainable based on the project’s indicators. However, they foresee significant disparities from community to community, an analysis that helps understand determinants of this expected sustainability: 1. Committees are expected to be sustainable, but this is expected to vary significantly depending on their strength at the time of the project end. Weak committees are expected to stop functioning within circa 6 months, while stronger ones are expected to function up to 24 months beyond project’s end. Staff also feel that whether local leaders recognize, liaise with and use the committees will be key to 25 | Page enhancing their sustainability beyond project’s end. If successful, IGAs might also increase the sustainability of the given committees, however this might backlash where those are mismanaged; 2. The knowledge gained by individuals trained by the project is expected to remain. However staff noted that the pattern of membership turnover or disengagement in committees is high and is likely to remain so, thus the knowledge gained through the trainings will progressively erode at the institutional level; 3. Community development plans (CDPs) are expected to be sustainable on paper as they were issued in a highly formal way. Whether they will continue to be implemented beyond project’s end will highly depend upon whether the communities implement those CDP activities that were identified as being implementable without external support or funding. The role of local leaders and mid-level authorities will also be key in keeping the CDPs alive; 4. Accountability meetings between the committees and the communities are expected to continue only if the committees continue to carry some activities, which is expected to be highly variable from one community to another as stated above; 5. Maintenance and operations of social and economic infrastructures is expected to be sustainable. Staff expect to see long-term processes in this field in sectors where permanent committees exist (such as health and education). Where the community subprojects focused on sectors where no permanent institutionalized committees always exist (such as water, roads, bridges, mills, etc.), staff feel that the sustainability of those and thus of the maintenance will highly depend on the leadership of local authorities and local leaders. In the opinion of the evaluators, sustainability of such institutions is possible only if they continue to have a function. In the absence of external support for community project activities, the experience of other similar projects suggest that the CDC might be sustainable in name, but not in activities or function. Result 4: Increased reintegration of marginalized people into targeted communities 1. Arrivals of Pygmies have already been observed in communities where the project has been implemented. According to project staff, marginalized groups will remain interested in the potential benefits they can gain from integration: public services, income generating activities, good governance. Conditional on the positive evolution of security, project staff expect the participation of marginalized groups and ex-combatants to increase over the next 5 years. In the opinion of the evaluators, this seems possible, but is highly dependent on questionable external factors; 2. After 2 years of activity, considering that not all marginalized groups have benefited from the project as the Pygmies may have (schooling, in some cases allocation of land), and given the negative stereotypes that remain and affect attitudes towards the ex-combatants, it is far too optimistic to expect a process of social inclusion developing without incentives, such as, for example, the capacity of local authorities and committees to implement the development plans. Conclusions and recommendations 1. The key determinant that cuts across the above appears to be ownership. This dimension is clearly expressed as both internal (from committee members) and external ownership (from local authorities for example) and staff seem to feel that the latter is the most critical, based on the indicators used. This suggests that more needs to be done in similar, future projects to increase and improve linkages between communities and local authorities; 2. Communities also need to continue to be accompanied and support by projects – and later ideally by the DRC government, to maintain incentives and mechanisms for stability through continued socio￾economic recovery investment and support to conflict prevention and management mechanisms; 3. Context considerations appear critical to the sustainability of the project’s results, starting with security considerations. This shows that a comprehensive approach is necessary to sustain 26 | Page stabilization, including also continued USG efforts towards the other key components of the stabilization frameworks (security, political processes, etc.). Contribution to the eastern DRC stabilization agenda 10. Was the project coherent with the community-related components of I-SSSS and STAREC strategies? Analysis The evaluators reviewed key STAREC and I-SSSS documents, and met with senior staff at the STAREC secretariat, the provincial ministries of planning, and MONUSCO’s stabilization unit. Findings 1. The I-SSSS and STAREC do not, strictly speaking, have a defined, approved strategy for their ‘community-related’ components. The answer to this question therefore has to rather rely on senior MONUSCO and STAREC staff’s perceptions; 2. STAREC and MONUSCO understand that at the time PSCRP was tendered, designed and started, both the coordination and strategic frameworks of STAREC and I-SSSS were only nascent. They therefore analyze PSCRP in terms of how the project adapted itself over the course of its implementation to those frameworks as they got in place; 3. Both STAREC and MONUSCO very much appreciate the proactive, continuous and senior-level (CoP) involvement of PSCRP in the coordination mechanisms and in the strategic dialogue; 4. MONUSCO’s thinking on the I-SSSS has very much evolved over the course of the time period where PSCRP was implemented. For example, they have moved away from the strict ‘axis’ approach to rather prioritize interventions through ‘zones’, and they have started to differentiate for each of the I-SSSS components which zones are relevant. MONUSCO noted that PSCRP very much preceded them in that thinking and that PSCRP helped them think through this evolution. The specific choices made by PSCRP in terms of geographical focus also appear particularly relevant to MONUSCO today; 5. Populations return during the course of PSCRP has been minimal in the areas of interventions of the project, and thus MONUSCO noted that it is difficult to ‘test’ the PSCRP approach in terms of its ability to ease and sustain return; 6. Both MONUSCO and STAREC very much appreciate PSCRP’s flexibility in its ability to rapidly finance training activities for the CLPCs, at a time where keeping that process going was critical; 7. While this was not raised by STAREC or MONUSCO, the evaluators note that the I-SSSS IPF states that agricultural recovery activities are needed under the ‘return and reintegration’ component. In the meantime, communities chose very few agricultural projects under the community recovery component of PSCRP, while it is generally known that livelihoods and income is a critical need and priority of communities, and that most of the target population is involved in agriculture. In this context, the evaluators welcome the inclusion by USAID of a livelihoods and agriculture mandatory component in the RFA for the CRLP project that appears to be following PSCRP. Conclusions PSCRP appears to have been highly coherent with the I-SSSS and STAREC in terms of their respective ‘return and reintegration’ and ‘humanitarian and social’ components, as perceived by the senior STAREC and MONUSCO’s staff met. In a context of continuous evolution of those frameworks, which remain quite general in terms of specific programming approaches, continuous senior level coordination appears to have been a key determinant to allow PSCRP to engage with, but also contribute to the evolution of those frameworks. USAID’s clear requirements to this end in the Task Order were probably key as well. 27 | Page Recommendations Future projects should continue to closely liaise with STAREC and MONUSCO and engage in coordination forums so as to both align to the stabilization frameworks and contribute to their positive evolution. This requirement should be clear in the performance planning of future CoPs. 11. Did it add value to the other components? Analysis The evaluators reviewed key STAREC and I-SSSS documents, and met with senior staff at the STAREC secretariat, the provincial ministries of planning, and MONUSCO’s stabilization unit. Findings The I-SSSS and STAREC do not, strictly speaking, have defined, approved strategies for their components. The answer to this question therefore has to rather rely on senior MONUSCO and STAREC staff’s perceptions. For ease of reading, the findings below are presented based on the components’ typology of the I-SSSS. The STAREC components are structured in a slightly different way, but topics overlap overall. Improve security MONUSCO’s view is that activities such as PSCRP should not be implemented in zones where the I-SSSS focuses on improving security, for do-no-harm reasons. Their expectation is that implementers of ‘return and reintegration’ projects develop a sound-enough understanding of the context so as to avoid those zones, and coordinate with MONUSCO and STAREC to this end. MONUSCO’s perception is that PSCRP met those expectations in its geographical targeting and in its coordination efforts. Support political processes MONUSCO’s analysis is that the political processes as understood in the I-SSSS have essentially been in a deadlock over the course of the implementation of PSCRP. While they acknowledge that the links between community-level and broader conflicts are extremely complex and not fully understood by the international community, they think that engaging at the community level is critical (although of course not sufficient) to address the bigger picture. While this was not articulated as such by MONUSCO, the evaluators understood that they believe that the conflict resolution component and approach of PSCRP can contribute to this pillar of the I-SSSS, although in a minimal way given that the links between local and global conflicts are complex and given that the project only works in a very small number of communities compared to the total geographical area concerned. Strengthening state authority The I-SSSS and STAREC have delivered a lot of inputs to date towards strengthening state authority. MONUSCO however acknowledges that this has not yet yielded much result (insofar as a lot of buildings have been built, for example, but they are not necessarily staffed by the relevant ministries’ staff). MONUSCO stated that their expectations vis-à-vis this component are very low at the moment, let alone what projects under other components might contribute. The idea would however be that as state authority is strengthened in a geographic area where a ‘return and reintegration’ project is implemented, that project should seek to contribute to bring communities and state actors closer to each other, with taking into account specific political dynamics at the very local level. It appears that PSCRP has very much done so, through various activities of the project (community development plans, community dialogues, maintenance plans, conflict resolution referrals, CLPC trainings, etc.). As state authority continues to be strengthened in Eastern DRC, future project will obviously have an opportunity to do more to this end. 28 | Page Combat sexual violence PSCRP has contributed to the first sub-component as stated in the I-SSSS framework (combating impunity and improving access to justice) through referrals under the conflict and reconciliation Result. This was however limited in a context where peace committees are legally not allowed to handled crimes. Those committees however handled other types of GBV-related conflicts, for example on inheritance. Women were also included throughout the project in community processes, institutions and decision making, and GBV programming best practices suggest that challenging gender roles can be critical on the long run to reducing sexual violence. More could however probably have been done in mainstreaming GBV issues within PSCRP. Conclusions Based on MONUSCO and STAREC perception, PSCRP seems to have contributed to a level they deem reasonable, given the context, to the other components of the stabilization framework in Eastern DRC. There are however certainly opportunities for improvements in the future, especially as those other components start to deliver results of their own and as the context evolves. Recommendations In future programming, USAID might want to require implementers to develop and present a plan on supporting the stabilization’s other components, so as to maximize and systematize opportunities for support, however this would require that I-SSSS and STAREC have much clearer, formal and written strategies per component. This could be required within annual Workplans, for example. 12. Was the combination of reconciliation with recovery an effective strategy for stabilization (i) in targeted communities and (ii) beyond those communities? Could specific strengths and weaknesses be identified? Analysis Referring to stabilization in the context of implementation of I-SSSS and STAREC in Eastern DRC, the evaluators specifically looked at the following three key dimensions of those frameworks: 1. The integration of ex-combatants, and their rehabilitation within communities; 2. The return of refugees and IDPs; 3. The conditions for economic recovery. Findings 1. The project has proved its potential in mobilizing forces at community level for economic and social integration: local development committees, community development plans, accountability meetings, etc.; 2. Regarding the ex-combatants, positive results have been achieved with regards to their economic reintegration: contribution to community development projects mainly; 3. Nonetheless, with regards to their social rehabilitation, as shown in monitoring tools regarding conflict transformation and the KAP surveys for example, there is still a long way ahead; 4. Stigmatization of ex-combatants is highly related to violations of human rights, whose responsibilities are at the level of FARDC and militias in the populations’ perception; 5. At community level, no reconciliation process between ex-combatants and the civil population can be foreseen through the project’s results. This is partly the responsibility of institutions which fail to address the collapse of the Rule of law; 6. The return of refugees and IDPs: the project has allowed addressing the land issue and build capacities of peace committees for land conflicts prevention and management. Conclusions 1. Socio-economic recovery activities may have the potential to lead to social reintegration of ex￾combatants; 29 | Page 2. In the meantime, peace committees and LNGO partners could emphasize their role in the process of reconciliation by focusing more on the needs of ex-combatants, rather than on ethnic minorities and ethnic conflicts; 3. The capacities built at community level have a potential to be further connected to higher levels (provincial, through the secteurs/chefferies administration). Recommendations 1. Emphasize on the impact on perceived security and the resume of links between the State (formal local authorities, legal empowerment of the poor and the marginalized) and the population, with the CDCs at a crossroad; 2. Emphasize on security and good governance through strengthened capacities of both CDCs and peace committees to advocate together towards local authorities as it has proven efficiency. 13. Was the engagement with CLPCs in North Kivu effective, with regards to project objectives and to CLPCs’ own training needs? Analysis The evaluators analyzed key documents describing the role of CLPCs and confronted this with the project’s Result Framework. In addition, 1 CPLC president was interviewed, as well as key MONUSCO and STAREC personnel. Findings 1. As mentioned in the evaluation methodology, given that the trainings for the CLPCs occurred towards the very end of the project and given that other political, financial and institutional challenges limit their ability to undertake activities to date, it is not possible to estimate effectiveness, and thus findings can be made only vis-à-vis relevance; 2. Engaging with CLPCs appears extremely in line with all four of the project’s Results, given their ToRs. Given their initial focus, supporting the CLPCs will however primarily contribute to Results 2 and 3; 3. STAREC and MONUSCO are overwhelmingly satisfied that PSCRP was able to provide technical and financial resources to support the CLPCs, and they very clearly make the link between such activities and advancing stabilization in Eastern DRC; 4. Furthermore STAREC noted that PSCRP is able to provide an opportunity for CLPCs to liaise with lower-level conflict resolution institutions (the peace committees) and sees this as a critical support to the future effectiveness of the CLPCs themselves; 5. At a very anecdotal level the one CLPC president met by the evaluators stated that he was satisfied by the relevance of the initial training package funded by PSCRP compared to his analysis of the CLPCs training needs. This however remains purely anecdotal; 6. An effort was obviously made to analyze CLPC training needs by the various agencies involved in delivering it, including the PSCRP team; 7. A review of the training schedule suggests that it might be a mixture of mostly training topics that are relevant to the CLPC needs and of a few sessions that might rather serve the interest of those UN agencies offering them, rather than of the CLPCs themselves. Conclusions PSCRP’s support to the CLPCs was probably relevant and the training outline defined is probably broadly relevant to their needs. Further time and analysis is however needed to answer such a question. It will be at the very least several months, after the elections, before the CLPCs start to fully take on their role in the field and before the effectiveness of the training and support can be fully assessed. 30 | Page Recommendations Support to and collaboration with CLPCs should most probably be continued in future similar projects. Engagement with other mid-local governance actors would however be critical to more fully contribute to what was Result 3 in PSCRP. Lessons Learnt / Good Practices 14. Are there areas where PSCRP can be deemed to have reached good practices, which could be turned into practical tools or guidelines for future projects? Are they specific to eastern Congo or could they be adapted to different contexts? SoW asked that the evaluation looks particularly into: (i) approach to profiling and determining adequate community size; (ii) election / redynamization of community committees (development and peace); (iii) elaboration of development plans; (iv) resource kits on conflict management and resolution for peace committees; (v) inclusion and participation of disenfranchised groups; (vi) collaboration between community development and conflict resolution entities. The evaluators held semi-structured interviews as well as a workshop with key project staff to identify good practices as well as lessons learned from the project. Literature on good practices in similar projects was also used. A diverse and context-sensitive size of communities – CDD experience continues to highlight the importance of clustering population settlements in a meaningful rather than arbitrary way to form ‘communities’ for the purpose of CDD projects. Dividing meaningful communities of people living together into several project communities can prove harmful against project’s aims. Similarly, clustering together settlements that do not share common public goods can render the project implementation complex. For those reasons, a CDD project should be ready to and capable of dealing with communities of varying sizes. PSCRP was able to successfully deal with this issue. Approaching special-interest groups through inclusion, not targeting – PSCRP rightfully adopted an inclusion approach to special-interest groups, instead of the direct targeting or earmark approach that some CDD projects take when they are pressured to show results on specific groups. Staff feel there are clear examples that show the current project has had a positive effect on the attitude towards marginalized groups, and in favor of their participation (ex-combatants, pygmies) as well as for women, as discussed above. Keeping a similar approach to blockgrant use, but clarify rules – Broadly speaking the rules determined on the utilization of their blockgrant by the communities in the project were relevant. They could however be clearer and formalized, to give a stronger image of transparency. For example, the recommendation to communities to use their blockgrant to fund 2 community subprojects, one small and one large, in that order, is generally considered a good practice in CDD projects. During PSCRP, different rules applied to the small and the large community subprojects in terms of community management of cash. Communities’ perception seems to be in some locations that the project changed the rules, while the truth is rather that different rules applied to the 2 types of community subprojects, from the onset. More formalized and clearer information on those rules, supplied early on to the communities would have reduced the risk of misunderstandings and frustrations. The approach chosen on this specific issue of cash management is in itself positive, and should be kept. Features such as the ability of communities to manage directly some of the aspects of the large community subprojects when those aspects were not subcontracted to a construction firm but relate to 31 | Page add-ons to the core construction component (for example small equipments producing to equip the facility been built) was positive. The feature allowing a community to decide to join its blockgrant, or part of it, with another community next-by to implement a larger community subproject was also positive, but safeguards might have been lacking. For example it is advisable to cap the number of communities that can join together or the maximum total amount that can be pooled, to avoid very large community subprojects that are difficult to manage16. In terms of community subprojects eligibility, the approach of PSCRP of using a negative list (= a list of what is not eligible is given to the community) is better than using a positive menu (= giving a list of what is eligible to the community) based on lessons learned from other CDD projects. However that negative list should have been clarified and formalized. ‘community subprojects that require more funding than the total blockgrant’, ‘community subprojects that require more than X months to be completed’ and ‘items that are on the USAID list of items requiring a waiver to be procured’ were on the negative list used. Other items should probably have been added such as ‘religious buildings’, etc. In terms of Blockgrant allocation, PSCRP should have considered some variability in the size of community blockgrant, especially given the high variation in populations’ sizes from community to community. CDC/CLD membership – The project has introduced the interesting feature of enabling the community to dismiss one or several CDC/CLD member(s) over the course of its mandate, if they are not satisfied with their work. This takes place during ‘accountability general meetings’. It seems to be an interesting feature, which, with appropriate safeguards, should probably be kept in future projects. Other good practices and challenges identified per step of the community process are presented in a table in annex 9. 15. Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming Communities appear to appreciate the services of the conflict resolution committees – Staff generally report that communities like the services of the peace committees and want to have those in their village, a situation that seems to match with what the evaluators perceived during community visits. They see that as an accessible, fair and free judiciary service. Furthermore, staff generally report that it is positive to have both a CDD and a conflict resolution component in a given project/community. They feel that the conflict resolution component can help the CDD component to function better by also resolving potential conflicts arising from the CDD component. They feel that the conflict resolution component has more positive than negative effects on social cohesion within the community and across communities. Balance land issues over other types of conflicts – There seems to be an unbalanced focus on land conflicts over other conflicts in the discourse about community conflict resolution in North Kivu. This might be a fair representation of the situation on the ground when speaking about very specific geographical areas near Goma (and where humanitarian and development stakeholders might be over￾represented). That issue might be however overplayed compared to other types of conflicts when looking at the broader potential geographical reach of the project and at root causes of instability. 16 In Afghanistan under the NSP 45 communities chose at one point to pool their blockgrants together to build a ‘large’ hydropower plant and electricity grid, and this created significant problems for the project, given that CDD projects are typically not set up to manage such larger community subprojects. 32 | Page Enhance linkages with other conflict resolution institutions – Staff have very diverse perceptions on this topic, showing that it should probably be studied further to understand realities, or variance between locations, for example. While in some areas it is reported that local authorities no longer see the conflict resolution committees of the project as competition, in other locations those tensions exist. In a global context of sometimes reduced profile and legitimacy of the traditional institutions, the new committees can be perceived as competition, especially in terms of competition over resources as traditional conflict resolution systems do not typically offer services for free. Linkages with the local administration or judiciary could also help in providing some level of quality assurance over the ‘decisions’ of the committees, and in formalizing those decisions by following them with the issuance of official documents/paperwork. Carefully balance scope with capacity, and build the latter – Staff and partners reported that during the project members of the peace committees sometimes received formal training late into the course of the project. Some of the peace committees also experience a low retention of their members, resulting in a low retention of the skills supplied through training. Capacity building of the members of the peace committees probably needs to be significantly enhanced. Capacity building also needs to be understood as a quality assurance mechanism over the analyses and ‘decisions’ the committees take. It has been pointed out that sometimes the ‘decisions’ of those committees are in question, as they are not necessarily aligned with the Congolese law, and as there is no concept of legal precedents, even within the network of committees set up by the project. At times, this has resulted in ‘decisions’ not being accepted, or in doing more harm than good by making the given conflict emerge again in a stronger way later on. Consider pros and cons of focusing on women membership or on traditional institutions – Some stakeholders raise questions as to the relevance of working in some communities with peace committees that are exclusively composed of women. It appears that those communities might have struggled in being accepted by men as a valuable conflict resolution mechanism for their own conflicts or issues. Where the project has supported traditional conflict resolution mechanisms (Barza or Baraza, thus composed exclusively of men, but that could probably be widened to include representation of women and ethnic minorities), it is reported that those might have been more widely accepted within the community, including across sexes. Carefully consider pros and cons of side components such as the physical huts and the IGAs – The project has, at least in some locations, supported the peace committees through supplying them with a small building (a hut) and/or capital for an IGA. The rationale of those actions was to enhance the sustainability or profile of the committees. Careful review of the outputs and outcomes of the IGA initiatives should further down the line be conducted before considering renewing such initiative. There are numerous instances of similar programs in Eastern DRC that have shown that when an institution which role is not primarily to generate profit is tasked with doing so this results either in a failure of the economic activity, or in a focus on that economic activity to the detriment of the initial social objective. Some staff have also reported that in some instances those members of the peace committees who stay engaged in that committee over time typically tend to be very entrepreneurial. This has reportedly resulted at times in a tendency to branch out into other activities not related to conflict resolution, and this to the detriment of the initial core mandate. Some staff have expressed the opinion that the buildings/huts in particular have rendered those conflict resolution committees much more visible within the community, and has provided them an edge against the CDCs/CLDs in terms of becoming the point of contact for other development initiatives looking for an institution to interact with within the village. Logically it should rather be the CDC/CLD that acts as an entry point for other development initiatives, rather than the conflict resolution committee. This is not meant to say that the visibility of conflict committee should be reduced, but that development committees should ideally be given similar tools. 33 | Page 16. Are there scalable elements in this project? Which ones? The evaluators reviewed literature on approaches that are broadly incorporated into the evaluated project, as well as documents on projects that have expected results that are broadly similar to that of the evaluated project. On this basis, it appears that there are three broad ways to understand “going to scale” in the context of PSCRP. Horizontal scale – There are extensive documented experiences of at-scale CDD projects that work at the community level through committees and deliver socio-economic, governance and social cohesion results. Because of the systematic nature of this approach, it is easily scalable and has been implemented at scale in various contexts, including DRC. Given that the reconciliation component of PSCRP operates through a similar, community-based, systematic model, it appears logical to think that it should be easily scalable as well. Horizontal scale in this context would mean implementing the same package of activities in more communities, covering a larger population. Both in DRC and globally, the programming and operational advantages of delivering a CDD program over continuous geographical areas and over entire administrative units continue to be demonstrated. In the specific context of DRC and considering the stabilization aims of USAID, going to scale horizontally should attempt to cover entire administrative units (groupements in this case). This will limit unintended negative effects on stability with communities that are not covered. This will also maximize the likelihood and substance of positive interactions with local public authorities. The project has received complaints of uncovered communities that felt ‘excluded’, and during field visits community members raised the issue of near-by uncovered communities. This is a usual pattern in CDD-type programs and hardly avoidable. Using official boundaries and covering entire administrative units would however mitigate the issue. This would also make it easier to make linkages between the CDC level and the above official administrative levels in terms of development planning. Vertical scale – Even simply in Eastern DRC, there are current experiences of other projects that aim at delivering fairly similar results than PSCRP, but at a different level. On the socio-economic side, for example, UNDP implements a project that sets up committees at the mid-local level, supports the production of development plans, and would fund social infrastructure if it had sufficient resources to do so. On the reconciliation side, UN-Habitat implements a set of projects that set up conflict resolution committees at the mid-local level as well. A project such as PSCRP could be scaled-up by replicating its approach at higher levels of the administrative pyramid, thus addressing also more complex, larger socio￾economic and conflict issues. Sector diversity – While PSCRP’s focus at the community level is not restricted (communities could choose to allocate their block-grant to virtually any type of community subproject, and to address within their peace committees virtually any types of conflicts as long as not crimes), the experience of PSCRP shows that under both pillars (socio-economic and reconciliation), the scope of what communities engage with remains fairly limited. Most socio-economic activities have been schools and health facilities and most conflict resolution has focused on individual conflicts. This appears to be a consequence of the limited resources and capacity that both the committees and the implementing agencies have, especially in terms of time and funds. With more time and resources, a project such as PSCRP could be sectorally diversified, and thus scaled-up, to address a broader scope of socio-economic needs or conflict causes. The focus of the CRLP project that USAID was preparing at the time of this evaluation and that appeared to be a follow-on to PSCRP very much builds on this logic, with additional discrete components of agriculture or gender-based violence. Such scaling-up would allow a project to provide a more comprehensive package of solutions to communities. 34 | Page Management considerations – When going to scale, under any of the three dimensions proposed above or with a combination of them, management considerations will have to be carefully considered. Managing scale generally suggests ensuring that a certain level of simplicity is maintained to ensure that projects are delivered within timeframe. Going to scale vertically or sectorally as well as horizontally, all at the same time would probably generate risks vis-à-vis complexity and manageability. A staggered approach, where scale is first achieved horizontally and then later vertically and sectorally (or the other way around: first increased complexity and then increased horizontal scale) would probably be more reasonable. Looking at the experience of other projects, donors should limit expectations that horizontal scale would bring significant efficiency gains, as additional levels of management need to be foreseen, by plateaus, as the number of communities is increased. 35 | Page Annexes 1. Scope of Work of the evaluation TERMS OF REFERENCE Final Evaluation of the Promoting Stabilization and Community Reintegration Project (PSCRP) I. BACKGROUND Under funding from USAID, MSI has been implementing in eastern DRC since October 2009 the Promoting Stabilization and Community Recovery Project (PSRCP), in partnership with International Alert. This project will end in November 2011. The overall purpose of PSCRP is to Enhance reconciliation and recovery in targeted communities in Eastern DRC. Under this purpose, PSCRP must deliver the following results: • Result 1: Increased social and economic peace dividends in targeted communities • Result 2: Strengthened social mechanisms for community reconciliation, conflict mediation and prevention capacity • Result 3: Enhanced participatory local governance processes with community and civil society organizations • Result 4: Increased reintegration of marginalized people into targeted communities. For this, PSCRP operates in 16 communities in South Kivu and 14 in North Kivu (see sites on annexed maps). PSCRP belongs to the USAID/DRC’s Peace and Stability Office’s priority goal of increasing stability by mitigating the causes and consequences of conflict in the country, including its focal area: Promote conflict prevention mitigation and resolution, as well as stabilization and recovery, with an initial focus on the on – going tensions in Eastern DRC. It belongs also to a programmatic framework set under the community recovery and stabilization components of MONUSCO’s International Security and Stabilization Support Strategy (I-SSSS) (component 4) and of the Congolese Government’s Stabilization and Reconstruction Program for zones emerging out of armed conflict (STAREC) (objective 3). For this reason, communities supported by PSCRP are located on I-SSSS and STAREC priority axes, with the exception of 6 communities in Ruzizi plain, which were pre￾selected in the Task Order. II. PURPOSE OF EVALUATION The purpose of this consultancy is to (i) assess the contribution of this community-based project to its purpose: reconciliation and recovery in targeted communities; (ii) assess its contribution to the eastern DRC stabilization agenda and (iii) identify those lessons learnt and good practices developed under PSCRP, which could inform a future programmatic strategy. 36 | Page III. EVALUATION TEAM MSI and International Alert will commission a two-person team to conduct this external evaluation. The evaluation team leader will be recruited by MSI and International Alert will identify the second member of the team with a specific remit to comment on the reconciliation components of PSCRP and the peace￾building lessons its experience suggests. Under the direction of the team leader, the evaluation team will submit a joint report, for eventual transmission to USAID as part of the project documentation. Supporting the evaluation team and providing guidance are PSCRP’s Chief of Party, Hervé de Baillenx, and Deputy Chief of Party, Zahed Yousuf. Franklin Nzinga, PSCRP’s M&E Officer, will also support the team. The Evaluation Team’s summary tasks are: 1. Before departure, conduct a desk review of documents pertaining to the project. 2. Design the methodology and approach for the final evaluation, which will address in particular aspects of relevance, impact (including efforts to minimize negative impacts) and effectiveness (with specific regard to timing constraints); 3. Approval of the above by MSI 4. Develop data collection instruments including questionnaires and interview guides; 5. Conduct interviews with project staff, project beneficiaries and other stakeholders – this will involve travel to field sites, including South Kivu; 6. Work closely with the project staff on the identification and documentation of lessons learnt; 7. Conduct data analysis 8. Produce draft report for review by the project COP and DCOP 9. Approval of the above by MSI 10. Interface with the USAID Mission to present initial findings; 11. Produce final report, addressing possible comments from USAID. IV. KEY EVALUATION ISSUES/QUESTIONS Note: any important findings or observations outside of these main questions will be appreciated. Contribution to community stabilization and recovery 1. Did the strategies in the original design meet the requirements of the Task Order? Did they need revising during the course of PSCRP? Can some of the difficulties experienced during project implementation be attributed to flaws in the original design?17 2. Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? 3. Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? In particular, how effective was the project in responding to the needs of ex-combatants and other marginalized groups? 4. To what extent were the four intended results achieved, in comparison to project indicators? (detail per result) 17 The exact language of this question was changed. See below sub-section on changes to SoW. 37 | Page 5. Were any community interventions more successful than others? If so, what appears to have contributed to disparate levels of effectiveness? Were there unforeseen negative impact? 6. Were changes in social dynamics or community cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? 7. Was the theory of change, as expressed in International Alert’s global conflict resolution strategy, adapted and relevant to the challenges of the security and policy environment? 8. Did the project contribute overall to reconciliation and conflict transformation in targeted communities? How effective were its strategies and achievements? What was the impact (positive or negative, short or long term) of the project’s various components on peace￾building at community level? 9. Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? Contribution to the eastern DRC stabilization agenda 1. Was the project coherent with the community-related components of I-SSSS and STAREC strategies? 2. Did it add value to the other components? 3. Was the combination of reconciliation with recovery an effective strategy for stabilization (i) in targeted communities and (ii) beyond those communities? Could specific strengths and weaknesses be identified? 4. Was the engagement with CLPCs in North Kivu effective, with regards to project objectives and to CLPCs’ own training needs? Lessons Learnt / Good Practices 1. Are there areas where PSCRP can be deemed to have reached good practices, which could be turned into practical tools or guidelines for future projects? Are they specific to eastern Congo or could they be adapted to different contexts? Look in particular into: (i) approach to profiling and determining adequate community size; (ii) election / redynamization of community committees (development and peace); (iii) elaboration of development plans; (iv) resource kits on conflict management and resolution for peace committees; (v) inclusion and participation of disenfranchised groups; (vi) collaboration between community development and conflict resolution entities. 2. Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming. 3. Are there scalable elements in this project? Which ones? 38 | Page V. EVALUATION METHODS Document Review: The evaluation team will review the following documents, as applicable, before conducting any interviews or trips to/within the region relating to the evaluation. • Task Order • Project Documents and PMP • Quarterly Reports • I-SSSS programmatic framework and STAREC ordnance • Training materials, notably on conflict prevention and resolution. • Evaluation Report conducted in May 2011 by dTS • Any other relevant documents Team Planning Meeting: There will be an initial Team Planning Meeting (TPM) with the COP, DCOP and M&E Officer upon arrival of the evaluators in Goma. The objective of the team planning meeting is to reach a common understanding regarding the status of the evaluation, the priority evaluation questions, the available data sources and data collection instruments, the evaluators’ detailed calendar of activities while in country, their needs for logistical support and an outline of the final report. The choice of field visits will be made jointly, depending on the evaluator’s chosen methodology and data needs. Individual Interviews: Interviews will be conducted with the following individuals: a. Staff members of projects; b. Project beneficiaries from supported communities, some of whom will come from marginalized groups; c. Members of Local Development Committees and Peace Committees; d. Selected individuals from groups familiar with the projects, to include but not limited to: • MONUSCO’s Stabilization Unit (Goma only) • STAREC team (Goma only) • Ministry of Planning (NK + SK if possible) • UN HABITAT (NK) • UNDP’s focal points for the Early Recovery cluster (NK + SK) • PSCRP’s partner NGOs (at least three amongst HEAL Africa, Alpha Ujuvi, PLD, DSF, SOPADI) • Members of Comités Locaux Permanents de Conciliation (CLPCs) having received training from PSCRP (NK only) Debrief after Fieldwork: The evaluation team will compose an informal de-brief for the COP and DCOP at the end of their field work, summarizing initial findings. This will be followed by a presentation in Goma to USAID of such initial findings. VI. DELIVERABLES AND MILESTONES OF EVALUATION The following is a schedule of tasks and anticipated duration of each: [To be finalized when the evaluators’ exact dates and availability is known.] month day key tasks total 39 | Page Sept. 15 background doc and analytical framework 16 background doc and analytical framework 17 18 19 travel to Goma and initial field briefing 20 finalize framework + produce instruments 21 interviews Goma and field visits 22 interviews Goma and field visits 23 interviews Goma and field visits 24 travel to Bukavu + work on lessons learnt 25 26 interviews Bukavu and field visits 27 interviews Bukavu and field visits 28 travel to Goma + work on lessons learned 29 work on lessons learnt 30 data analysis Oct. 1 produce draft findings 2 3 produce draft findings + submit to MSI 4 draft presentation to USAID 5 update presentation for USAID with MSI comments + send USAID 6 presentation of findings to USAID 7 travel back to Europe 8 draft report 9 10 draft report 11 draft report 12 draft report 13 send to MSI for comments 14 update final report 15 16 17 update final report and submit VII. REPORT The evaluation team will complete a draft report following the outline below and in line with the dates for milestones and deliverables, above. After review and comments by the COP, the evaluation team will present their draft findings and conclusions to USAID in Goma. They shall produce a re-draft incorporating USAID’s comments where appropriate. The final version of the report will follow the format below (page lengths by section illustrative only) and be no more than 30 pages in length, excluding the annexes: 40 | Page Report 1. Title page (1) 2. Table of Contents (1) 3. Acronyms (1) 4. Executive Summary (2) 5. Background (1-2) 6. Purpose of Evaluation (1) 7. Evaluation Methods (1) 8. Findings, Conclusions, and Recommendations (no more than 20 pages) This section should be organized around the TOR key issues and include findings, conclusions and recommendations for each. Annexes 1. Terms of reference 2. List of Meetings and Interviews 3. Any other relevant documents including baseline survey, new indicator memo, project update18 4. Success stories 18 The CoP later indicated to the evaluators that there was a mistake in this line of the SoW and that those three specific documents were not expected from the evaluators. 41 | Page North Kivu sites South Kivu sites Figure 2: Map of PSCRP community groups in North Kivu Figure 3: Map of PSCRP community groups in South Kivu 42 | Page 2. Changes to SoW and rationale 1. The SoW was slightly adapted early on into the process in terms of precise timeframe, given delays in deploying one of the evaluators due to flight issues; 2. There was a discrepancy in between various documents on the exact phrasing of the 1st question. The 1st question’s language as referenced in the evaluators’ own terms of reference was “Did the strategies in the original design meet the requirements of the Task Order? Did they need revising during the course of PSCRP? If yes, why?” and the 1st question is thus treated that way in the evaluation report. 43 | Page 3. Annex to evaluation of Impact Baseline and End-line comparison for Purpose-level indicators Indicators in PMP Baseline End-line % of community members that believe a member of a marginalized group could serve on the decision making body of their local development committee Walungu Uvira Fizi Rutshuru Masisi Nyiragongo 73% 99% 99% 46% 47% 47% Walungu Uvira Fizi Rutshuru Masisi Nyiragongo 61% 84% 83% 72% 54% 100% % of community members who indicate freedom to express ideas and concerns in community meetings Walungu Uvira Fizi Rutshuru Masisi Nyiragongo 81% 64% 52% 43% 36% 36% Walungu Uvira Fizi Rutshuru Masisi Nyiragongo 86% 52% 61% 62% 53% 77% % of observed community meetings where members of marginalized groups present idea or concerns Walungu Uvira Fizi Rutshuru Masisi Nyiragongo 31% 32% 29% 20% 15% 15% Walungu Uvira Fizi Rutshuru Masisi Nyiragongo unkn own Number of community based reconciliation projects completed with USG assistance (no baseline: PMP) 30 17 (covering a total of 29 communities) During the initial debrief of the findings of the evaluation, USAID asked whether indicators’ targets might have been underestimated at project design or start. In the project PMP, there are no targets for the above indicators; the PMP only refers to the baseline survey. The baseline survey did measure the baseline situation of the above indicators, and those figures are reported above. As far as the evaluators are aware, no targets were ever set for those indicators after the baseline was conducted. The evaluators understand that the expectation was that the values for those indicators will increase during the course of project, but it was not clarified what the order of this increase was expected to be. It is thus not possible to make a judgment as to whether or not the targets for the above indicators were underestimated. 44 | Page 4. Annex to Question 1: Comparison of select key strategic documents Reference documents Strategic directions Task order USAID-MSI PSCRP’s overall conflict resolution strategy Practices Principles, positioning, definitions Tangible short-term peace dividends should promote foundations for peace. Location: Ituri District of Orientale Province – North Kivu Province – South Kivu Province; Ituri being the most stable. Community-focused nature of the support and activity Community-based development focuses on local actors – participatory and inclusive decision￾making processes and structures + inclusive development and mobilization to have communities prioritize their own needs and respond to them Socio-economic recovery is considered primary Do-no-harm principles Targeted communities criteria: high concentration of ex-combatants (ref UNDP, MONUC) Target beneficiaries: communities, and more specifically returnees, ex-combatants, marginalized, survivors of SGBV, vulnerables Ex-combatants: the ones participating in the disengagement process who are ineligible for the national DDR Program The understanding of livelihoods covers - but is not limited to - human capital (education and training, sensitization, agricultural practices), finance capital (income generating activities) Develop a mechanism for rapid response to targets of opportunity to mitigate conflict and advance peace and stability Stabilization activities: can be implemented in Socio-economic activities constitute a peace dividend and contributes to social cohesion (result 1 of the PSCRP) Peace building as a cross-cutting issue Location: North and South Kivu Provinces Community focused Participatory and inclusive principles applied in elections, prioritization processes, labor force contribution to infrastructure projects – monitored through General Assemblies of accountability Socioeconomic recovery as contributing to social cohesion Conflict sensitivity Target beneficiaries: broad understanding of ex￾combatants and no partition among this group has been done The understanding of livelihoods activities covered human capital (training in the field of peace and finance capital (mills as income generating activities) 45 | Page Reference documents Strategic directions Task order USAID-MSI PSCRP’s overall conflict resolution strategy Practices the short to medium term + directly contribute to the mitigation and prevention of renewed armed or destabilizing conflict + support return and reintegration of refugees, IDPs and ex￾combatants DDR reintegration activities: support the psycho-social and socio-economic reintegration of ex-combatants Supra-ordinate objectives (MDGs, human rights) Achieve post-conflict reconciliation and recovery 4 IR – key words: productive infrastructure – community empowerment and transformation – participatory local governance – reintegration of marginalized people Human rights: reintegration of ex-combatants, including child soldiers Productive infrastructure included schools and health centers Community empowerment: peace committees and local peace development; coordination meetings; community development plans Local participative governance : General Assembly of accountability – Coordination meetings Reintegration through sensitization and integration in all projects activities Child soldiers not considered as a specific target Alignment (with partner needs, priorities, STAREC, I￾SSSS) Complement and support the stabilization programs of MONUC, other donors, and other US-government agencies in Eastern Congo Administrative alignment (electoral delineations) involvement with stabilization programs of MONUC Alignment with local partners needs: needs in management capacities addressed from Sept 2010 Incorporation of specific LP core competencies Targeted communities in Uvira 46 | Page Reference documents Strategic directions Task order USAID-MSI PSCRP’s overall conflict resolution strategy Practices territory outside of STAREC strategic axes Aligned with the new proposed strategic plan from MONUSCO Harmonisation (coordination with others) Building on IA current work in community reintegration and conflict mitigation Coordination with USAID programs, with other donors and programs (ARD, NRC, IRC, Care International, Caritas, Save the Children) to promote project synergy, prevent duplication, leverage project funds Extensive coordination on DDR activities Coordination with MONUC (UNOPS, WFP) is crucial vis a vis military stabilization Demand-driven coordination Built on Alert’s initial local partners and experience in community reintegration and conflict mitigation (Uvira) UN Early Recovery Cluster Coordinated with I-SSSS programs - UNHabitat Cooperation system (performance and cooperation capacities of stakeholders, ownership) Broad community ownership Use Alert to ensure a rapid start-up Work closely with local communities (peace committees, provincially based local NGOs) Involve local authorities (district or province) Coordination between the two organisations – MSI and AI – and between the project and its local partners: see Changes in the performance measurement plan: 2 new indicators are incorporated at the level of result 2, including a process indicator and an “ownership” indicator Support to socio-economic activities through Local Development Committees Support to conflict management / transformation through “Noyaux de Paix” and Conciliation and Mediation Committees Convergence points: coordination meetings + local partners Alert experience incorporated within the project Cooperation with Local authorities at groupement and chefferie levels Formulation of Local Development Plans, with 47 | Page Reference documents Strategic directions Task order USAID-MSI PSCRP’s overall conflict resolution strategy Practices validation by the population and Local Authorities Training of customary leaders in land conflict management in North Kivu (together with UNHabitat) Regional perspective (positioning and relationship in relation with Eastern Congo peace building agenda) Conflict analysis originated at the community level (by the community and for the community) Conflict analysis at coordination meetings Contributes to strategic planning covering Eastern Congo Implementation guidelines Implementation should be forward learning and catalytic Communities are actively engaged in peace building and reconciliation and apply this learning in the context of participatory self￾helped projects Components: community-based stabilization activities + DDR reintegration activities + governance Geographic focus on 30 communities in North and South Kivu to ensure programmatic efficiency + develop a critical mass of communities to advocate at local authorities level Rapid start-up and implementation Marginalized groups are directly engaged in identification of priorities +design + implementation Provide support to develop community development plans The support at community level through separated committees, avoiding tensions between conflict resolution responsibilities and resource management responsibilities Coordination meetings between Peace Committees and Local Development Committees. Their reports are expected to become tools for participatory monitoring Suggests the development of tools for conflict analysis, project monitoring and activities coordination Suggests that a final KAP survey will be undertaken at the end of the project Communities have contributed to the implementation of local development projects with a peace building approach Close monitoring of elections of committees’ members – and sometimes establishment of quotas – to reach representation of all minorities and marginalized Community-based activities covered stabilization, (re)integration and governance Advocacy at local authorities level Marginalized groups have been directly engaged in identification of priorities + design + implementation CDPs 48 | Page 5. Annex for Question 3: Evolution of schooling levels of marginalized groups Populations Pygmies Ex-combatants Returnees Girls Sex Girls Boys Girls Boys Girls Boys Year 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 % 2011 % Sange I 1 2 2 3 3 2 8 15 0 0 5 3 10 5.5 21 8.3 Sange II 0 0 0 0 0 0 1 3 26 31 9 11 75 57 98 57 Sange III 0 1 1 3 4 6 6 8 13 15 18 18 34 29 73 41 Mutarule I 14 14 13 14 15 20 10 13 5 0 8 0 24 26 27 38 Bwegera 0 0 1 1 5 6 10 11 12 15 18 21 51 28 123 41 Total 15 17 17 21 27 34 35 50 56 61 58 53 194 28 342 35 13.33 23.53 25.93 42.85 8.93 -8.62 In % of total student population 2010 2011 Evolution 2010 2011 Evolution Total students 700 972 0.39 700 972 0.39 Pygmies 0.05 0.04 -0.96 32 38 0.19 ex-combatants 0.089 0.086 -0.91 62 84 0.35 Returnees 0.16 0.12 -0.28 114 114 0.00 Girls 0.28 0.35 0.27 Boys 0.72 0.65 -0.10 49 | Page 6. Annex to Question 4: Effectiveness per Result This annex presents for each Result a table with the agreed indicators, set targets and achieved figures. Given that some targets were not set in the PMP but only in project’s reports, and that some targeted were increased over the course of the project, the tables below display targets and achievements against (a) the PMP and (b) recent project’s reports. Result 1: Increased social and economic peace dividends in targeted communities Targets and achievements for Result 1 indicators Indicators Targets Achievements 1. Number of USG-supported activities that demonstrate the positive impact of a peace process through the demonstration of tangible, practical benefits PMP: TBD 58 (N/A) Recent reports: 45 58 (129% of target) 2. Number of people reached through completed USG supported community-based reconciliation projects PMP: 100,000 278,685 (278% of target) Recent reports: 278,685 278,685 (100% of target) The PMP did not set a target for the 1st indicator. For the 2nd indicator, a very low target was set. The early documents of the project always made reference to a number of communities, rather than to a number of beneficiaries. As seen during the course of the project, the number of inhabitants per community in such a project varies significantly from one community to another. One can say that the target in the PMP for the 2nd indicator was set at a reasonable level when considering the average number of inhabitants per community that this required and when comparing it with those of other CDD-type projects. The evaluators do not believe that this target was underestimated. Result 2: Strengthened social mechanisms for community reconciliation, conflict mediation … Targets and achievements for Result 2 indicators Indicators Targets Achievements 1. % of people who demonstrate increased understanding of conflict mitigation/resolution issues after training PMP: TBD 57% (N/A) Recent reports: 80%19 57% (71% of target) 2. Number of peace-building structures established or strengthened with USG assistance that demonstrate active engagement of conflict affected citizens in peace and/or reconciliation processes PMP: N/A 25 (N/A) Recent reports: 25 25 (100% of target) 3. Average number of community members who attend meetings of community organizations PMP: N/A 34 (N/A) Recent reports: 24 34 (142% of target) 4. Number of dialogues / coordination meetings in each community PMP: N/A When established: 40 114 (285% of target) Recent reports: 100 114 (114% of target) 19 This is also due to a change in the measuring instrument. Reports at the end of year one were mostly based on information provided by one local partner. Because of the lack of reliability of information provided by other partners, MSI decided to proceed to a self-assessment by members of community structures of their own progress. MSI accepts that this change of instrument is less than ideal but was necessary in order to capture information relevant to this indicator in a reliable manner instead of merely registering information of questionable value. 50 | Page Indicator 1 was not achieved in full. Indicator 2 was achieved in full. Indicators 3 and 4 were overachieved. Targets set for indicators 3 and 4 appear reasonable, but possibly conservative. For indicator 3, the actual number of members in the community ‘organizations’ (the peace committees in this case) obviously has a significant effect on the performance of this indicator. When reviewing project documents, the evaluators believe that many peace committees might include more members than was initially planned, which might in part explain why this target was overachieved. In any case, in future projects it might be more relevant to express this indicator in terms of percentage rather than as an absolute value. For indicator 4, this indicator was not initially present in the PMP, so no target was initially set. This activity was thought as an add-on to the project’s approach, and the project team might have been very conservative at first as to the likely number of meetings to expect. 0% 50% 100% Q4 2010 Q4 2011 2A: % of people who demonstrate increased understanding of conflict mitigation/resolution issues after training Target Achieved 0 20 40 Q4 2010 Q1 2011 Q2 2011 Q3 2011 Q4 2011 2B: Number of peace-building structures established or strengthened with USG assistance that demonstrate active engagement of conflict affected citizens in peace and/or reconciliation processes Target Achieved 0 10 20 30 40 Q4 2010 Q1 2011 Q2 2011 Q3 2011 Q4 2011 2C: Average number of community members who attend meetings of community organizations Target Achieved 0 20 40 60 80 100 120 Q4 2010 Q1 2011 Q2 2011 Q3 2011 Q4 2011 2D: Number of dialogues / coordination meetings in each community Target Achieved 51 | Page Result 3: Enhanced participatory local governance processes with community & civil society org. Targets and achievements for Result 3 indicators Indicators Targets Achievements 1. Number of non-governmental constituencies built or strengthened with USG assistance PMP: 6 34 (566% of target) Recent reports: 34 34 (100% of target) 2. Number of individuals trained in participatory local governance PMP: TBD 952 (N/A) Recent reports: TBD 952 (N/A) 3. Number of community development plans ratified by the community PMP: 30 17 (56% of target) (97% of target)20 Recent reports: 17 17 (100% or target) 4. % of community organizations receiving project grants that hold at least one accountability meeting with the community during the grant period PMP: 100% 100% (100% of target) Recent reports: 100% 100% (100% of target) 5. % of infrastructure projects initiated through project grants with formal local government support and/or agreement for project maintenance PMP: 80% 100% (125% of target) Recent reports: 100% 100% (100% of target) In terms of discussing whether indicators’ targets were underestimated at first, this applies here to indicator 1 and 5 only. For indicator 1, the fact that the achieved target is significantly higher than the set target as per the PMP is not due to the fact that the latter was underestimated. As discussed in the main text of the report, it is rather due to the fact that the project reported under this indicator units that as per the PMP were not expected to be accounted for here: the project also counted all the CDCs here, while the PMP was foreseeing that only the LNGO partners will be reported here. For indicator 5, the PMP took a slightly conservative but reasonable approach to setting a target that was overachieved. The evaluators do not believe that this is a case of underestimating the target as it does not appear unreasonable to assume at first that a small number of communities might not be able to achieve indicator 5. Result 4: Increased reintegration of marginalized people into targeted communities Targets and achievements for Result 4 indicators Indicators Targets Achievements 1. Number of ex-combatants, armed belligerents who complete USG-assisted transformational programs PMP: TBD 157 (N/A) Recent reports: 200 157 (78% of target) 2. Number of individuals from marginalized groups, excluding ex-combatants, participating in community development projects PMP: TBD 314 (N/A) Recent reports: 200 314 (157% of target) 3. % of members of community organizations who are from marginalized groups PMP: TBD 23% (N/A) Recent reports: 20% 23% (115% of target) The PMP did not set targets for the above indicators. It is thus not possible to discuss whether targets were underestimated at first, as there were none. 20 Some communities joined together to develop their plan, thus a total of 29 communities developed 17 plans, some jointly. The 30th community did not develop a plan. It however appears that it was agreed with USAID that the project will deliver only a limited SoW in that 30th community, not to include a development plan. 52 | Page 0 100 200 300 Q4 2010 Q2 2011 Q3 2011 Q4 2011 4A: Number of ex-combatants, armed belligerents who complete USG-assisted transformational programs Target Achieved 0 200 400 Q4 2010 Q2 2011 Q3 2011 Q4 2011 4B: Number of individuals from marginalized groups, excluding ex￾combatants, participating in community development projects Target Achieved 0% 10% 20% 30% Q4 2010 Q1 2011 Q2 2011 Q3 2011 Q4 2011 4C: % of members of community organizations who are from marginalized groups Target Achieved 53 | Page 7. Annex to Question 6: Changes in social dynamics and community cohesion Sources: « Rapports de mission de suivi des Noyaux de paix » + « Tableau de suivi des indicateurs paix/violence » + staff workshop + « tableaux de changement communautaires ». Changes in social dynamics and Community cohesion Relations with Pygmies Gender relations Uvira Pygmies have become proactive with the support of PC members to advocate for their access to labor Potential of women to share responsibilities with men outside the family nucleus, including on security matters, has been highlightened Solidarity towards the vulnerable and marginalized: one-year process of sensitization campaigns initiated by committees’ members (PC + LDC) to stimulate access to school for Pygmies + financial contribution to support schooling fees Access to labor market in the local administration (chef d’avenue) Access to education Access to land Access to labor market in the local administration (chef d’avenue) Masisi Re-emergence of some signs of solidarity between women from different ethnic groups (visits at the maternity) Enlargement of solidarity network Nyiragongo Rupture with traditional practices for improved health care for women: advocacy from LDC and PC members to have Health Committees sensitizing women to give birth at the hospital Improved consideration from men for the women’s health Baraka Women field workload shared with men Sharing other ethnic group traditions (milk consumption with Banaymulenge Cohabitation dans l’exécution de petits projets Improved equity between men and women for family support Unchanged Relations with Ethnic groups / other identities Relation with ex-combatants Walungu Stigmatization of ex-combatants suspected of committing rapes, robbery, etc. Stigmatization of children coming from sexual violence No access to education to women Mistrust Rutshuru Distrust vis a vis ex-combatants (but still sharing responsibilities within the LDC / PC) Disdain / Pygmées Frightness towards Tutsis Social isolation Nyiragongo Women victims of sexual violence leaving apart from men Unmarried women not registered in school records Stigmatization of ex-combatants Stigmatization of child soldiers with regards to vandalism Isolated children Isolated ex￾combatants No access to education for women Limits the women civic participation 54 | Page 8. Annex to Question 8 1. Cooperation with Local Authorities and other conflict management entities July 2011 / March 2011 Walungu Ruzizi Baraka Izege Mulamba Kaniola Bwegera Mutarule Sange Kalunja Mazungum uzo & Mshimba. Number of cases introduced in the 3 last months 4 / 7 (3 settled) 8 / 12 (6 settled) 20 / 17 (13 settled) Members not available / 4 (4 settled) 4 / 4 (4 settled) 14 / 4 (4 settled) 5 / 4 (2 settled) 5 / 3 (2 settled) Number of cases transferred from CMC to other entities (last 3 months) 0 / 2 (including 1 between cultivator and pastoralist ) 1 to customary court at “groupeme nt” level 1 to customary court at chefferie level / 1 to the Mwami 1 0 0 / 1 1 to the Cité which then referred to the judiciary – between a group of cultivators and 1 pastoralist / 0 1 case to the police / 1 4 / 1 Number of cases transferred from LA to CMC (3 last months) 1 / 3 (including 1 between cultivator and pastoralist ) 1 from PNC 3 from chef de groupemen t / 0 0 2 0 / 3 1 from Bureau de la cité / 2 1 (power struggle between leaders of two communiti es) / 2 1 / 0 Number of cases transferred from the judiciary to CMC (3 last months) 0 1 0 / 0 0 0 / 0 Are cases documented within the “Noyaux de paix” or CMC No (LP) No (LP) No (LP) No (only record book) Only record book, by the LP Only record book No Unexpected Meetings The chef Bad 55 | Page Walungu Ruzizi Baraka Izege Mulamba Kaniola Bwegera Mutarule Sange Kalunja Mazungum uzo & Mshimba. effects by LA (Chef de groupemen t et other developme nt entities) in the « paillotte de paix » – contributes to strengthen the visibility of the CMC de groupemen t appreciates the support from the CMC in dealing with minor cases – Rule of law has been strengthen ed as people do not refer to the police anymore cohesion among the members of the baraza => affected negatively the mediation capacity of the NP Frustration regarding funds availability compared to the CLD – CLD and CMC fight for leadership, has been solved through the involveme nt of the CMC in the manageme nt of the mills Level of cooperation Chef de groupemen t member of CLD and CMC / mediation process involves the Chef and the comité des sages Resistance from chef de groupemen t – has been mitigated by the use of the paillotte de paix Worries expressed by customary Chef de groupemen t takes part to CMC meetings Collaborati on between the PC, LDC, the police and the Local Authorities Communit y leaders are members of Baraza Tensions with chef de chefferie regarding competenc ies on dowry cases – request the RAS The Chef of Kalunja participate s to mediation sessions Chef de groupemen t is member of the NP 56 | Page Walungu Ruzizi Baraka Izege Mulamba Kaniola Bwegera Mutarule Sange Kalunja Mazungum uzo & Mshimba. courts regarding competenc ies in law document to refer to on the Baraza competenc ies Sources: rapport de suivi des noyaux de paix July 2011 / March 2011 Rutshuru Nyirago. Masisi Mabungo I et II Ntamuge nga I et II Bugara/B unyenzy Kibumba Karuba Rushoga Macha I Mubam. Kingi Kimoka Number of cases introduced in the 3 last months 8 (7 settled) / 6 (5 settled) 22 (19 settled) 5 (4 settled) / 9 (9 settled) 3 (2 settled) 7 (4 settled) 9 (5 settled) 4 (2 settled) 8 (5 settled) / 15 (15 settled) 7 (3 settled) With mediatio n settlemen t signed by both parties Number of cases transferred from CMC to other entities (last 3 months) 1 / 1 1 0 2 1 0 1 1 Number of cases transferred from LA to CMC (3 last months) 1 1 0 3 3 0 1 2 Number of cases transferred from the judiciary to CMC (3 last months) 0 0 Are cases documented within the Book record / No No registre Book record / No Book record Book record Book record Book record Book record Book record 57 | Page Rutshuru Nyirago. Masisi Mabungo I et II Ntamuge nga I et II Bugara/B unyenzy Kibumba Karuba Rushoga Macha I Mubam. Kingi Kimoka “Noyaux de paix” or CMC registre registre Unexpected effects Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Chef de localité consults CMC Chefs coutumie rs se sentent menacés par crainte de voir perte de pouvoir des comités des sages Level of cooperation OK OK Sources: rapport de suivi des noyaux de paix 2. Did the project lead to create or reform institutions or political mechanisms aiming at answering to people demands and addressing injustice? Sources = CAP survey – discussions focus groups Perception des CP / adresser les injustices Capacités des CLD / répondre aux demandes Walungu Démobiilisés considèrent que leurs demandes ne sont pas consiédérées par les DLC (pas intégrés dans la work force) – La défaillance des dirigeants de CLD - devoir de redevabilité des CLD n’est pas perçu comme une obligation / population Uvira Démobilisés considèrent leurs doléances pas prises en compte (exception à Bwegera) – femmes et pygmées perçoivent leurs demandes pas considérées – pygmées ont une appréciation positive des CLD / leur participation aux travaux Démobilisés et Burundais apprécient positivement le travail des CLD pour esl intégrer à la workforce des chantiers Rôle et activités des CLD mal connus apr les femmes 58 | Page Sources = CAP survey – discussions focus groups Perception des CP / adresser les injustices Capacités des CLD / répondre aux demandes devoir de redevabilité des CLD n’est pas perçu comme une obligation / population Masisi satisfaits des NP car eprmet de vivre en paix – NP et CLD facilitent le dialogue ts les marginalisés pvt exposer leurs doléances, sauf les tutsis – seuls les CLD arrivent à impliquer els marginalisés et comprendre leurs doléances - partout devoir de redevabilité des CLD n’est pas perçu comme une obligation / population Nyiragongo Sensibilisation des NP a aidé à comprendre comment vivre pacifiquement – peur d’accéder aux paillottes de paix placées à côté des bureaux de la police et l’administrateur conscients du devoir de redevabilité des CLD / population – CLD ont renforcé l’arbre à palabre traditionnel brisé par la guerre 3. Did the project stimulate communities to take upon themselves initiatives to prevent and settle conflicts? Sources = rapport de mission de suivi des noyaux de paix + tableau de suivi des indicateurs de paix Réduire les facteurs de dissension Renforcer les facteurs d’entente Adresser les causes des conflits Ruzizi (Mutarule) Délimitation des parcelles => réduction des conflits entre agriculteurs et éleveurs Baraka NP est intervenu pour faciliter la cession de champs aux marginalisés considérés comme étrangers (Bazoba) ; 30$ to 50$ / plot ; concerned 8 households Nyiragongo Construction d’un « pont » pour relier els communautés de Kibiriga et Kalangala, CLD et noyaux de paix Masisi Sollicitation de la police pour sécuriser un axe stratégique pour la circulation des 59 | Page Sources = rapport de mission de suivi des noyaux de paix + tableau de suivi des indicateurs de paix Réduire les facteurs de dissension Renforcer les facteurs d’entente Adresser les causes des conflits populations des hauts plateaux (crainte que les arrestations arbitraires et tracasseries policières ne dégénèrent les relations entre population de Masisi et celles des hauts plateaux) Idem auprès de militaires pour sécuriser un quartier – puis pbme entre population et militaires plusieurs cas référés à la paillotte de paix – a résulté notamment dans une médiation entre un FERDC et un villageois Uvira Analyse des conflits CP + CLD enter agriculteurs et éleveurs -> propositions à l’ensemble des chefs communautaires pour gérer les espaces de culture et d’élevage -> sensibilisation des agriculteurs par les chefs communautaires + CLD + CP -> draft d’un code communautaire présenté à la communauté, adoption à la réunion de coordination pour soumission en AG de redevabilité -> libération des espaces effective en septembre 2011 60 | Page 4. Did the project contribute to improvement of the security situation? Tableau de suivi des indicateurs de paix + KAP Survey + Nyiragongo Plaidoyer auprès des autorités locales suite à des vols à main armée => intervention de patrouilles de militaires Masisi Sensibilisation de la population sur les procédures favorisant la protection contre les violations de droit humain + plaidoyer par les CP et CLD auprès des autorités (politiques, administratives, policières) pour diminuer les tensions entre les autorités et la population Collaboration entre CP et AL pour sensibiliser la population à la sécurité suite à un vol au centre de santé Sollicitation de la police pour sécuriser un axe stratégique pour la circulation des populations des hauts palteaux Construction école à Kimoka réduit les distances et l’insécurité Rutshuru CP+CLD font le plaidoyer auprès de chef de groupement face à l’insécurité – réduction insécurité sur les axes Rubare et Kako Uvira Construction école à Bwegera réduit les distances et l’insécurité 5. Did the project improve relationship between antagonist parties within the community? Sources : voir CAP survey, tableau sensibilité au conflit, TCC Oui Non Conflits individuels : Règlement par médiation Maintien de relations pacifiques Conflits autorités-population : arrestations arbitraires Plaidoyer auprès du commandant de police et/ou AL (Masisi, Rutshuru) ; restauration de l’autorité de l’Etat Conflits intrafamiliaux Sensibilisations sur les questions de genre, notamment concernant les successions (Sange I et II) : restauration de l’Etat de droit 61 | Page 9. Annex to Question 14: Good practices & lessons learnt on community recovery The chart below presents the lessons learnt and good practices identified for the community recovery component, by step of the community process. Step Positives Challenges 1 Profiling/ identifying communities - Staff involvement in the process allowed them to gain a good knowledge of the communities before starting project’s implementation per se - Subcontracting of enumerators allowed to cover a large number of sites more rapidly than if it had been done internally only Tools: profiling checklist, report template Process not sufficiently formal initially. Various staff came to the process with their own experiences and vision, and this clashed at time, or slowed down the process and resulted in slight differences from one community to another. There would have been a need for standardized tools from the onset. Not enough time was allowed for the entire process which took a lot of time given the geographical spread of the area covered 2 Selecting communities - Communities selected are in locations where it is efficient to work - Clear decision early on about the maximum size of the community in terms of administrative entity, and utilization of administrative boundaries as basis to define communities - Involvement of local leaders for the final selection of communities, as during profiling some data was captured only at an administrative level that was too large Tools: scoring chart - All potential communities might not have had an equitable chance of being selected, given some of the criteria used - Local leaders might find themselves in a conflict of interest when weighting in the selection process, something that the project might need to manage better 3 KAP surveys - Allowed to have a baseline to be later used to measure changes at the outcome level (however no control, so no attribution possible) - Feedback loop to the community on the findings - It was somewhat difficult at times to obtain genuine answers from respondents, as they had started to learn about the project during the profiling and thus were sometimes giving answers that they thought were likely to affect how the project was going to support them - Data was under-utilized during the life of the project - In summary, it might be better to implement a simpler KAP survey, earlier on and that could also serve the purpose of the profiling exercise 62 | Page Step Positives Challenges 4 Setup of CDCs/CLDs and training - Process fast and efficient based on the possibility to ‘validate’ pre￾existing committees - Relatively simplified election process - Difficult in multi-ethnic communities to combine ethnic, geographic and sex representativeness of the committee - Need to better understand power dynamics within the community to appropriately prevent elite capture (chiefs, Churches), especially when pre￾existing committees are validated - Need for more facilitated discussions within the community about the criteria of a ‘good committee member’ before candidates identify themselves and community votes - Initial training for CDCs/CLDs insufficient and might need to be improved in terms of quality. All the training objectives might not have been reached at first. Speed of training was too ambitious - Need for a functional adult literacy training program for committee members - Insufficient utilization of exchange visits between committees as a means of training 5 Needs identification - Process was transparent and fast - Project used a methodology they called ‘enlightened choice’ based on comparative ranking to avoid elite/male capture in community priority selection - Hard to manage priority needs that are more expensive than the maximum budget available - Hard to enforce the rule requiring that communities select ‘complete’ subprojects (if a school, then it needs to come with latrines, furniture, etc.) 6 Community subproject design - Used as an opportunity to train committee members and enhance ownership - Used later as a tool for community control during implementation Tools: project proposal template - Too many forms which were too complex. This resulted in the project’s staff filling them, instead of this being a participatory process with the CDC/CLD - Many delays while the overall project timeframe was very tight - Committees would have needed more training (and literacy training) before this step starts - It would have been better if the process had been more participatory, with more community control 63 | Page Step Positives Challenges 7 Community subproject approval - By MSI office in Goma fast - By USAID fast - After some time, the number and complexity of MSI’s forms were streamlined and it became efficient - By MSI home office slow for the first few proposals processed because project staff were struggling in gathering all the required paperwork and in filling it in full. Staff were frustrated by back and forth with home office 8 Tendering and contracting - contractual arrangements by which there are two contracts, one between the CDC/CLD and MSI, and one between the CDC/CLD and the contractor - learning opportunity for CLD - Biding contractors in the most remote areas struggled to comply with tender requirements - Training might have been needed - Despite tender requirements and safeguards, some selected contractors still had capacity or honesty issues - There were no incentives/disincentives for contractors to finish the job well and on time (there is no bonus clause, and the CDC/CLD have reportedly been reluctant to enforce penalty clauses) - CDCs/CLDs should be trained in ‘good governance’ before the tenders are issued, as some CDC members might have been running for a specific contractor (‘insider dealing’ issues) - A fixed-cost, quality-only type of competition might be better suited to the environment, given the impossibility to keep secret from the contractors the budget the CDCs/CLDs had available 9 Payments - Finance department was very fast, simple21 - Payment of contractors through advances allowed smaller contractors to bid, and allowed faster start and implementation of works - Simple and effective database - CDCs/CLDs felt unhappy about not being able to manage the cash themselves before it is transferred to the contractors. This required a lot of discussions - The fact that the last milestone of construction contracts was not an advance created cash flow problems for contractors to address issues identified during the provisional handover. This could be addressed by splitting this further into 2 milestones 21 Note this is a particular achievement given that payment approvals and disbursement delays are regularly reported as a key bottleneck and problem in CDD programs. 64 | Page Step Positives Challenges 10 Community subprojects implementat ion - Completed community subprojects are generally perceived as of quality - Very relevant to have a small subproject before the larger one, to be used as a training field for the committees, and to allow communities to ‘test’ the people they have elected to the committee - Initial time planned was typically too short for realistic community subprojects implementation - Need for further standardized construction processes to ease monitoring 11 Community subprojects monitoring - Allowing a small budget out of the block grant for community monitoring of the contractors’ work allowed monitoring to take place, as well as community engagement - CLD quite involved in controlling contractors (however sometimes lack the technical skills to do that as appropriately as possible) - Community probably needs more training to be able to carry monitoring role effectively - Need for further, systematized training and empowerment of community members to monitor construction sites. Mandatory community meetings should also be organized on this topic 12 Handover of community subprojects outputs - Time between provisional and final handover by contractors essential - Used as an opportunity for community advocacy towards local leaders and line ministries and to push those to make commitments for additional support - Used as an opportunity to showcase equity and the involvement of marginalized groups - Not enough was done to engage civil servants: Sometimes, difficulty to engage officials in this process as they ask for perdiems, something the project cannot provide 13 Maintenance plans - Maintenance committees key, and good link with official sector￾specific committees such as COPA and CODESA Tools: maintenance plan template - Need to involve mid-local level line ministries’ offices as well as Churches (in the case of Church-owned/operated facilities) - Need to approach equity in a broader way (not just financial access) – to include topics such as staff recruitment, etc. 14 Expanded community development plans - Allowed CDC to understand that its role continues beyond project’s end - Provides a common vision for the future, as the plans were done in an inclusive manner - Used as an opportunity for self￾evaluation of the CDC - Allows to empower the CDC for advocacy towards authorities and duty bearers - Done too late in the overall project process, which did not allow sufficient facilitated engagement with relevant external stakeholders after the completion of the plans - Local authorities had a hard time to accept that those plans include a section on ‘good governance’, that they perceive as a threat 65 | Page 10. Evaluation design matrix Questions → Design considerations → Methods / Sources → Data Analysis → Answers Evaluation questions Type of answer / evidence needed: description / comparison / cause & effect Methods for data collection: records, structured observation, key informants, survey Sampling or selection approach (if needed) Data analysis methods: frequency distributions, trend analysis, content analysis, cross- Method Data source tabulations, etc. 1. Did the strategies in the original design meet the requirements of the Task Order? Did they need revising? If yes, why? Comparison Task Order vs. work plan and work plan vs. implementation practices Document review interviews Task Order Work plan Quarterly reports Project staff Latest 4 quarterly reports only due to time constraint Content analysis 2. Were the sharing of roles & responsibilities between MSI, IA, local partners, and the overall project architecture suited to PSCRP’s objectives? Comparison initial and current organizational charts, comparison expected roles and responsibilities with actual practices Document review interviews Organizational charts Interaction graph Project staff Local partner staff Only 3 Local partners were interviewed due to time constraint Content analysis 3. Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? Description of proportional access to project benefits per category of intended beneficiaries and for other stakeholders Document review interviews Task Order Project reports Project staff Local partner staff Communities Only 3 Local partners and 4 communities were interviewed due to time constraint Content analysis 4. To what extent were the four intended results achieved, in comparison to project indicators? (detail per result)? Comparison of targets with achievements, per indicator Document review Interview indicator chart Project staff Content analysis 66 | Page Questions → Design considerations → Methods / Sources → Data Analysis → Answers Evaluation questions Type of answer / evidence needed: description / comparison / cause & effect Methods for data collection: records, structured observation, key informants, survey Sampling or selection approach (if needed) Data analysis methods: frequency distributions, trend analysis, content analysis, cross- Method Data source tabulations, etc. 5. Were any community interventions more successful than others? If so, what appears to have contributed to disparate levels of effectiveness? Was there unforeseen negative impact? Comparison of interventions with each other and from communities to communities Document review Interviews Group discussion Work plan Project staff Content analysis, cross-tabulation 6. Were changes in social dynamics or cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? Comparison of situation in communities at project start and at time of evaluation, analysis of determinants of sustainability Document review Interviews Group discussion Project reports Project staff Communities Only 4 communities were interviewed due to time constraint Content and trend analysis 7. Was the theory of change, as expressed in International Alert’s global conflict resolution strategy, adapted and relevant to the challenges of the security and policy environment? Comparison of logic underlying IA’s memo with the environment Group analysis Overall conflict strategy Project staff Content analysis 67 | Page Questions → Design considerations → Methods / Sources → Data Analysis → Answers Evaluation questions Type of answer / evidence needed: description / comparison / cause & effect Methods for data collection: records, structured observation, key informants, survey Sampling or selection approach (if needed) Data analysis methods: frequency distributions, trend analysis, content analysis, cross- Method Data source tabulations, etc. 8. Did the project contribute overall to reconciliation and conflict transformation in targeted communities? How effective were its strategies and achievements? What was the impact (positive or negative, short or long term) of the project’s various components on peace-building at community level? Comparison between situation at start of project and at time of evaluation. Analysis of project’s tools and processes Document review Interviews Project reports Project staff communities Only 4 communities were interviewed due to time constraint, as well as only a sample of project reports Content and trend analysis 9. Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? Analysis of determinants of sustainability Group discussion Indicators’ chart Project staff Content analysis 10. Was the project coherent with the community-related components of I-SSSS and STAREC strategies? Comparison of those strategies with project design and implementation Document review Interviews Task Order Work plan I-SSSS and STAREC program strategies MONUSC, STAREC and Ministry of Planning staff Project staff Content analysis 68 | Page Questions → Design considerations → Methods / Sources → Data Analysis → Answers Evaluation questions Type of answer / evidence needed: description / comparison / cause & effect Methods for data collection: records, structured observation, key informants, survey Sampling or selection approach (if needed) Data analysis methods: frequency distributions, trend analysis, content analysis, cross- Method Data source tabulations, etc. 11. Did it add value to the other components? Comparison of those strategies with project design and implementation Document review Interviews As above Content analysis 12. Was the combination of reconciliation with recovery an effective strategy for stabilization (i) in targeted communities and (ii) beyond those communities? Could specific strengths and weaknesses be identified? Comparison of outputs and outcomes achieved with an estimation of what would have been achieved should the components not have been combined. Identification of determinants Document review Task Order Work plan Quarterly reports Latest 4 quarterly reports only due to time constraint Content analysis 13. Was the engagement with CLPCs in North Kivu effective, with regards to project objectives and to CLPCs’ own training needs? Comparison of activities with CLPCs with project objectives and contribution of those activities to indicators. Comparison of delivered training with training assessment Document review interviews Task Order Indicators’ chart STAREC team CLPC members Report on training needs Report from training sessions Only 1 member of 1 CLPC could be interviewed due to time constraint Content analysis 69 | Page Questions → Design considerations → Methods / Sources → Data Analysis → Answers Evaluation questions Type of answer / evidence needed: description / comparison / cause & effect Methods for data collection: records, structured observation, key informants, survey Sampling or selection approach (if needed) Data analysis methods: frequency distributions, trend analysis, content analysis, cross- Method Data source tabulations, etc. 14. Are there areas where PSCRP can be deemed to have reached good practices, which could be turned into practical tools or guidelines for future projects? Are they specific to eastern Congo or could they be adapted to different contexts? (shortened) Analysis of project’s performance and challenges, identification of tools or practices related to those Comparison with other key similar projects and their tools and practices Suggestions of tools to standardize / finalize Document review Interviews Group discussion Tools used and developed under PSCRP Websites of other projects Project staff Content analysis 15. Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming Identification of causal link between specific project’s practices and changes in terms of reconciliation and reintegration Document review group discussion Project’s reports Project staff Content analysis 16. Are there scalable elements in this project? Which ones? Comparison of project scale and components with practices in other similar projects at-scale Document review Task Order Work Plan Websites of other projects Only 3 other projects were reviewed due to time constraints Content analysis 70 | Page 11. Meetings held September 24, initial briefing between evaluators September 24, initial briefing between the evaluators and the project CoP and DCoP September 25, two meetings with the project CoP and DCoP on detailed understanding of SoW and on methodology September 26, meeting with MnE/Knowledge management staff of the project on data and community profile September 26, meeting with CoP and DCoP on question 2 September 27, call with MSI HQ Conflict and evaluation specialist September 28, meeting with South Kivu Program delivery manager, and then with the International Alert project staff in South Kivu, on Questions 2 and 3 September 29, meetings with key informants in South Kivu: Provincial Minister of planning and provincial coordinator of STAREC, UNDP program officer in charge of early recovery September 29, meeting with PLD, a LNGO partner of the project September 30, field visit to communities Sange I, Sange II and Bwegera in South Kivu and meetings with members of the peace committees and development committees September 30, meeting with International Alert’s program coordinator for Eastern DRC October 3, field visit to communities Karuba and Macha-Mubambiro in North Kivu and meetings with members of the peace committees and development committees October 3, meeting with the president of the CLPC on Sake October 4, staff workshop – see agenda below October 4, meeting with UN-Habitat October 5, meeting with MONUSCO stabilization unit October 5, meeting with STAREC North Kivu October 5, meeting with project local NGO partners Alpha Ujuvi and Heal Africa October 5, meeting with Provincial ministry of planning All meetings required in the evaluation SoW were held, with the exception of UNDP in North Kivu who were not easily available to meet. 71 | Page 12. Bibliography/sources of information Project documents and tools: Contract DFD-I-00-05-00252-00 Data quality assessment for the MSI standard indicators, USAID/DtS May 2011 Enquête CAP sur la promotion de la stabilisation et de l’intégration communautaire dans le territoire de Fizi, Communauté de Sange, August 2011 Formation pilote pour les membres des CLPC, report, July 2011 Enquête CAP sur la promotion de la stabilisation et de l’intégration communautaire dans le territoire d’Uvira, Communauté de Bwegera, August 2011 Grant Tool: Excel-based Data Management Solution, version as of September 19, 2011 Indicator performance tracking table; as of October 5, 2011 Monitoring des tensions communautaires, August 2011 Narratif du schéma des interactions sur terrain (undated) Note d’information sur le projet MSI de stabilisation et réintegration communautaire (undated) Notes on a new methodology, Community Based Conflict Transformation, Comparison between Community Driven Reconstruction and Community Driven Conflict Transformation (undated) Organigramme PSCRP (undated) Partage des responsabilités suivi subventions communautaires (undated) Plan de développement communautaire pour une durée de 5 ans, communauté de Bwegera, April 2011 Plan de développement communautaire pour une durée de 5 ans, communauté de Sange I et II, April 2011 Principes de base de la transformation des conflits et du genre, atelier de formation, rapport, May 2010 Project’s location map, North Kivu (untitled, undated) Project’s location map, South Kivu (untitled, undated) Project’s Excel database (untitled, undated) Project’s Excel conflict tracking sheet (untitled, undated) Project’s Excel global evaluation North Kivu and South Kivu sheets (untitled, undated) PSCRP – DRC Performance Management Plan, December 2009 PSCRP – DRC Work Plan, October 2009 (file name indicates it was revised in January 2010 but cover page title indicates October 2009) PSCRP fifth quarterly performance report, January 2011 (note this document is titled fourth report but it is understood that this is a typo and that this is indeed the 5th report) PSCRP fourth quarterly performance report, October 2010 PSCRP sixth quarterly performance report, April 2011 PSCRP seventh quarterly performance report, July 2011 PSCRP overall conflict resolution strategy (undated) Rapport d’analyse des besoins en formation des CLPC (undated, untitled) Rapport de la mission de suivi des noyaux des structures de transformation des conflits, deuxième étape (undated ; spelling mistakes in actual titles are corrected in this bibliography) 72 | Page Rapport des missions d’évaluation des structures de transformation des conflits au Sud Kivu (undated) Rapport d’évaluation des capacités acquises par membres des structures communautaires du projet PSCRP de la plaine/ Sud Kivu (undated) Rapport de la mission I sur le profilage des communautés dans l’axe Bukavu-Uvira-Fizi, December 2009 Résultat des discussions en focus group (undated) Résultat des réunions de coordination (undated) Rapport réunion facilitateurs sur le tableau de changement communautaire (undated) Schéma des interactions sur terrain (untitled, undated) Security briefing, September 2011 Tableau de suivi des indicateurs réduisant la violence et ceux favorisant la paix par communauté, Bwegera (undated) Tableau de suivi des indicateurs réduisant la violence et ceux favorisant la paix par communauté, Sange I et II (undated) Tableau sensibilité au conflit (undated) Technical proposal, Promote Stablization and Ex-combatant Reintegration in North and South Kivu in the Democratic Republic of the Congo, MSI, version dated 18 August 2009 Une maintenance des ouvrages communautaires sensible au conflit, Guide du facilitateur/animateur sur terrain (undated) Project’s email communications: Those emails were shared by the CoP with the evaluators to substantiate USAID’s concurrence with deviations from Task Order. Set of emails titled “FW: implication des ex-combattants dans le projet” Set of emails titled “FW: ISSSS” Set of emails titled “FW: UNHABITAT” External documents: Checklist for assessing USAID evaluation reports, V1.0, USAID, 2011 Evaluating development co-operation, summary of key norms and standard, second edition, OECD (undated) Glossary of key terms in evaluation and results based management, OECD, 2002 Integrated program framework 2009-2012, I-SSSS, January 2010 Mid-term evaluation report of the stabilization and community reintegration in the East of the DRC, DtS, July 2011 Ordonnance numéro 10/072 du 1er novembre 2010 modifiant et complétant l’ordonnance numéro 09/051 portant institution, organisation et fonctionnement d’un programme de stabilisation et de reconstruction des zones sortant des conflits armés, dénommé « programme STAREC », Journal officiel de la RDC, novembre 2010 Performance monitoring and evaluation, TIPS, constructing and evaluation report, USAID, 2010 73 | Page Programme de Stabilisation et de Reconstruction des Zones sortant des conflits armés, STAREC, June 2009 Request for Application Number SOL-623-11-000031 – Community Recovery and Livelihoods Project in the DRC, USAID, August 2011 Quatrième Composante de la Stratégie Internationale de Soutien à la Sécurité et la Stabilité de l’Est de la RDC – ISSSS, UN agencies, July 2010 Termes de référence pour les comités locaux permanents de conciliation (undated) Termes de référence pour les comités locaux permanents de conciliation (revised, 2010) Termes de référence des comités locaux permanents de conciliation (revised, 2011) Termes de référence, sous commission humanitaire et social, North Kivu STAREC (undated) Theories of change and indicator development in conflict management and mitigation, USAID, June 2010 Untitled booklet issued by the North Kivu STAREC technical commission collating a set of key documents on STAREC in North Kivu, February 2010 USAID evaluation policy, USAID, January 2011 Internet sites: World Bank CDD portal: http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTSOCIALDE VELOPMENT/EXTCDD/0,,menuPK:430167~pagePK:149018~piPK:149093~theSitePK:430161,00.html Websites of/on at-scale CDD projects: http://nspafghanistan.org/default.aspx?sel=16 http://web.worldbank.org/external/projects/main?Projectid=P073025&theSitePK=40941&piPK=73230& pagePK=64283627&menuPK=228424 http://www.dfid.gov.uk/Media-Room/Case-Studies/2010/DR-Congo-at-50-community-reconstruction/ http://www.rescue.org/our-work/post-conflict-development www.gsdrc.org All the above internet link were valid and functioning on October 6, 2011. 74 | Page 13. Glossary of terms Committee, CDC, CLD – Some project’s documents refer to CDC (Committee development committees) and other to CLD (in English: Local development committee) when referring to the committees in charge of the recovery component. Those terms can be understood interchangeably. Community – for the purpose of the project and for this evaluation report, a community is a geographic area that includes one or several human settlements, or forms part of a human settlement, and that is used as a unit for the delivery of the project. Community-driven development – this refers to a program methodology or approach that works with communities (as defined above) and provides them with technical and financial support to identify, select and implement one or several community projects (generally social infrastructure). This approach or methodology typically combines socio-economic, governance and social cohesion objectives. Conflict sensitivity – applying do-no-harm principles and using conflict analysis as a basis to implement strategy (IA’s definition). Conflict transformation – a process of conflict analysis, awareness and sensitivity (IA’s definition) Peace-building – The non-violent management of conflict (IA’s definition) Project – this refers to PSCRP Subproject – this refers to the projects implemented by the communities under grants from PSCRP. In most communities there was a small subproject and then a large subproject Theory of change – a set of hypotheses (if-then statements) and critical assumptions and risks underpinning the design for how a goal will be achieved. Reconciliation – non-violent cohabitation following a situation of distrust Recovery – Is a process of transformation that occurs after a crisis and that aims at establishing the conditions for self-sustaining equitable growth and human development while addressing key risk factors for the renewal of crisis. (Adapted from Governance and social development resource center) Stabilization – Stabilization is a rather loose notion which definition is typically very context-specific. In initial I-SSSS documents, it appeared to broadly make reference to a linear clear-hold-build approach to counterinsurgency. In the project Task Order, stabilization activities are described as those that ‘contribute to the mitigation and prevention of renewed armed or otherwise destabilizing conflict’. The Task Order further lists three relevant stabilization activities as follows: activities that have an immediate impact on the socio-economic recovery or communities; activities that reduce community tensions through conflict resolution and reconciliation; and activities that support the return and reintegration of refugees, IDPs and ex-combatants. For the purpose of this evaluation report those three types of activities will thus be understood as being stabilization activities, by convention. 75 | Page 14. Suggestions for future success story write-ups The evaluators did not identify specific success stories during the evaluation. 76 | Page 15. Copy of evaluation tools Detailed description of methodology22 Approach to data collection and analysis - Recognizing that resources for the evaluation are so limited, secondary sources need to be prioritized. Evaluators will make use of staff and secondary stakeholders’ interviews and pre￾existing documents, so as to be able to focus face-to-face time with communities on key questions that cannot be resolved through secondary sources; - Given the inability to visit all or even a large number of communities, evaluators will focus on a small purposive sample of communities only. This interface will be used to deepen understanding of key analyses generated from secondary sources that are relevant to this performance evaluation’s questions, and the other way around; - Given the importance of components’ integration in project strategy and evaluation’s questions, the consultants have decided to conduct all interviews and field visits jointly. Documents’ review was generally conducted by one consultant only for each given document, except for the most important ones. Overall analysis on key normative evaluation dimensions Relevance After having responded to the specific questions listed in the SoW and below, the evaluators will offer a summary judgment on the relevance of the project design to the context at the time of its design and any key evolution to date. This will include the security and policy environment. Impact The evaluators will provide a summary across all specific questions on project achieved impact against the stated purpose, using as an additional basis the purpose-level indicators. Effectiveness The evaluators will provide a summary across all specific questions on the effectiveness of the project design and implementation to deliver the intended results. Contribution to community stabilization and recovery Did the strategies in the original design meet the requirements of the Task Order? Did they need revising during the course of PSCRP? If yes, why? The evaluators’ understanding of the question is that ‘revising’ is to be understood both as formal and non-formal processes. Therefore it includes looking at whether strategic documents were added or modified over the course of the project, as well as looking at whether practices during the implementation itself deviated from original design. Evaluators will (a) compare strategic documents listed as follows with the original Task Order and (b) compare implementation practices with those earlier two elements (i.e. the Task Order and the additional strategic documents). 22 This methodology was approved by MSI on September 27, 2011. A few small modifications were made to this text since, for clarification purposes only. 77 | Page Strategic documents: - Workplan; - PSCRP overall conflict resolution strategy (May 2010), including MSI’s comments. Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? Evaluators will interview key project staff, LNGO partners and select community committees so that they describe the roles and responsibilities as they perceive them, and discuss the suitability of the same, and recommendations for improvements. Key staff to be interviewed: - CoP and DCoP; - MSI’s program delivery managers; - IA’s community development managers. Reference documents: - MSI-IA contract; - Project interactions document; - Organizational chart; - Salary scale and other key operational documents (or a discussion with IA’s head of administration and finance) Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? In particular, how effective was the project in responding to the needs of ex-combatants and other marginalized groups? Evaluators will produce a list of stakeholders indicated in the Task Order and use it as a systematic framework to break down questions and considerations during interviews. Marginalized groups is understood as ex-combatants, GBV survivors, pygmies and location￾specific ethnic minorities. Needs of marginalized groups is understood as their specific needs that are directly the consequence of their marginalized ‘category’. To the extent possible, evaluators will disaggregate projects’ benefits to those categories of population and their special needs. Evaluators will collect perceptions of key project staff as well as hold semi-directive group interviews with community and cross reference with community development plans and as available maps of communities. To what extent were the four intended results achieved, in comparison to project indicators? (detail per result) Evaluators will compare PMP targets with indicators’ achievements as of September 30, 2011. Additional indicators reported upon by the project but not included in the PMP will also be taken into account. Were any community interventions more successful than others? If so, what appears to have contributed to disparate levels of effectiveness? Were there unforeseen negative impact? 78 | Page Evaluators will determine a list of key community products (such as community plans, conflict mediations, etc.) and answer the question at two levels: 1. Are given products more important than others in delivering the projects’ results; 2. Are there variations in the performance of communities compared with each other in the delivery of those products, and if so can determinants be identified for those variations. While doing so, negative impacts will be identified as possible as well. List of community products: - Community development plans (initial and end-project); - The subprojects’ outputs; - The maintenance plans; - The community accountability meetings’ key decisions; - The dialogue meetings’ end reports; - The conflict mediation services; - The joint meetings between the 2 types of committees. Were changes in social dynamics or community cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? Evaluators will review the dialogue meetings’ end reports produced by project staff to determine a global picture of positive and negative social dynamics and community cohesion changes, including gender and marginalized groups considerations. Attribution will strictly speaking not be possible given the absence of a randomized selection of treatment and control communities, but evaluators will formulate a judgment on the perceived likely attributability of changes where possible. Sustainability will be considered through interviews with staff, partners and community members. Was the theory of change, as expressed in International Alert’s global conflict resolution strategy, adapted and relevant to the challenges of the security and policy environment? It is understood that the correct title of the document is ‘PSCRP overall conflict resolution strategy’. The evaluators will use the version that includes the comments of MSI on the draft produced by IA. The evaluators will determine what concrete interventions were undertaken within the project as a result of implementing the Theory of change. They will then identify whether those interventions faced any issues related to the security and policy environment. As an additional but essential dimension of this question, the evaluators will also analyze the intrinsic quality of the said Theory of change based on normative aspects using a basis the USAID guidelines/definition. 79 | Page Did the project contribute overall to reconciliation and conflict transformation in targeted communities? How effective were its strategies and achievements? What was the impact (positive or negative, short or long term) of the project’s various components on peace-building at community level? The evaluators understand that the first sub-question is focused at part of the purpose level of the project. The second sub-question is focused at the results’ level and has been addressed under question 4 above; it will thus not be discussed further here. Evaluators will review the relevant purpose indicators. Attribution will be estimated based on logic, given the absence of a proper control to be able to attribute change. Peace-building is defined by IA for the purpose of this question as ‘increased trust, improved understanding and the removal of violence from conflict resolution’. Given this definition, the evaluators consider that reconciliation together with conflict transformation equal peace-building and thus the last sub-question is not treated separately. Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? The evaluators will determine likely sustainability factors for each of the project’s outcomes, based on knowledge of other projects and interviews with staff and communities. They will then verify whether those factors are present. They will identify determinants for the presence of those factors. Contribution to the eastern DRC stabilization agenda Was the project coherent with the community-related components of I-SSSS and STAREC strategies? Evaluators understand this means the community recovery component of I-SSSS and STAREC. Evaluators will review the project’s strategy and implementation and compare with the description of the relevant components of the I-SSSS and STAREC strategies. This draft analysis will then be discussed with relevant senior staff within MONUSCO and the DRC provincial ministry of planning. Did it add value to the other components? The evaluators understand this refers to the other components of the I-SSSS and STAREC. The evaluators will review projects’ outputs and outcomes with those expected from the other components of I-SSSS and STAREC and ask relevant senior staff within MONUSCO and the DRC provincial ministry of planning for their opinion. The CoP of the project will also be asked to describe any additional possible contribution. Was the combination of reconciliation with recovery an effective strategy for stabilization (i) in targeted communities and (ii) beyond those communities? Could specific strengths and weaknesses be identified? The evaluators understand that the question focuses on whether combining the two components improves project effectiveness towards its stabilization purpose. The evaluators will review whether those activities, processes and outputs that are primarily related to a given component (reconciliation or recovery) have provided an ancillary additional positive contribution to the achievement of the other component that is relevant to stabilization. 80 | Page The evaluators will seek to identify spill-over effects relevant to stabilization through interviews with staff on non-targeted communities and will then determine whether any of those effects would likely not have taken place if the project had not combined a reconciliation and a recovery component. Identification will be done through staff and community interviews. Was the engagement with CLPCs in North Kivu effective, with regards to project objectives and to CLPCs’ own training needs? Evaluators will compare the role of CLPCs with the description of the expected results of the project to determine relevance. Other requirements of the Task Order will be considered. Given the extremely short time frame between the implementation of the given activities and the time of the evaluation, the evaluators estimate that they can judge relevance of those activities to the strategy of the project, but that effectiveness is not yet observable. Evaluators will compare PSCRP support to the CLPCs with their training needs as identified in the report of the June 2011 workshop on determining CLPC’s training needs. Lessons Learnt / Good Practices Are there areas where PSCRP can be deemed to have reached good practices, which could be turned into practical tools or guidelines for future projects? Are they specific to eastern Congo or could they be adapted to different contexts? SoW asked that evaluation looks in particular into: (i) approach to profiling and determining adequate community size; (ii) election / redynamization of community committees (development and peace); (iii) elaboration of development plans; (iv) resource kits on conflict management and resolution for peace committees; (v) inclusion and participation of disenfranchised groups; (vi) collaboration between community development and conflict resolution entities. The evaluators will review project’s processes through review of project documents and tools as well as interviews with staff as well as IA’s Eastern DRC program coordinator, and will identify possible good practices for each to the key steps and components of the project. Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming. Evaluators will interview project staff and communities and as possible other stakeholders (LNGOs, construction contractors, as well as IA CD etc.) to identify perceived contribution of specific project community processes to reconciliation and reintegration, as well as opportunities for improvements. Are there scalable elements in this project? Which ones? Based on experience of other at-scale projects using similar methodologies, the evaluators will offer an opinion on what components of PSCRP are scalable and why. 81 | Page Explanation of selection of communities to be visited23 Based on time available, logistics and security considerations it was established that the evaluators could allocate a total of 2 days to field visits, and that each of those 2 days could be spent in only 1 axis of the project. It was also established that no more than 2 communities could be visited during each day or in each axis. The SoW of the evaluation also required that communities be visited in both North and South Kivu. It was also decided to exclude from the universe the 30th community in Hombo where only part of the project process was implemented. Taking into account those constraints, the evaluators determined a set of 5 criteria to select communities, and worked with PSCRP MnE/knowledge management staff to mark each community based on those criteria (see table below). Somme communities are presented jointly to reflect on actual implementation. The marks were then presented in a graph form (see chart below) to identify commonalities and differences between communities. The evaluators then determined for each criteria how to best spread communities given the evaluation questions and the spread of communities within the given criteria. Based on those final section criteria, communities were picked for visits, based on those that together (a) matched the desired spread and (b) complied with logistics, security and SoW requirements. The resulting community selection is highlighted in green in the chart below. Territories Communities Variables Conflict transformat ion involveme nt of marginaliz ed Conflicts resulting from implementatio n of project Accountability meetings CLPC supported by the project FIZI Kalundja 2 1 N 2 FIZI Moma 2 1 N 2 FIZI Mushimbakye 2 1 O 2 MASISI Karuba 1 1 N 1 MASISI Bushuhe+Karuba 1 3 N 1 MASISI Rushoga 1 1 N 2 MASISI Kimoka 3 3 N 2 O MASISI Macha-Mubamb. 2 4 N 2 O MASISI Kingi 2 2 O 2 O NYIRAGONGO Kibumba I+II 2 2 O 2 RUTSHURU Mabungo I+II 3 3 O 2 RUTSHURU Bugara 2 3 N 3 RUTSHURU Ntamugenga I+II 1 2 O 3 RUTSHURU Bunyenzi 2 3 O 3 UVIRA Mutarule I+II 1 1 O 1 UVIRA Bwegera 2 1 O 2 UVIRA Sange I+II+III 3 3 N 3 WALUNGU Mulamba+Bulonge 3 3 N 2 WALUNGU Izege + Ciravanyi 2 2 O 2 WALUNGU Kaniola + Cagala 3 2 O 2 23 This section was not initially part of the methodology that was approved by MSI. It was developed later by the consultant. 82 | Page 0 2 4 6 8 10 12 14 16 18 conflict transformation involment of marginalized conflicts from project use acct meeting CLPC supported Postive Medium-P Medium-N Negative 83 | Page Interview guides CoP/DCoP (et autres staff senior) Did the strategies in the original design meet the requirements of the Task Order? Did they need revising during the course of PSCRP? If yes, why? Discuter avec le CoP et la IA’s HQ des changements identifiés par les consultants entre le Task Order et les autres documents stratégiques et les pratiques de mise en œuvre. (Chaque consultant pose la question par rapport aux outils stratégiques dont il a la charge et par rapport aux évolutions de pratiques dans la mise en œuvre qui l’intéressent) Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? Why was a partnership put in place – what were each other’s expectations when setting up the partnership – what was shared in terms of vision – what was each other’s scale of priorities – what experience of each in cooperation, what core competencies in each – what was shared in terms of values – Organizational chart (see also questions on overall project architecture) – what level of agreement was reached between both organizations when designing the project architecture. Describe the various roles and responsibilities in project management and how it evolved in the course of the project and why, the ones which were shared and the ones that were exclusive – how did it play a role in the coherence and the harmonization of activities. The overall project architecture: the perception of the project architecture – from MSI and then from IA perspective, what was the strategic orientation of the project – how did the project architecture serve it – where there demands to review it and was it done – perception of key staff (management and facilitators) on whether it supported PSCRP’s objectives. Chaque consultant pose les questions qui lui paraissent importantes dans celles ci-dessus. Was the theory of change, as expressed in International Alert’s global conflict resolution strategy, adapted and relevant to the challenges of the security and policy environment? Identifier les éléments clés du contexte sécuritaire et politiques publiques au travers d’un interview du CoP (relancer sur le minier, l’agriculture, le foncier, les relations régionales) et les conséquences sur la mise en œuvre et le degré d’atteinte des résultats. Was the project coherent with the community-related components of I-SSSS and STAREC strategies? Demander l’avis et relancer sur la base d’une analyse préalable. Did it add value to the other components (or ISSSS et STAREC? Demander l’avis et relancer sur la base d’une analyse préalable. IA/MSI Managers Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? Why was a partnership put in place – what were each other’s expectations when setting up the partnership – what was shared in terms of vision – what was each other’s scale of priorities – what experience of each in cooperation, what core competencies in each – what was shared in terms of values – Organizational 84 | Page chart (see also questions on overall project architecture) – what level of agreement was reached between both organizations when designing the project architecture. Describe the various roles and responsibilities in project management and how it evolved in the course of the project and why, the ones which were shared and the ones that were exclusive – how did it play a role in the coherence and the harmonization of activities. The overall project architecture: the perception of the project architecture – from MSI and then from IA perspective, what was the strategic orientation of the project – how did the project architecture serve it – where there demands to review it and was it done – perception of key staff (management and facilitators) on whether it supported PSCRP’s objectives. Chaque consultant pose les questions qui lui paraissent importantes dans celles ci-dessus. Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? In particular, how effective was the project in responding to the needs of ex-combatants and other marginalized groups? Lister les stakeholders / marginalized (on complète après coup si besoin) ; questionner si négligés ou non (implication dans les décisions communautaires, workforce, bénéfice retiré), pourquoi, y-a-t-il eu des groupes marginalisés dont les besoins n’ont pas été adressés ? Was the engagement with CLPCs in North Kivu effective, with regards to project objectives and to CLPCs’ own training needs? Demander au staff du projet ayant suivi les communautés se trouvant dans le groupement du CLPC en question de définir si jusqu’à présent les communautés ont tiré un bénéfice de l’appui du projet au CLPC. STAREC et ISSSS staff Was the project coherent with the community-related components of I-SSSS and STAREC strategies? Demander l’avis et relancer sur la base d’une analyse préalable. Did it add value to the other components (or ISSSS et STAREC? Demander l’avis et relancer sur la base d’une analyse préalable. SSI Comités de Paix / Comités de développement Communautaire Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming. Les déterminants de la cohésion sociale au sein des communautés (ex : le dialogue, les échanges économiques, les mariages interethniques, les tontines). Si vous allez visiter un autre village à quoi jugez vous si les gens de ce village se font confiance les uns les autres et s’entraident ? Racontez des histoires vécues de réconciliation Racontez des histoires de réintégration 85 | Page Quels sont ceux qui ont été favorisés par le projet ? En quoi le projet contribue-t-il à répondre aux attentes des personnes prises dans des conflits ? En quoi le projet contribue-t-il à limiter/résoudre les injustices ? Dans votre village, comment la confiance et l’entraide entre les gens ont changé ces derniers deux ans ? Si vous repensez aux différentes choses que MSI, International Alert, LNGO partners ont fait avec vous au cours du projet, quels sont d’après vous celles qui étaient les plus utiles pour aider à améliorer la confiance et l’entraide - Les élections des membres du CLD et noyau - La sélection des travailleurs pour les projets - Le noyau de paix/le comité de paix / les services de médiation fournis - Le CLD / le petit projet, le grand projet - Les réunions de coordination entre les deux comités - Les forums communautaires - Les activités de sensibilisation/d’information - Autres - (liste à décliner pour chaque communauté sur la base de ce qui a été réalisé) Qui en a bénéficié en particulier ? (pour mémoire / implication dans les décisions communautaires, workforce) Des exemples où les populations ont résisté à la violence et aux provocations ? En quoi le projet a facilité cette résistance L’insécurité a-t-elle augmenté/diminué ces deux dernières années ? Comment le projet favorise-t-il le renforcement du sentiment de sécurité ? Were any important beneficiaries or stakeholders identified in the Task Order neglected or less impacted by the projects than others? In particular, how effective was the project in responding to the needs of ex-combatants and other marginalized groups? Lister les stakeholders / marginalized (on complète après si besoin) Questionner si négligés ou non, pourquoi, Y a-t-il eu des groupes marginalisés dont les besoins n’ont pas été adressés ? Were changes in social dynamics or community cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? Citer des transformations observées dans les attitudes, comportements et interactions, notamment / marginalisés ? Qu’est-ce-qui contribuera à leur durabilité ? Quels sont les facteurs qui pourraient limiter le développement de ces transformations ? Voire les annihiler ? Partenaires locaux Quels objectifs vous ont été fixés dans le cadre de ce projet ? Par qui ? 86 | Page Were the sharing of roles & responsibilities between MSI, IA and local partners, and the overall project architecture suited to PSCRP’s objectives? Comment percevez-vous l’organisation des intervenants dans le projet ? Quels ont été vos forces dans la mise en œuvre du projet / construction de la paix + / suivi des activités des comités ? Vos faiblesses et comment avez-vous pu les contourner? Were changes in social dynamics or community cohesion observed (positive or negative)? How were gender relations or relations with marginalized groups affected? Can observed changes be attributed to the project? Can they be expected to be sustainable? Quels sont les déterminants de la cohésion sociale au sein des communautés sur lesquels le projet a eu un effet ? En quoi le projet contribue-t-il à répondre aux attentes en lien avec le conflit + à limiter/résoudre les injustices ? Quelles sont les initiatives que vous avez observées au sein des communautés et lancées indépendamment du projet ? Des exemples où les populations ont résisté à la violence et aux provocations ? En quoi le projet a facilité cette résistance ? Parmi les communautés cibles, lesquels ont vu l’insécurité augmenter ? Comment le projet favorise-t-il le renforcement du sentiment de sécurité ? Citer des transformations observées dans les attitudes, comportements et interactions, notamment / marginalisés ? Qu’est-ce-qui contribuera à leur durabilité ? Quels sont les facteurs qui pourraient limiter le développement de ces transformations ? Voire les annihiler ? Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? Comptez-vous prolonger ce type d’interventions dans les communautés ou dans d’autres communautés ? Comment ? Are there any outcomes that appear to have a good chance at sustainability? How was this achieved? Parmi la liste (ci-dessous), quels sont les + durables ? les – durables ? pourquoi, quels risques ? Y a-t-il des différences d’une communauté à l’autre ? Infrastructures et équipement, gestion des stocks de fournitures et de matériel, réunions pour définir les plans de développement communautaires, médiation entre parties en conflits Prévoyez-vous de prolonger l’appui aux communautés ? Comment ? Specifically, identify practices, drawn from the project’s experience of community processes, which contribute to reconciliation and reintegration, and make recommendations for future programming. Liste des processus : élections, dialogue, définition des plans de développement, gestion des relations avec les fournisseurs de services ? Dans des processus, quels sont ceux qui ont contribué à la réconciliation ? à la (ré)intégration ? 87 | Page Agenda of the workshop on project staff’s contribution to the evaluation Atelier du mardi 4 octobre 2011 8h00 Installation des participants 8h15 Présentation des objectifs et du processus de la consultance, du rôle de l’atelier et de son déroulement, et questions 8h30 La contribution des différents processus communautaires à l’atteinte des différents résultats 9h20 Les changements dans les dynamiques sociales et la cohésion au sein des communautés 10h20 Pause café 10h30 La durabilité des résultats atteints 12h00 Pause déjeuner 13h00 La pertinence de l’approche définie par IA par rapport au contexte 14h05 Les bonnes pratiques et les leçons apprises 15h30 pause café 15h40 Contribution des bonnes pratiques à la réconciliation et à la réintégration 16h20 Clôture 16h30 Fin de l’atelier