P E R F O R M A N C E E V A L U A T I O N USAID/Dominican Republic Community Justice Houses Project May 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Hugo J. Pirela, Ph.D., M.A. & Emmanuel Silvestre, Ph.D., for USAID Dominican Republic under Contract No. AID-517-O-16-00001. i ACCRONYMS AMCR ALTERNATIVE METHODS FOR CONFLICT RESOLUTION CHJ COMMUNITY HOUSES OF JUSTICE CSO CIVIL SOCIETY ORGANIZATION DR DOMINICAN REPUBLIC LOGFRAME LOGICAL FRAMEWORK NGO NON GOVERNMENT ORGANIZATION NSAJ NATIONAL SYSTEM OF ACCESS TO JUSTICE (now: of Alternative Resolution of Conflicts) OECD ORGANIZATION FOR THE ECONOMIC COOPERATION AND DEVELOPMENT SPSS STATISTICAL PACKAGE FOR SOCIAL SCIENCES S.O.W. SCOPE OF WORK USAID UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT ii TABLE OF CONTENTS PAGE EXECUTIVE SUMMARY a I. Introduction ……………………………………………………………………………………………………………………………………… A. Evaluation purpose & questions ……………………………………………………………………………………………. B. Project background ………………………………………………………………………………………………………………. C. Methods & limitations …………………………………………………………………………………………………………. II. Findings: Project achievements and observable transformations ……………………………………………………. A. Development results: a quantitative perspective ……………………………………………………………………….. 1. Project outputs ……………………………………………………………………………………………………………………. a. Product 1: Program of Access to justice and citizen safety coordinated and concerted … i. Houses & agreements ……………………………………………………………………………………………… ii. Organizational involvement …………………………………………………………………………………….. b. Product 2: Alternate methods of conflict resolution and access to justice implemented. i. Protection of Human Rights …………………………………………………………………………………….. ii. Aiding vulnerable women in the prevention of violence ………………………………………….. iii. Aiding vulnerable youth in the prevention of delinquency ………………………………………. iv. Training judicial system officers and community leaders …………………………………………. v. Training citizens ………………………………………………………………………………………………………. vi. Legal orientation to residents ………………………………………………………………………………….. c. Product 3: NSAJ & municipality established and draft law submitted …………………………… i. Drafting of the NSAJ Law (currently: Alternative Resolution of Conflict Law) ……………. ii. Developing a consensus around the draft law …………………………………………………………. iii. Incorporating the CHJs program into the National Budget ………………………………………. 2. Project outcomes ………………………………………………………………………………………………………………… a. Observable expected effects ………………………………………………………………………………………… i. Project´s response effect …………………………………………………………………………………………. ii. Project´s judicial decongestion effect ………………………………………………………………………. iii. Project´s sustainability effect …………………………………………………………………………………… iv. Project´s resolution effect ……………………………………………………………………………………….. b. Other project results ……………………………………………………………………………………………………. i. User satisfaction ……………………………………………………………………………………………………… ii. Perception of justice ………………………………………………………………………………………………… iii. Process recognition …………………………………………………………………………………………………. 1 1 2 3 3 3 3 3 3 4 4 5 5 6 7 8 9 9 9 10 10 11 11 11 12 14 15 16 16 19 20 III. Analysis & conclusions …………………………………………………………………………………………………………………… A. On the project development results …………………………………………………………………………………………. 1. Project performance in delivering expect outputs ……………………………………………………………. a. Image and credibility of the AMCR approach …………………………………………………………….. b. Activation of a pent-up demand for restorative justice ………………………………………………. 2. Project influence on the target communities concerning access to justice ……………………….. a. Response to demand for restorative justice ……………………………………………………………….. b. Contribution to the general administration of justice …………………………………………………. c. Contribution to conflicts resolution habits ………………………………………………………………….. d. Induction of gray markets for legal practices around restorative justice …………………….. e. Stimulation of changes in social attributes of beneficiaries ………………………………………… f. Stimulation of changes in the countenance of law enforcement ………………………………… 3. Project influence on the target communities in other related aspects ……………………………… a. User satisfaction ………………………………………………………………………………………………………. b. Perception of justice ………………………………………………………………………………………………… c. Process recognition …………………………………………………………………………………………………. B. On the Project progress toward sustainability ………………………………………………………………………….. 21 22 22 22 23 23 23 24 25 25 25 26 26 26 26 27 27 iii C. On how USAID/DR can build on the results of Project’s lessons ………………………………………………… 1. Role of the social penetration strategy ……………………………………………………………………………… 2. Communities best endowed for success …………………………………………………………………………… 3. Determinants of long term self-sustainability ……………………………………………………………………. 4. Role of a Law formally establishing a Restorative Justice System …………………………………. 5. Ways to strengthen the arguments for the CHJs ……………………………………………………………. 6. Risks of over-regulation and politicization …………………………………………………………………… 7. Role of discovery in house prosecution services ………………………………………………………………. 8. Role of human capital formation and extra intelligence about population needs …………….. 9. Requirements for alliances with the private sector ………………………………………………………….. 10. Pros and cons of lawyers’ underground economy around the CHJs …………………………………. 11. Role of pro bono legal aid for vulnerable plaintiffs …………………………………………………………… IV. Recommendations ………………………………………………………………………………………………………………………… 28 28 28 29 29 29 30 30 31 31 31 31 32 ANNEX I: Methodological Approach & Research Instruments ANNEX II: Field Survey Sample Validation ANNEX III: Qualitative outlook on the CHJ Project. A case study ANNEX IV: Research Methodology and Instruments (on a physical CD) ANNEX V: Field Statistics (on physical CD) a EXECUTIVE SUMMARY This document presents what USAID defines as a performance evaluation of the Project: “Sustainability and Expansion of Community Houses of Justice” (CHJ), commissioned by the D.R. mission to assess: (i) how it has influenced the targeted communities concerning access to justice, respect for human rights and citizen security; including how effective has been the violence unit operating at the CHJ Cienfuegos in Santiago and if it is worth to open such unit in the remaining Houses; (ii) what were the development results obtained by the project, especially in terms of access to justice by vulnerable population in the D.R.; (iii) the extent to which the results obtained can be sustainable in the long run; and what factors will contribute to the future sustainability of the Houses; and (iv) whether and how the USAID/DR mission can build on the results of the project, based on lessons learned for future design, implementation and sustainability. Answers to research questions (i) and (ii) are based on a quantitative, statistical analysis of project data, both previously existing and newly generated through field surveys. Statistical methods and instruments used in the research are detailed in Annex I -including a discussion of research limitations that had to be overcome- and findings are discussed in Chapter II. All four questions were also qualitatively researched using non statistical tools, such as semi-structured interviews with stakeholders, including project partners, executors and authorities, as well as discussions in focus groups of beneficiaries. Results of this qualitative analysis are presented in a separate annex as a Case Study. Detailed conclusions and recommendations are presented in Chapter III and IV. Conclusions are organized around the research questions and focus on whether the Project´s own goals were achieved. Unintended results and risks are also discussed and specifically flagged. WHAT WERE THE PROJECT´S DEVELOPMENT RESULTS. At the project´s inception, appropriate metrics were identified for measuring outputs; but not quite for measuring outcomes. For these, the original design defined indicators lacking in quantity and quality. Inadequacies in data-gathering during execution additionally hampered measurements of outcomes. Measures taken by researchers to deal with these limitations are discussed in Annexes I and II. In this context, the evidence shows that the project attained or exceeded most of its targets on deliverable products. This operational performance was not only high but also accelerated by the project midpoint; a fact that had positive intended consequences down the road, but also some not necessarily intended. The high performance -chiefly attributable to the implementing partner´s pro-activity in raising awareness of the underlying issues- provided a strong footing for the CHJs´ good image and the credibility the restorative justice methods. On the other hand, it is difficult to explain the success of the training and promotion effort without recognizing the role of people´s needs for access to justice being previously not covered and suddenly unleashed as actual demand; a fact that, in turn, may soon overwhelm the CHJs´ installed capacity. The exception in the delivery of outputs occurred in the promotion of a Law sanctioning the restorative justice system, which did not advance as expected. This shortfall, though quantitatively minor, was qualitative crucial for it implies that the most important long run sustainability determinant of the project results was not secured. The Project outcomes are summarized in the next section. HOW THE PROJECT HAS INFLUENCED THE TARGET COMMUNITIES CONCERNING ITS ORIGINAL GOALS. Direct evidence exists of the Project´s effects on communities (development outcomes) only concerning access to justice. Evidence of influences relating to human rights and citizens security is much weaker, mainly because no benchmark indicators were defined ex-ante to assess such outcomes. On protecting women rights, the violence unit set up in 2007 at the Cienfuegos CHJ complemented for a while Santiago´s Family & Gender Violence, but lacked the resources and mandate needed to properly protect victims, and was closed by authorities in 2015. (See a discussion of this experience in the Case Study of Annex III). Now, on access to justice there is clear evidence of a project response effect; i.e. the CHJs´ efficacy to answer the demand for restorative justice, as measured by the number of conflict cases processed. The latter looks just like a project output, but the fact that cases are brought to the CHJs in such numbers is, arguably, at least a precursor measure of changes in people´s attitudes towards conflict resolution and the protection of rights through restorative justice means: a transformation the Project sought. The evidence also suggests some project contribution to the general justice administration. The data born out, albeit indirectly, an incipient judicial decongestion effect of the Project; measured by the number of persons or cases derived to the b CHJs from other public agencies or ordinary justice venues. The CHJs are increasingly seen by judges and prosecutors as legitimate venues to hear cases less suited for the ordinary justice processes and, therefore, as allies for alleviating their workload. The Project appears to have also contributed to changes in people´s conflict management habits. The derivation rate diminished from a high rate at the start; but not because the number of derivations (numerator) declined, but because the total number of persons/cases showing up in the CHJs (denominator) increased faster: a likely result of the outreach pro-activity of the Project plus an increasing need or pent-up demand for the houses’ services. Perhaps the outcome best supported by the evidence is the resolution effect the Project had in the communities. The fact has been established with high statistical significance that the CHJs made possible to achieve a settlement in at least half of the cases treated. This show-cased the reality of changing habits in dealing with confrontations and has contributed to the credibility the CHJs so far. The data also suggests unintended effects, specifically on lawyers´ demeanor and the profile of users. There is circumstantial evidence that the Project opened opportunities for “under-the-radar” legal practices around the CHJs functions. This involves no inappropriate actions by the CHJs themselves, and consists of privately hired lawyers inducing plaintiffs to use houses´ mediation for quick resolutions. The fees charged by this low-cost “grey market” practice can actually represent savings, as well as faster and more satisfying results, for poor people who might view such charges as acceptable opportunity costs, compared to those of the ordinary justice. Also, and however focused the Project has been on vulnerable areas, its renown has enticed the inflow of cases from outside the CHJ´s “territories”, and of clients such as real estate brokers, business people etc. not belonging to the most vulnerable groups. Being residents of poor areas, these users are small business owners or marginal operators who justly fall within the Project´s target population, whose participation does not entail a departure from its philosophy. Yet their increasing resort to AMCR does diversify the market for the CHJs services and pressure their installed capacity. On the issue of citizens security, and though the evidence is feeble, it appears that the Project has had some influence on local police commanders who, being aware that residents´ complaints about security can be heard, have promoted a better focus of police services on protecting people´s safety and rights. HOW THE PROJECT HAS INFLUENCED THE TARGET COMMUNITIES CONCERNING OTHER RELATED ASPECTS. Recognizing the importance -beyond the resolution effect- of CHJs users´ satisfaction, perception of justice, and correct association of results with the nature of the AMCR–based processes, the study researched the Project effectiveness in these respects. Statistical results suggest that beneficiaries generally tend to be satisfied with the services received, regardless of end results. This conclusion was further validated when users were probed on their perception of justice, and the basis for that sentiment. The average response was above 4 in a perception scale from 1 to 5 in all groups surveyed; with no statistically significant differences in the responses due to the independent variables used in the study. The data showed also that users were not only consistent in their perceived process-results connection, but that they also acquired a deeper grasp of the restorative justice concepts, nature and limitations. TO WHAT EXTENT THE PROJECT HAS PROGRESSED TOWARD SUSTAINABILITY. That sufficient funds were raised domestically to allow CHJs to operate without interruption in the period evaluated is evidence of the Project immediate sustainability. The study stresses the importance of not confusing the effort to sustain the Project with the effort to achieve self-sustainability for its results, which is the strategic goal here. The fact that the relevant available data did not come from an accounting audit of the CHJs funds flows and financial results statements precludes definitive conclusions on their self-sustainability in long run when the Project or its executing partners will not be around. The study also stresses the need to recognize that the key condition for the CHJs´ long term financial sustainability resides on its institutional sustainability; i.e. on their evolution into a fully sanctioned branch of the judicial establishment, permanently required to function within the official administration of justice. while social ownership of, and private empathy toward, the CHJs are important ingredients for their maintenance, their long term financial viability is a function not of fetching voluntary funds and pro bono contributions, but ultimately depends on a State decision to make the CHJs part and parcel of its official structure and routine operation. Given this premises the Project sustenance was secured, but the key condition for long term sustainability was not. c HOW USAID/DR CAN BUILD ON THE RESULTS OF PROJECT’S LESSONS. The following are the Project lessons from which the study has derived specific recommendations: (i) Role of the social penetration strategy: The strategy of “leading with community mobilization”, i.e. recruiting the grassroots organizations and natural leaders for the Project early in its implementation seems to have paid off in high performance as well as quality of results; (ii) Communities best endowed for success: certain qualitative features of communities, not necessarily their size or location, might make a restorative justice project more likely to succeed in them. Specifically, provided a successful lead-off social penetration strategy is applied, and a good project performance achieved, communities with high strife levels may stand a better chance to reap deeper rooted success and earlier positive demonstration effect and appreciation by residents; (iii) Determinants of long term self-sustainability: If eliciting social and private grassroots mobilization in favor of the CHJs is a proven strategy for their success, and a requisite for their adequate operation, the long term vocation of any restorative justice system should be no other than to become a formal, permanent component of the official Judicial System of a country, whose sustenance is not a matter of voluntary commitment but the result of annual exercises of budget allocation to a state branch in charge of executing normal public functions and policies, as any other judicial venue; (iv) Role of a Law formally establishing a Restorative Justice System: While it is necessary to accumulate lessons for operational effectiveness, and generate enough positive demonstration effect about the CHJs, to build a convincing case capable of overriding attitudes that might be opposing them, in a parallel manner or once this initial thrust has been given to the process, a diligent effort is necessary to achieve a.s.a.p. the above mentioned formal sanctioning of the system as part of the regular Judiciary; (v) Ways to strengthen the arguments for the CHJs: A full accounting audit appears to be of the essence to provide evidence that: a) funding of the CHJs system can be permanently assured as part of the annual public budget allocation exercise; b) funding of the system will not necessarily require additional request for resources every year, but mostly a resource allocation exercise; and c) such resource redistribution will prove to be cost-effective; (vi) Risks of over-regulation and politicization: The Law necessary to sanction the CHJs system must avoid: a) an excessive regulation that would make it to resemble so much the ordinary justice system that people might not appreciate its relative advantages; and b) dependence of the restorative justice system on funding decisions made by the executive branch of government; (vii) Innovative use of authority in CHJ services: Experience shows that, because of existing nuances in the social perception of authority, summoning acts tend to be more effective in getting people to appear before the houses if the subpoena is served by a house prosecutor, even if its purpose is only to start a mediation process -which is not strictly improper. Other proper forms of using this “grey area” of cultural perception of authority to promote restorative justice methods are the house prosecutors´ “descents” to the theater of conflicts -provided intimidation is avoided- as part of their responsibility to establish or corroborate facts in the field on a case by case basis; (viii) Role of human capital formation and extra intelligence about population needs: The observed rapid unleashing of a pent-up demand for restorative justice services might in the near future tax the CHJs´ capacity the hardest in the area of human capital available with the adequate professional competencies to properly attend and process cases. Emphasizing the training efforts, as well as researching potential demands from people within and without the targeted areas seems advisable; (ix) Risks implicit in alliances with the private sector: There appear to be a number of ways in which private sector support to alternative justice methods and institutions can avoid or significantly mitigate the associated risk of conflict of interest. For instance: if the support comes from business organizations rather than from particular businesses; or if said support indirectly benefits the effort but does not relate directly to sustaining any particular function, process or case; (x) Pros and cons of lawyers’ underground economy around the CHJs: Experience appears to show that, as long as proper measures are in place to impede the contagion of the CHJs system from the workings of the lawyers´ underground economy around it, private arrangements between lawyers and poor plaintiffs can be viewed as win-win solutions, that both benefit alternative justice methods and generate Pareto-optimal satisfaction in all parties involved; (xi) Role of pro bono legal aid for vulnerable plaintiffs: The help that term law students or interns can provide to plaintiffs in poor communities may have an additional positive effect in helping mitigate the need for lawyers’ informal practice around the CHJs. d Recommendations. Based on the lessons above the following are recommendations for the design and execution of other like initiatives or for future stages of the present project. 1. Lead the implementation of CHJ projects in new locations with an early social penetration strategy specifically designed to (i) enlist the allegiance to the project of natural leaders and grassroots organizations in the targeted communities, as well as (ii) to co-opt de support of like-minded CSOs and NGOs engaged in like-minded community and social development issues and efforts. 2. In choosing areas for new CHJ projects in the future, give priority to communities with high conflict levels, for maximum possible demonstration effect over the short and medium term. 3. In parallel to, or shortly after, an initial phase of experiment to accumulate valuable lessons for effectiveness in the field, and to build up a convincing case for a restorative justice system, give priority to the formal sanctioning of such system through the requisite law and regulations. To the extent possible, avoid negative demonstration effects during the experimental phase due to lack of adequate endowment of resources and mandate, that might diminish the initiative´s credibility as in the failed experience with the Cienfuegos CHJ´s violence unit. 4. In parallel to an initial phase of experimentation to build up a convincing case for the social desirability of a restorative justice system, or shortly thereafter, make a quantitative study on the feasibility of funding the system based on a cost-effective redirecting of budget funds traditionally allocated to ordinary justice venues. 5. In developing a draft Law and its By-laws to sanction a restorative justice system avoid: (i) excessive regulations that might make its procedures resemble too much those of the ordinary justice system, and (ii) any possible dependence of the system on funding decisions made by the executive branch of government 6. Whenever possible, provide the CHJs projects with resources to finance vehicles for the exclusively official use of house prosecutors to discharge their discovery function through fact-finding visits to the theater of conflicts, done with no show of force or intimidation of the individuals involved 7. Give priority to the formation of human capital with the adequate professional competencies required to properly attend and process the demand for alternate justice services, and expand the knowledge of populations beyond the project target communities that might get enticed to demand those services in the future. 8. Support alliances with, and co-opt the support of, the private sector for promoting a restorative justice systems, provided that the support comes from business organizations rather than from particular businesses; and safeguards are in place to avoid conflict of interest, such as that said support indirectly benefits the project but does sustaining any particular CHJ hearing process 9. Make sure that proper measures are in place to impede the contagion of the CHJs system by the workings of any underground economy that might crop up in lawyer’s private practices around the operation of CHJs, including the addition of legal orientation capabilities, through pro bono professional work or voluntary help from term law students or interns. 1 I. Introduction 1.1 The present study was commissioned by the USAID mission in the D.R. to assess the performance of the project: “Sustainability and Expansion of Community Justice Houses” (CHJ Project, for short) an initiative funded by the Agency in the Dominican Republic to facilitate access to justice, promote respect for human rights and enhance citizen security in communities with vulnerable population. A. Evaluation purpose & questions 1.2 The study aims at gaging the Project´s performance and analyzing the sustainability of its results, at the level of outputs and outcomes. This, in response to the request in the study´s S.O.W. for a “Performance Evaluation” which, in the Agency´s Policy parlance, focuses on measuring results as “the direct and near-term consequences of project activities”. 1 The evaluation also purports to identify lessons for future design and implementation practices; and provide feed-back to stakeholders on the Project’s operational and strategic guidelines. Among other issues, the evaluation focuses on answering, to the extent possible, the following questions: (i) How the Sustainability and Expansion of Community Justice Houses Project has influenced the targeted communities in what concerns access to justice, respect for human rights and citizen security; and, specifically, how effective has been the violence unit operating at the Casa Cienfuegos in Santiago and if it is worth to open such unit in the remaining Houses; (ii) What were the overall and specific development results obtained by the project, especially in terms of access to justice by vulnerable population in the Dominican Republic; (iii) The extent to which the results obtained are, or can be, sustainable in the long run; and, specifically, what factors will contribute to the future sustainability of the Houses; and (iv) Whether and how the USAID/DR mission can build on the results of the project, based on lessons learned for future design, implementation and sustainability. 1.3 To comply with the development effectiveness emphasis that these questions require of the research, and in line with the approach stipulated in the technical proposal, the consultants have tried first to assess whether the intervention achieved or not its own stipulated objectives (expected results). Other unintended or indirect project consequences, as well as the role of any external factor at play in shaping the actual results, have also been taken into consideration, but only second to judging effectiveness against the project´s original intent. 1.4 Researchers have also placed a premium on trying to infer attribution; that is: argue, to the extent possible, that the observed results, intended or unintended, direct or indirect, are imputable to, or at least time-associated with, the project; or, else, may be due to some other external factors, including those that might entail risks of ineffectiveness or distortion of results. Notice that, although the present research´s S.O.W. did not require the application of statistically appropriate counterfactual control tests -which is the gold standard for full inference of attribution- just arguing for a simple time-association between observed results and the project, as well as the possible interplay of risk factors, requires clearly identifying the intervention´s expected results chain -a.k.a. development hypothesis or project´s “theory of change”- as in the generalized example depicted in figure 1.1 of next page, which is based on consensus definitions established by the OECD. 1 See www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf, p.2. Notice that the “direct and near term consequences of project activities” are the conceptual equivalents to project outputs and outcomes in the Results Chain expected from the Project, based on the international consensus definitions of the OECD´s Glossary of Key Terms in Evaluation and Results Based Management, which the Consultants have adopted for the present study. 2 1.5 Hence the researchers have first endeavored to clarify the intervention´s expected Result Chain, as expressed in the project documentation, by identifying: (i) which specific deliverables to be produced by the Project activities were supposed to induce what specific institutional transformations; and (ii) which behavioral changes those transformations in turn were supposed to contribute to. (See right-hand side of Figure 1.1.) In doing so, the researchers made precisions to Project´s Result Framework which originally presented noticeable divergences from internationally accepted standard concepts. These divergences, as discussed bellow in the section Methods and limitations, included definitional inaccuracies such as presenting outputs (for instance: programs coordinated with local authorities: a project deliverable) and outcomes (for instance: communities know their rights and demand respect: a change in knowledge/attitudes) at the same level in the Results Chain; thus confusing both concepts; or mixing up the narrative of results with the narrative of their indicators; or including metrics that do not quite conform to the requisites of “benchmark indicators” as defined in the LogFrame standard practice. 1.6 Concerning the feedback, the study has been framed so as to provide specific practical and strategic orientations to USAID/DR, its partners and government institutions for justice and public prosecution in the D.R., as well as private entities and other stakeholders which might be interested in continuing the effort to help vulnerable population on the relevant issues of intervention. This, in line with the Paris Declaration on Aid Effectiveness – OECD principles of ownership, alignment, harmonization, managing for results and mutual accountability. B. Project background 1.7 The project evaluated herein represents the last phase in an effort started by USAID in 2006 to support the Community Houses of Justice (CHJ) in the D.R., aimed at facilitating the access to justice of vulnerable people in the country. This long term program has been implemented through a sub-grant under a contract with local partner Participación Ciudadana, a local NGO, during the period 2006-2007 and 2008-2012 (DPK-DAI); and a Cooperative Agreement with USAID for the period 2012-2015, which is the phase studied herein. 1.8 Based on a model implemented in Colombia, the Community Houses of Justice emerged in June 2006 as part of the executing partner Participacion Ciudadana’s Justice Program. The initiative represents a specific exercise in restorative justice: the approach to solving cases of wrongdoing that focuses on the needs of the victims and the offenders, as well as those of the community. Because in this approach it is the voluntarily engaged citizens the one party that must take up the majority of the responsibility in healing the pains caused by crime -and law professionals may have only a secondary roles in facilitating solutions- the restorative justice process contrasts with more punitive approaches to justice where the main aim is to punish the offender, or satisfy more abstract legal principles. For these reasons, the Community Houses of Justice project was deemed by USAID an appropriate means to provide valid alternate access to justice to vulnerable populations living in conflictive communities and, consequently, also to assist the Dominican Justice System in reducing case backlog and in enhancing the economy of procedure at tribunals and prosecutors offices. 1.9 Interest in this initiative is particularly relevant in a country immersed for some twenty years now in a judiciary modernization process which is still largely unfinished and appears to have stalled, or even back-pedaled lately, causing a resurgence of law infringement, corruption and impunity, which contributes to the worsening of violence, judicial uncertainty and citizens insecurity as reported by Amnesty International and other international watchdog agencies. Under these conditions, citizens at large and especially vulnerable populations such as Figure 1.1: Attribution & External Factors in the Results Chain 3 women and the young in poor, high-strife, communities find their legal rights increasingly unprotected and their access to justice hampered. So, the alternate methods for conflict resolution that the restorative justice approach affords appear the more suitable within this social context. As a consequence of the support by the USAID/DR to the present initiative over the years, there are currently eight Community Houses of Justice created in the Dominican Republic; namely, in: Santo Domingo (1); La Vega (1); Santiago (2); San Francisco de Macorís (1); Mao (2) [Valverde and Esperanza]; and La Ciénaga (1). The present study assess the performance of this initiative during the period 2012-2015 in the CHJs that were then operating; i.e., the ones in Santiago, Santo Domingo and La Vega. C. Methods & limitations 1.10 The findings and analysis in this report result from the application of mixed research tools, including the statistical examination of data on project results, both already available and newly produced by the study, through field surveys. The quantitative analysis aims to answer the two research questions that relate to the Project development outputs and outcomes. All methods and instruments for the statistical research are extensively discussed in Annex I: Methodological approach and research instruments and its appendices, including the details of important research limitations that had to be overcome by the researchers. In turn, the qualitative analysis presented addresses all four questions, including the ones on sustainability and lessons, and is based on the use of non statistical tools, more applicable to the research modality of case studies; such as interviews with key stakeholders -including project partners, executors and authorities- as well as discussions in focus groups with beneficiaries of CHJs (See Annex III: Qualitative outlook on the CHJ Project. A case study). II. Findings: Project achievements and observable transformations. A. Development results 2.1 The findings discussed below mainly focus on the Project achieving its own targets of outputs and outcomes, as identified in the retrofitted expected Results Chain of figure A, Annex I, and measured by the specified benchmark indicators. Findings on outputs are based on data provided by the project coordination verified, when appropriate, with the data from the CHJs files. Findings on outcomes, unless indicated, are based on data generated by the researches through their own field studies, using surveys of users, focus groups and interviews with key stakeholders. 1. Project deliverable outputs a. Product 1: Program of Access to justice and citizen safety coordinated and concerted 2.2 Concerning the coordination and launching of a program to promote the access to justice by vulnerable population in the DR, the Project delivered effectively on the quantitative products it set out to achieve. Of particular relevance here were outputs instrumentally crucial to sustain the operation in the future; especially: the number of CHJs officially registered and functioning in communities with vulnerable population and the number of institutional agreements signed and working in support of the Project. i. Houses & agreements 2.3 Although there was really no target set out for the number of CHJs to be created, during the period being presently evaluated 5 new houses were installed and the Project more than doubled the number of agreements it set out to sign with institutions of the public sector, as well as with CSOs, NGOs and organizations of the private sector to support the effort and help make it sustainable in the long run. Twelve agreements were signed for the support of the Project with institutions of such importance as the General Prosecutor Office, the Ministry of 4 Culture, the City Council of the National District and distinguished educational institutions and foundations (See figure 2.1).2 ii. Organizational involvement 2.4 Also the Project exceeded its operational targets by a factor greater than 3 in what concerned the number of organizations it set out to co-opt for the promotion of the initiative. Given the community-oriented nature of the Project and its social approach, a significant majority of these allied organizations -almost 50% of them- are indeed grassroots neighborhood boards and NGOs active in issues of social and community development. But the Project also managed to elicit the support of numerous other organizations, such as women and youth groups, business associations, educational institutions, churches etc., as illustrated in figure 2.2. All these organizations played an important role in sustaining the CHJs in the targeted communities; not only operationally through work-based, logistical and in-kind contributions to the houses´ activities, but also through functional collaboration and complementation of the CHJs services, in each organizations´ respective area of expertise or interest. 2.5 The role played by this organizational and outreach effort on the Project´s actual performance during the period June 2012 - July 2015 will be discussed in the present report; but suffice to say now that there is enough field evidence showing an involvement both visibly pro-active and sustained through time by these organizations co-opted through promotional efforts or formal inter-institutional accords. This institutional engagement has been particularly central to the Project concerning the strategic steering role played by the several social stakeholders represented in the Consulting Councils of CHJs. b. Product 2: Alternate methods of conflict resolution and access to justice implemented 2.6 Beyond the individualized attention to users -i.e. through mediation, conciliation, legal orientation and other personalized services regularly provided by the CHJs in the high-conflict areas where they operate- the Project´s implementation of AMCR in said communities can be also indirectly measured by the number of people who have participated and received orientation and support through the Project´s collective activities for outreach; 2 To be clear, the target of 4 CHJs in Figure 2.1 does not refer to new houses to be created, but to houses that were to be incorporated in the Justice system; and the “achieved” number of 8 does not refer either to the CHJs so incorporated, but to the houses that have been legally registered within the NGO Participación Ciudadana. However, these figures are presented because they do convey the efficacy of the Project in extending CHJs to other vulnerable areas in the country. Figure 2.1 # of Houses registered and Institutional Agreements signed. Figure 2.2 Organizations that promote the CHJs work and services. 5 such as trainings, meetings, door to door campaigns, etc. to promote awareness on issues concerning defense of human rights, protection against violence and public security. i. Protection of Human Rights 2.7 The houses have been active in what concerns the protection of human rights; area where the Project set target of installing, equipping and launching into operation four units to support victims of human rights violations was achieved. Those units have been operating in the houses presently evaluated, in the localities of Cienfuegos and La Joya, of Santiago; Las Caobas, of Santo Domingo; and in that of La Vega. ii. Aiding women on the prevention of violence 2.8 The target set for women in vulnerable areas to receive orientation and support on the prevention of violence was also greatly exceeded by the Project during the period under evaluation. The benchmark indicator of 10,500 women receiving such orientation and support must be understood to mean individual women attended to by the houses on that respect, or attending related training and/or promotion activities, for there may be repeat instances of attention or participation; i.e. the same women occasionally showing up several times with cases of violence, or threatened violence against them, or showing up in house-organized meetings or other activities on the subject matter. The Project aimed at dealing with 10,500 such cases of service or of involvement of women between 2012 and 2015, and ended up reporting almost five times that figure in the four houses evaluated. This attests to the high importance women give the issue and/or the high incidence of violence against women in the targeted communities, as well as the ability of the CHJs to handle such a high interest and demand from the population. Figure 2.3 illustrates how the more than 51,000 cases of attention and involvement were distributed by year of the Project and by the nationality of women. It should be noted that although the great preponderance of cases corresponded -as expected- to Dominican women, and dealing with this segment of the population weighted the most in the Project´s surpassing of its operational goals, the number Haitian women serviced or attending CHJs related activities also exceeded the target of 1,500 cases for the period. See figure 2.4. Unfortunately, the attempt at actually protecting victims of gender crime through the violence unit in the Cienfuegos CHJ did not meet similar success. (See Case Study presented in Annex IV). 2.9 With the performance shown on this indicator, the Project exhibits an important focus on one of its central missions, because, if judged by the high incidence of femicides the D.R. regularly registers in the crime statistics, women living in poverty or near poverty in the targeted communities is one of the most socially vulnerable populations in the Country, especially concerning their human rights to protection against violence. It is worth noting, however, that men are also regularly participating in CHJs activities and use the houses services as well, concerning this and other subject matters; a fact shown by the important male presence in the evaluation´s focus group meetings. The CHJs activities of orientation and support for women vulnerable to violence not only scored Figure 2.3 Women oriented on violence prevention by year, nationality Figure 2.4 Ratio of women getting orientation by nationality 6 high marks in the quantitative achievements of the Project, but also showed continued increase frequency across the board, as the number of women and cases turning up in the CHJs augmented year by year. 2.10 This upward trend can be appreciated in the dotted black line of figure 2.5, depicting the aggregate tendency of all four houses evaluated on this indicator, but also in the continuous, colored lines, depicting the individual trend of each of the houses. Notice that the number of cases or individual women involved per year in the figure corresponds to the accumulated value achieved at each point. This increasing flow might be suggesting either that women give increasing importance to the problem of domestic violence, as the matter keep getting worse for them; or that there is a previously uncovered, pent-up need for such orientation and support that will just keep showing up in the future as additional demand for related information and services; or that both factors are at work. Of course, the possibility cannot be ruled out that the executor´s own pro-activity in reaching out to beneficiaries is largely responsible for the outstanding output results. This is a welcome effort and part of the Project design strategy toward success. At any rate, should this trend keep up as it presently shows, the implication is clear that effective demand on the matter might eventually run up against the CHJs´ installed capacity and resource availability to respond appropriately. iii. Aiding youth in prevention of delinquency 2.11 Similar performance in the delivery of Project expected outputs is observed concerning vulnerable youth receiving orientation and support for preventing delinquency and violence. The indicator of 12,000 youth receiving such attention, as in the case of the indicator for vulnerable women, should also be understood to mean “cases of youth” attended by the CHJs on that respect, or individuals attending related training and/or promotion activities. In this case also there may be repeated instances of attention or participation; i.e. the same youth occasionally showing up several times with cases before the CHJs, or participating in house-organized meetings or other activities on the matter. Here again, the Project stipulated a target of 12,000 such cases of service or youth involved in orientation activities to help avoid violence and delinquent behavior during the period in the targeted communities. The Project exceeded that figure by a factor greater than 2.4 in the four CHJs evaluated. As in the case of the attention to women, this evidences that youth delinquency and violence is also highly present in young people´s lives and interests in the targeted communities. The performance in this area also reveals the CHJs capacity to respond to such high mobilization and demand by the population around the issue. Figure 2.6 is illustrates the number of young people or youth violence/delinquency cases with which the Project got involved, through orientation and support services by the four houses under evaluation in, which also Figure 2.5 Cases, women receiving orientation by CHJ (cumulative). Figure 2.6 Youth oriented on delinquency by year, nationality Figure 2.7 Ratio of youth getting orientation by nationality 7 depicts the number of young Dominicans involved as well as those of other nationalities, during the three annual periods under consideration. Figure 2.7 shows that a preponderance of Dominican nationals were involved in the activities and services as compared with Haitian youth, as expected in this case as well. Other nationalities were virtually absent on this respect. 2.12 Being the years of adolescence a delicate and critical phase in the formation of character and in the honing of behavioral patterns that would become habitual in adulthood, the CHJs high performance in delivering attention and orientation services to the youth should set this Project in the path to make a visible differences in the future crime and delinquency rates in its areas of influence, and the products delivered in this area should also be seen as an important contribution to fulfilling the Project´s ultimate mission of promoting a reduced violence and a more peaceful coexistence in society. An intensification of activity is also observable across the board in this area of the operation, similar to the one observed on the attention to vulnerable women. The number of youth showing up, or youth-related activities, kept mounting through the years studied. If we consider the cumulative number of individuals or cases, rather than just the separate yearly figures, this intensification trend becomes apparent; i.e. as with the attention to women, the turn up rate of youth in the CHJs and the number of youth cases not only increased, but accelerated throughout the period. Once again, this trend suggests that youth delinquency might be gaining momentum as a concern in people´s daily life, or that there was an unaddressed need for related orientation and support that will just keep on turning to actual demand for those services in the future, as the houses presence becomes better known and understood by people in the targeted communities. Again, both factors might be at work arguably feeding on each other to strengthen the trend, and the executor´s pro-activity cannot be ruled out as another factor explaining the salient results, attesting to the Project´s a successful outreach strategy. Also, should this trend preserver it is conceivable that the actual demand for the houses´ services will eventually run up against the CHJs installed capacity, resource availability and flexibility to respond. 2.13 The dotted black line of figure 2.8, depicting the aggregate tendency of this particular benchmark indicator in the four houses presently studied shows this acceleration in the delivery of services. But notice that the continuous, colored lines depicting the individual house´s trends also show the same trend in the majority of them, i.e.: with the exception of the one in La Vega, all houses present upward inflection points. It is important to notice, as well, that even the CHJ that shows an increasing but not accelerating tendency in the indicator -the CHJ in La Vega- nevertheless reached and considerably surpassed the target of 12,000 for the whole Project all by itself; and it did so even before the end of the stipulated period. iv. Training judicial officers and community leaders 2.14 Within output #2 the Project set out to train functionaries from the country judicial system, as well as community organization leaders on the subjects of human rights, municipality issues, access to justice and alternative methods for conflict resolution, as a platform for multiplication of such training down the road to an ever Figure 2.8 Cases, youths getting orientation & support by CHJ (cumulative). Figure 2.9 Functionaries, leaders trained by year, subject. 8 increasing number of community members and citizens at large. Figure 2.9 shows the distribution of individuals trained by the subject and the year of training. As its operational target, the Project aimed at training 400 such individuals by the end of the period 2012 – 2015. Project records at the four CHJs evaluated show that the total of individuals trained reached 545, thus exceeding the target here as well. The specific experience with graduate and college level training is discussed in the qualitative Case Study of Annex IV. 2.15 This operational result of the effort to train trainers is, arguably, a key factor behind the Project´s shown ability to reach a great number of people in the training activities, as well as in counseling and orientation services relating to the relevant subject matters. It also constitutes an important seed planted for the future functional sustainability of the CHJ system, should it become consolidated. 2.16 Each year´s cumulative numbers in figure 2.10 show in the activity of training functionaries and leaders, the familiar upward inflection point and acceleration already seen in other outputs. Judging by these results, the effort put out at the midpoint of the Project by project executors and partners to achieve and surpass the operational targets is obvious in the present case, and no other demand factors can be argued as possible explanations for the salient performance. This again attests to a wining operational strategy and argues for a supply-induced, yet demand￾driven, strengthening of the initiative. v. Training citizens 2.17 The target was to train 4,000 citizens, including promoters in the so-called Human Rights Networks in the communities. The training included the subjects of human rights, conflict management issues of municipality, organization, gender, etc. (See Figure 2.11) The Project exceeded this target by nearly 30%, as the training reached 5,190 individuals in the period. This provides more evidence of effectiveness in the delivery of development products relating to the initiative´s promotion and raising awareness on restorative justice, equal rights and public security. 2.18 Results of this wide outreach effort to train and raise consciousness on the underlying issues among community residents and citizens at large have naturally complemented the training of judicial officers and community leaders examined in the previous section, and showcase the effectiveness of the multiplicative training strategy put forth by the CHJs. It also helps explain the relative speed at which the general initiative has become known and better understood among people under vulnerable conditions, as well as the increasing amount of grievances and conflict cases the four houses received and processed through the years, as will be discussed below in more detail. Remarks similar to the ones made about the cumulative performance trend of benchmark indicators discussed previously can be made about the citizens training Figure 2.10 Functionaries, leaders trained by CHJ (cumulative) Figure 2.11 Citizens trained by year, subject matter Figure 2.12 Citizens receiving training by CHJ (cumulative) 9 indicator. Figure 2.12 depicts such accumulated performance during the period under study, with the familiar accelerating inflection point at mid project and the large margin by which the target was surpassed. Some demand factors and interest on the part of the population targeted must at least partially explain this success. The Project routinely co-opted CSOs and grassroots organizations to launch its training activities, adding to the efficacy of the events´ publicity. Yet, attendance to trainings might have not been only a function of the promotion effort, but also of the relevance that people accorded in their minds to the subject matters of the training. vi. Legal orientation to residents 2.19 In part as consequence of its performance vis-à-vis the operational targets so far discussed, the Project can report a large number of persons resident in the targeted communities who have been provided legal orientation about violations of their human rights, through the CHJs activities. The Project aimed at giving such legal orientation to 130,000 persons in the period under study: an ambitious operation goal and the only one that was not met within Project output # 2. The number of people reached by this legal orientation service is vast, however, as figure 2.13 illustrates showing the annual performance of this indicator. The orientation was about equally provided to men and women in the communities, as depicted in figure 2.14. So, the fact that the more than 104,000 people receiving the service split pretty much half and half between sexes is evidence of both a fairly homogenous focus on the part of the Project and of equal involvement from the part of both the male and female populations with situations concerning human rights in the targeted communities. 2.20 Figure 2.15 summarizes the Project cumulative performance with respect to this indicator. Notice that, despite the pro-activity of field operators and the executing unit, in this case the increasing and even slightly accelerating pace in which services were provided in all houses -except for one whose services kept increasing but somewhat decelerated after midpoint in the execution- the target of 130,000 persons getting legal orientation about violations of their human rights could not be attained. The achievement of 80% of the target appears, however, as an important achievement, given the total number of residents in communities reached by the service. c. Product 3: NSAJ & municipality established and draft law submitted 2.21 As will be extensively discuss later in the present report, this area of development outputs is perhaps the single most crucial one for the long term sustainability of the present project results and for the preservation of the CHJs themselves; yet it is the area in which the Project´s delivery of expected products was less effective. i. Drafting of the NSAJ Law (currently: Alternative Resolution of Conflicts Law) 2.22 Developing a draft proposal for the Law which would formally and definitely incorporate the CHJs into the formal Judicial System of the Dominican Republic, was a key operational goal of the Project, and one indispensable prerequisite for the law itself to be eventually promulgated and enforced. Given the fact that promulgation of such a law is the one requisite to make the development results of the Project permanent and institutionally Figure 2.13 Persons being oriented on H. rights by year, sex. Figure 2.14 Persons getting legal orientation by sex Figure 2.15 Persons getting legally oriented by CJH (cumulative) 10 sustainable in the long run, it is difficult to ignore the enormous importance of delivering this particular output. It is also hard to understate the urgency of the subsequent necessary actions -however not included in the Project￾of submitting the draft law to Congress and achieve approval, for the present initiative to transition from an experiment in restorative justice in a country with a judicial reform and modernization still incomplete and uncertain, to a fully-fledged, recognized and enforceable legal reality. In this context, the Project did deliver in producing a first draft of the law, but did not deliver, completely, in what concerned the further steps envisioned to consolidate the CHJs system, as depicted in figure 2.16 ii. Developing a consensus on the draft law 2.23 There is evidence of due diligence in the consulting process to build a consensus from all parties on the text of the draft law. However, this process did not come to fruition and the draft is still far from being presented to the legislature for consideration. Institutional reservations by representatives of the General Prosecutor´s Office on the role of the several institutions of the Judiciary in implementing the law, and some lack of alacrity in solving those issues by the stakeholders involved, are mostly responsible for the fact that the end of the execution period finds the Project with a draft not entirely ready, or agreed on sufficiently, for presentation to Congress. 2.24 A basic agreement was reached among authorities in the Judicial Systems, and a wide spectrum of opinions from qualified stakeholders in the NGO, CSO and private sector community was taken into account; so that the current version of the draft is more consensual than the original one. Yet, lingering anxieties that the draft might contradict dispositions in other laws or the Constitution itself, possibly triggering unforeseen hurdles to its enforcing, will further delay an acceptable draft until the requisite contextual analysis of its legality is complete. Also, given the DR´s political culture, the fact that the Country is already in the midst of a presidential election campaign ensures that additional delays will be experienced in the presentation of the final text to Congress. So, in the researchers´ judgment, this Project deliverable cannot be said to be more than just 70% complete.3 iii. Incorporating the CHJs program into the National Budget 2.25 Finally, under development output #3 the Project also aimed at having the CHJs funding requirements officially incorporated into the National Budget. The operation of the houses under study has been sufficiently funded during the period, as to allow the CHJs to offer their services permanently in the communities targeted. This has been made possible chiefly by the institutional agreements brokered by the Project with the Supreme Court of Justice, the General Prosecutor´s Office and the several municipalities involved. Also additional sources of funds are being negotiated with Congress, to further buttress the houses permanence. Yet, the legal incorporation of the CHJs operating costs into the National Budget was not achieved. 3 A clarification is in order here: in Figure 2.1 at the beginning of the present section, the expression CHJs registered did not mean “CHJs legally incorporated to the justice system” as required by the relevant output indicator; and neither reference is made in figure 2.16 to such legal incorporation. However, that indicator is an expected measure of output #3 and was not achieved either. One thing is to register the CHJs under an NGO status; but their formal recognition as part of the judiciary is quite another and a condition that can only be brought to existence by the promulgation of the NSAJ Law (currently: Alternative Conflict Regulation Law) which, as indicated is still pending. Figure 2.16 Progress toward NSAJ Law & CHJs national budget 11 2. Project development outcomes 2.26 With very few and noted exceptions, the findings discussed in the present section result not from the examination and verification of project documents, but from a field survey and analysis done separately on a statistically valid sample of CHJs users chosen from the beneficiary list of the four houses under study. The sample was fairly homogeneous; that is: was chosen from mostly comparable households, and included individuals of comparable employment situations at the time of services; comparable groups by age, gender and location of the CHJs; and comparable educational levels. The results of the validation analysis done on the sample -including the χ2 (Chi square) test applied to each relevant comparison to make sure there were no statistically significant differences between the individuals chosen- are presented in Annex II to the present report. a. Observable expected effects 2.27 In the context of the retro-fitting done on this project´s expected Results Chain, and based on the arguments advanced in other sections of this report (see paragraphs: 1.5; 1.6 and 1.10 of Annex I: Methodological Approach and Research Instruments; and Appendix 1.2: Review of project metrics and measurements, for an in depth review of arguments) the researchers measured the Project development outcomes based on a set of benchmark indicators identified at the inception, but much of whose descriptive attributes had to be also re￾worked out to make them conform to acceptable evaluation practices. The indicators focus on: (i) the magnitude of the flow of cases handled by the CHJs´ over the years [RESPONSE EFFECT]; (ii) the proportion of the cases presented to the CHJs that came derived from other branches of the Judiciary [DECONGESTION EFFECT]; (iii) the extent to which the operating costs of the CHJs were covered by in-country resources [SUSTAINABILITY EFFECT]; and (iv) the extent to which the cases presented to the CHJs resulted in settlements [RESOLUTION EFFECT]. i. Project´s response effect 2.28 Although the indicator originally identified for assessing the Project´s response to the need for alternate ways to solve conflicts was coughed in terms of measuring “access to justice” the arguments advanced in the present study (see, in particular, Appendix 1.2, footnote #4) disproved the viability of such measuring and demonstrated the convenience of changing the original wordy narrative: Mediation, legal assistance, public prosecution and emotional support services improve in 30% the Access to justice by vulnerable population in La Vega, Santiago and Santo Domingo Oeste, for the simpler and more practical: the number of cases of conflict processed by the CHJ through AMCR increases by 30%. 2.29 Thus defined, the indicator resembles more the measure of a project output than a proper outcome metric. Yet, to the extent than an increased number of people showing up at the CHJs´ doors with their grievances and conflicts is a measure of an improved awareness of ways toward justice, other than the ordinary civil and penal venues and processes -and shows an increasing knowledge and recognition of the role of the house itself- the indicator re-worded that way is arguably an indirect or at least precursor measure of attitudinal changes occurring in the population concerning restorative justice, conflict resolution and the protection of human rights. For the latter reason, the researchers accept the reworded metric as indicator of an early development outcome, possibly induced by the Project. Unfortunately, no full treatment of this metric -i.e. as a true benchmark indicator- was feasible because no sufficiently clear baseline data was provided to warrant the computation of a change in numbers. Hence, the present study resorts just to count the number of cases of conflict processed by the CHJ through AMCR reported in the Project´s internal files, as a measure of this “response effect”. 2.30 An additional difficulty in using this metric arises from the way the data for it was collected during execution. The figures presented in the data file for the indicator appear to lump together all individuals -including repetitions￾who received any one of all the services provided by the houses; including training, promotion of awareness, emotional support, etc. related to, but not always referred to, the actual processing of conflict cases brought by 12 the individuals themselves. Therefore, data compiled that way arguably do not measure the extent to which the Project helped individuals to access restorative justice through AMCR, which is the original intent of the indicator. Persons covered (all activities) Persons covered (individual services) Persons involved in conflict cases Conflict cases processed by the CHJs PERIOD: JUL 2012 – JUN 2015 120,763 1 90,097 2 32,141 1 16,618 1 2.31 Table 2.1 clarifies the different meanings of metrics. Accordingly, the total number of individuals/cases covered - i.e. receiving services/attention one way or another by the Project, including collective activities (such as training and meetings) and individual (personalized) services- reached 120,763 from July 2012 through June 2015, with the table showing the various subsets of services provided by the CJHs during the period. Table 2.2 shows that this total splits evenly between the female and male population served. Now, although there is no indication that older people were necessarily targeted more than the youth, 36 years old and older individuals were covered more than twice as much as people of 18-35 years: a probable reflection of actual demand. Of course, given the arguments advanced in the previous paragraphs, the figure for total coverage is not a proper measure of the extent to which the Project helped people, through personalized services, handle conflict cases actually brought up before the CHJs. A closer measure to that Project response would be the figure of 90,097 registered as the total number of cases reported coming in to be heard, and processed, by the CHJs during the period. 2.32 It is important also to understand that not all cases of conflict that were brought or appeared before the CHJs were eventually attended or processed by the houses themselves; for a portion of them were, in all likelihood, not amenable to the application of AMCR; and so, on the contrary, were dismissed, derived away to the ordinary justice channels or otherwise, through legal orientation. So strictly speaking the figure of 90,097 does not represent either the services given by the Project in the specific realm of alternative justice. Consequently, the only acceptable measure of the Project “response effect” in terms of offering mediation and conciliation services for individuals is represented by the 16,618 cases so processed. (See 4th column of table 2.1 and 3rd of figure 2.17) ii. Project´s judicial decongestion effect 2.33 One of the chief development outcomes expected from this project is the effect of helping to relieve the gridlock in the ordinary justice process, by providing a venue for cases better suited for restorative justice; thus also benefitting the justice administration as a whole. However -as researchers presently proved- the original narrative of the designated indicator: the program processes 20% of cases derived from the General Prosecutor´s office and the Ministry of Justice was equivocal and inappropriate for what was the intended measurement. The implicit syntactic mistake made in the original indicator was never corrected during project execution, but in other parts of the project documentation the indicator’s narrative was changed to the more cogent: 20 % of cases processed by the CHJs were derived to them. Total, % covered (all activities) Female 61,285 51% Male 59,478 49% 18-35 years 37821 31% 36+ years 82,942 69% Table 2.1 Population covered, conflict cases handled. Sources: (1) Project coordination; (2) CHJs data files Figure 2.17 Aggregate individuals, cases during the Project Table 2.2 Persons by gender, age. Source: Project coordination. 13 2.34 The study incorporated that new definition of the indicator as a “proxy” measure of the “judicial decongestion” effect that the Project was expected to induce. Unfortunately additional ambiguities, in the indicator’s data file and the manner in which the data was gathered in execution, resulted not in one, but in two alternative definitions implicit in the re-written indicator; namely: (i) 20 % of cases processed by the CHJs were derived by the General Prosecutor´s Office and the Judicial System, which for clarity in the present study we have code-named: b indicator; and (ii) 20 % of cases processed by the CHJs were derived by the General Prosecutor´s Office, the Judicial System, and other public institutions, which we have code-named: c indicator. For the record, both the b indicator and the c indicator were taken into consideration, although the latter does not measure the effect of judicial decongestion in a strict sense, for it includes among the possible originators of cases derived to the CHJs, institutions which do not truly belong in the judicial system, such as the municipalities. Results from this double-meaning metric are summarized in figure 2.18., which shows indicator c yielding a different higher number, for the reason just mentioned.4 2.35 At any rate, under either one of the 3 possible ways to compute the metrics (the b and c alternatives or the one explained in footnote 4 below) the project did attain and marginally surpassed the target set of 20%. Also, the Civil Courts were by far the main originators of cases derived to the CHS, as can be seen in Figure 2.19, which depicts the proportion of cases which were referred by each of the public institutions involved. This result was somewhat expected, since not outright criminal cases but those of a civil nature are more amenable for treatment through the CHJs alternative justice means. So the metric that really can claim to measure the judicial decongestion effect we refer to herein does born out, at least indirectly, a possible such effect having been induced by the Project. 2.36 Further down the text a qualitative analysis of this outcome is offered; but, from a quantitative performance point of view, the fact that an increased portion of the cases processed were derived from the ordinary justice can be viewed as an indirect evidence that the Project might indeed have started to alleviate the judicial workload. However, when this indicator is analyzed in terms of year by year trend, salient and curious facts are revealed, and ones that throw doubts about the conclusions on the decongestion effect that appear to follow from the total aggregate figures for the period as a whole, both under the b and the c meaning. Figure 2.20 depicts the % of cases processed accumulated by the end of each period, which was derived from public institutions (c indicator). As customary in 4 Notice that captions at figures 2.18, 2.19, 2.20 and 2.21 indicate not “cases”, but “persons” derived. The reason is that the data here come from the CHJs data files, where the numbers registered as cases really refer to persons. A third way of calculating the metric, applied centrally by the Executing Agency -which reportedly counts true cases and not individuals￾compute the value of the indicator at 29% by the end of the Project. Figure 2.18 Persons derived from institutions, accumulated Figure 2.19 Persons derived by period, originating agency Figure 2.20 Cumulative persons derived as a % of total (c) 14 other graphs of this sort above, the black dotted line represents the trend in the yearly accumulated values for the four houses as a whole, and the colored lines represent the progression of values for each separate house, with the exclusion of the CHJ in La Vega. The exception of La Vega is the first salient fact revealed by this analysis, and one that plays a considerable role in its conclusions. The numbers for La Vega CHJ are not shown in figure 2.20 because in that house the ratio of cases derived to total cases processed is equal to 1. In other words, 100% of the cases processed by this house during the period were derived from other public institutions. Inclusion of those figures in the chart would have resulted in a flat line way above that of the other houses, where not all cases processed came referred that way, and would have obscured another salient fact of the analysis; namely that, as can be noticed and contrary to what has been observed in other graphs of this sort above, the trend is descending as a whole and for the other individual houses with the exception of that of the CHJ in Las Caobas, which is actually upward sloping. 2.37 Notice that the same exception for La Vega -for the same reason (100% of cases processed being derived)- and the same downward sloping trend are observed in the numbers for the b indicator in figure 2.21; i.e. counting only as originating institutions the General Prosecutors Office and the Judicial System. Notice also that the high rate in La Vega maybe responsible for the generally high aggregate rate for the Project as a whole; especially compared to the baseline value in 2012. But the high, flat line that La Vega would have exhibited plus the upward sloping trend in Las Caobas were not sufficient to compensate for, much less reverse, the aggregate downward sloping trend in this indicator. The high rate in La Vega (the newest of the four houses studied, and the one whose scale of operation is not commensurate with the other three) is clearly a factor in producing the high aggregate rates of 28% and 30% for indicators b and c in 2013; virtually ensuring that the trend could not go but downward the following years. Yet, off-the￾chart rates in July 2013, like 100% for La Vega and 58-60% for La Joya, do not jibe well with the aggregate rate of 10% the Project documentation claims to be the metric’s base line value just a year earlier (June 2012). At any rate, the observed downward trend suggests that the Project’s judicial decongestion effect diminished constantly throughout the period, and reached the target only by coming down from peak derivation values in 2013. iii. Project´s sustainability effect 2.38 In terms of sustenance, the Project target was that, starting from a proportion of 55% of the CHJs’ operating costs being covered by Judiciary institutions, Municipalities and the Private Sector of the DR in mid 2012, a 100% of said costs would be eventually covered by such domestic contributions. This target is taken to be a benchmark outcome indicator because it can denote changes in the attitude of stakeholder toward the long term sustenance of the CHJs’ effort; and the greater the progress toward 100% coverage of operating costs by such national stakeholders, the greater their commitment should be presumed toward principles represented by this initiative and toward its consolidation in the long run. 2.39 On this respect the evidence suggests that the Project has had some effect on the sustainability of the houses, by raising sufficient funds from domestic sources, both in cash and in-kind contributions, so that the CHJs have been able to operate without interruption during the period evaluated. Particularly, the income & expenditures data provided by the executors purport that the cost-coverage target has been actually attained (See table 2.3). Jun. 2012 Cost coverage: 55% Jul. 2012 - Jun. 2015 Domestic contributions: RD$62,232,619 Operating costs: RD$57,589,315 Cost coverage: 108.1% Table 2.3 Domestic funds, CHJ costs Source: Project coordination Figure 2.21 Cumulative persons derived as a % of total (b) 15 These numbers, however, do not permit to draw conclusive findings on whether the Project has had a lasting effect of the CHJs sustainability, since the data do not come from an accounting audit of funds flows and financial results statements from the houses. Beyond the need for a quantitative audit review of the CHJs costs accounting records, as well as of their cash and fund flows statements, additional efforts to deal with the institutional determinants of the CHJs financial sustainability should be put in place. These qualitative issues relating to sustainability are addressed in Annex IV. iv. Project´s resolution effect 2.40 The best defined metric and clearest evidence of a development outcome induced by this project is the one offered by the benchmark indicator: 50% cases processed by the CHJs reach agreements or solution through AMCR. The unequivocal definition of this indicator and the unambiguous way in which data for it was collected during execution allow to assess what we have called the Project’s resolution effect: i.e. the extent to which the Project have helped individuals settled the conflicts they have brought to CHJs’ attention. The base line value stated in the Project documentation was that by mid 2012 the resolution rate of the CHJs was 30%; i.e. less than 1 in every 3 conflict cases brought to the CHJs got to be settled through some restorative justice solution. 2.41 The fact that the Project attained the target in this outcome indicator, i.e. that for the period analyzed the CHJs have, through their agency, made possible that at least half the cases handled by them through AMCR reach some sort of settlement (see Figure 2.22) is a factor contributing to the level of credibility the initiative has achieved through the years. Now, given the importance of this outcome for the development effectiveness of the Project as a whole, the researchers did not limit their examination to the data on file provided by project executors, but investigated the matter in the field including a direct verification question posed to the house users during the survey, about the actual results they got from the services provided by the CHJ they approached for help. Tests on results yielded high statistical significance: Chi-square (1)=5, p=.025; medium effect size w=.25; medium high Power=.61. 2.42 Field findings do not disproof the Project’s success rate reported by the executors in what concerns the resolution effect. Actually, 80% of respondents to the field survey indicated having achieved some sort of resolution of their cases through the agency of a house. Statistical tests performed on the data gathered also showed no significant differences of results that might have been due to the gender or age of the respondents. Figures 2.23 and 2.24 show how consensual was the answer received from the individuals surveyed, when classified by gender and age, as they corroborated to have achieve a solution on the issue or conflict situation they brought up to the CHJs’ attention. 2.43 Also the statistical test shows no significant differences either in the response of individuals that might be associated with any particular house. The answers received from the survey respondents were fairly Figure 2.22 Cases processed & settled Figure 2.23 Respondents with solved cases by gender Figure 2.24 Respondents with solved cases by age 16 homogeneous when classified by the location of the house, corroborating that when they solved the issue or conflict situation they brought up to the CHJs’ attention, they did so regardless of what location they visited. Figure 2.25 shows these results, which confirm how consensual the users´ opinion was about the resolution effect of the Project, as measured by the indicator used herein. There are however other aspects of the resolution effect, relating not to the quantity of cases resolved, but to the quality of the solutions reached, whose measure provides valuable information in terms of satisfaction, nature and variety of cases, difficulty in settling conflicts and perception of justice. Such aspects, not originally intended for measurement by the project can, however, give additional context to the findings on outcomes, and were included in the present field research. Their analysis is presented below. b. Other project results i. User satisfaction 2.44 In recognizing that the Project’s resolution rate is one issue, and whether or not the CHJs’ users are content or not with the way a case was handled is another, the researchers included in the field survey a specific inquiry on the satisfaction about the services rendered. It is certainly conceivable that individuals find the end results of mediation and conciliation acceptable, and still feel uncomfortable or even resentful about the manner the process was conducted or people were treated, etc. Conversely, a case might not have come to a resolution, or might have been settled in a way not completely satisfactory to the plaintiff, and yet he or she might still feel to have been treated correctly, or that everything reasonably possible was done to solve the conflict, etc. Hence the question on satisfaction posed to the individuals surveyed and the tests applied to the results on the matter, to ascertain the statistical significance of the answers. For the answers to the question, the survey presented a nominal scale of satisfaction, including the grades of: NONE, LOW, MEDIUM and HIGH for respondents to choose from. The resulting survey data show that 85% of respondents were highly satisfied with the services received by the CHJs in handling their cases, and that the satisfaction expressed did not vary significantly with the gender or age of respondents, or with the location of the CHJ visited by the persons surveyed. Figure 2.26 shows this high satisfaction expressed by respondents of both sexes. 2.45 No significant difference in the frequency of the high rate given to the services by beneficiaries in the targeted communities were observable either in individuals of different age groups or by people attending different CHJs, as can be seen in figures 2.27 and 2.28. In all groups, the same high satisfaction level is expressed by the preponderance of individuals surveyed. Finally, to test how or whether the formal schooling of the respondents Figure 2.25 Respondents with solved cases by CHJ Figure 2.26 Users satisfaction with services, by gender Figure 2.27 Users satisfaction with services, by age Figure 2.28 Users satisfaction with services by CHJs 17 may influence the level of quality they assign to the CHJs services, researchers use the educational level of individuals as an independent variable against the response on satisfaction. The levels used for the education variable were: BASIC and below; HIGH SCHOOL; TECHNICAL; and COLLEGE and above. Figure 2.29 shows how the levels of education have a small influence on people giving high marks to the services provided by the CHJs. Notice how the differences hover around 3 to 7 points above or below the 85 mark% for the high satisfaction opinion. The exception in figure 2.29 is the case of technical education people, 100% of whom expressed high satisfaction. 2.46 Therefore results discussed in this section generally confirm a fairly widespread high approval among users across the board of the quality of CHJs services received. This factor also may have contributed to the credibility the Project and the CHJs have achieved in pursuing their mission. These results are also obtained against the backdrop of a wide variety of problems or conflict situations that motivated people to appear in the CHJs, and triggered the full gamut of help, support and orientation provided. So the observed satisfaction levels do not appear confined to any particular type of issue confronted, or service rendered; even though some differences were found in the “success” (resolution) rate depending on the problem or conflict in each particular case. Also the vast majority of processes reported by the individuals surveyed corresponded to mediation services. These additional qualifications of the Project outcomes are discussed below. 2.47 The services least mentioned by respondents in the survey are the ones related to training activities by the CHJs; yet, respondent who did mention them rated the training as being highly practical. The vast majority of services reported were of the personalized kind; i.e. the ones rendered as a result of individuals bringing up personal problems or conflict cases to the attention of the house. As can be seen in Figure 2.30, the great majority of the reported services consisted of Mediation (84%) followed by CHJ prosecutor services (8%) and Legal Assistance (7%). The test showed this time a high statistical significance for this distribution: Chi-square (3) = 189.65, p = 0, with a very high effect size (w = 1.36) and a perfect power (1). Figure 2.31 and 2.32 present the analysis of individuals receiving the different types of personalized services when classified by the independent variables of gender and age. Figure 2.29 Users by satisfaction with services, formal education Figure 2.30 Types of service received by respondents 18 2.48 that Now, despite the figures showing more male users receiving mediation services and more female users receiving prosecutor and legal orientation services, as well as more youth receiving mediation and legal orientation services, and older people receiving in greater proportions prosecutor services, the chi-square tests show there is no statistically significant differences induced by such variables in the proportion of people getting those services. The distribution of cases in these figures still show the great majority of surveyed individuals receiving mediation services, as compared to those receiving the other types of service offered by the CHJs. The same can be said about the difference in proportions of individuals in the targeted communities receiving each type of service when classified by the location of the house they approached to request help concerning their problem of conflict situation case. See this analysis in Figure 2.33 2.49 The underlying issues, problems or motives that compelled the individuals surveyed in the target communities to request the houses’ services also ran a wide range of different cases. They included: (i) collection of debts; (ii) disputes between neighbors; (iii) family conflicts; (iv) work paid and undone; (v) rent/tenant problems; (vi) domestic violence; (vii) threats; (viii) rape; (ix) alimony payments; (x) division of property; (xi) damage/loss of property, etc. Almost half of the conflicts involved in the cases brought by the individuals surveyed referred to the issues of: a) debt collection (26%); b) rent/tenant problems (13%) and c) overdue alimony payments (11%). See the total proportion distribution of cases depicted in Figure 2.34. In the fourth place of frequency (10%) came the cases lumped together under the category of other; which included conflict situations involving defamation, theft, invasion of property, etc. Therefore, when answering the question about satisfaction the respondents to the survey were judging the quality of the houses’ services not only on the simple, easy to redress grievances, but on cases that ranged from Figure 2.31 Services received by respondent’s gender Figure 2.32 Services received by respondent’s age Figure 2.33 Services received by CHJ visited Figure 2.34 Respondent’s reasons for appearing before CHJs Figure 2.35 Respondent’s reasons for appearing before CHJs, by gender 19 relatively common conflicts, such as rent/tenant problems, or work paid and not done, to more serious and grave wrongdoings, such as cases of rape and or domestic violence. It is also important to note that, although both genders were involved in most underlying issues of conflict, some of the reasons or motives for bringing up cases before the CHJs were, as expected, the exclusive o preferred provinces of female or male plaintiffs. In effect, notice in Figure 2.35 that problems mentioned exclusively by the males are damage/loss of property, paid work undone and theft; while the females complained more frequently about rent/tenant problems, alimony payments, family conflicts, threat and rape. 2.50 When classified by age the frequency of motives to seek help also involved most of the reasons in the range, although here too some problems are predictably seen to be more pressing at particular times in life. Figure 2.36, for instance, shows the same frequency of debt collection cases by individuals of all ages and sexes; but the young did not brought alimony payments as a problem, while the old did not mention rape. Figure 2.37 shows the frequency of motives by CHJs and some local particularities are noticed. For instance, while generally high, the frequency of respondents with problems of debt collection was notably higher in the CHJ at La Joya, a house that also stands out in cases of division of property. Respondents at Las Caobas CHJ stand out in domestic violence cases, and those at La Vega CHJ in neighbors disputes. Another factor giving context to the findings about the resolution rate and beneficiary satisfaction is the relative difficulty observed in solving cases, depending on the problem. Figure 2.38 shows the frequency of success/failure to settle by type of case. Notice how the cases with the highest settlement rate are those on the first half of the list: debt collection (29%); rent/tenant problems (16%); and alimony payments (14%); while those in the 2nd half, such as division of property, had a lower success rate. Also notice that when lawyers were hired or properties were damaged, lost or invaded, the settlement rate was zero. ii. Perception of justice 2.51 The researchers also recognize that even if a case has reached settlement and the plaintiffs are satisfied with the services provided, the perception of whether “justice” was rendered or not by the solution reached -however voluntarily accepted- is still a separate issue for people. In other words, individuals might Figure 2.36 Respondent’s reasons for appearing by age Figure 2.37 Respondent’s reasons for appearing by CHJs Figure 2.38 Frequency of settlements by reasons for appearing before CHJs Figure 2.39 Respondent’s perception of justice by age, gender 20 appreciate that a “solution” was found to their problems and recognize the quality of treatment during the process, and still feel that the settlement achieved was not quite “fair” compared to their own expectations. To assess this nuance about the Project’s resolution effect, the survey included an inquiry about the perception of justice received by individuals, offering a differential semantic scale with a numerical correlate and the following alternatives for respondents to choose from: Agree on the highest extreme (5) and Disagree on the lowest extreme (1), with the numbers 2, 3 and 4 as options to register intermediate degrees of agreement. Figure 2.39 shows that, on average, individuals surveyed expressed a perception of justice higher than 4 across the board on the scale applied for measuring. This comes out of an analysis of variance with gender and age as independent variables, and the perception of justice rating from participants as dependent variable. As shown, the result is that as they interacted the former variables failed to produce any significant differences in the ratings; i.e. the average perceptions by gender and age remained consistently between 4.1 and 4.5, at the positive end of the scale.5 2.52 Also using the perception of justice thus measured by the individuals surveyed as dependent variable, an analysis of variance was performed with the location of the CHJ as independent variable, to find any statistically significant differences in expressed opinions that might be due to particular CHJs. Here again, all the average responses on the perception of justice were very high, ranging from 4.38 to 4.63 in the scale, and no significant differences were found due to the location of the CHJ visited, as can be observed in the comparison depicted on Figure 2.40. So, the high perception of justice rendered is also a solid, widespread result from the works of the Project and the houses, as expressed by the beneficiaries surveyed, regardless of gender, age or the particular CHJ individuals brought their grievance cases before. iii. Process recognition 2.53 An additional question was included in the survey about the reasons behind the perception of respondents, to get an idea of the connection they made, or not, between the degree of “justice” received and the nature of the 5 Because the AMCR approach to solving wrongdoings is qualitatively different from that of the ordinary justice, -which is rendered through judicial sentences that ultimately can acieve the character of “things definitively judged”- the notion of restorative justice is a relatively involved philosophical concept which can be arcane to the non initiated. Therefore, the inquiry of the present investigation about the “perception of justice” by the users in the CHJs target communities do not seek to assess whether or not the respondents understand the nuanced difference between both concepts of “justice”, but merely ascertain if the individuals have a common-sense feeling that the settlement reached was fair to them. Also, even though it cannot be reasonably expected that answers from the people surveyed could really go beyond this common-sense perception, the surveyed included additional consistency-checking question, to determine if respondents understood the question asked. Figure 2.40 Respondent’s perception of justice by CHJ Reasons for perception of justice Sign Mean N satisfactory agreement + 5.00 1 good service / support + 5.00 6 The other family was summoned + 5.00 1 I witnessed = 5.00 1 there were no witnesses = 5.00 1 the problem was solved initially but then persists = 5.00 44 the problem was solved but not in a timely manner - 3.50 2 the other party did not showed / did not meet - 3.25 8 the case never left research / it was dismissed - 3.00 2 the CJH said it was not competent / no clarity of roles - 3.00 2 in the CJH they only advised me - 3.00 1 I had to change my attorney - 3.00 1 The case is not solved - 2.00 1 we could solve the problem + 2.00 2 Total 4.48 73 Table 2.4 Reasons for perception of justice. Source; Evaluation survey, Feb 2016 21 process undertaken to reach the solution. Table 2.4 summarizes the answers verbatim. The first column of the table contains the reasons openly given by respondents for the perception of justice they expressed in the survey. The second column classifies the reasons by sign: + for reasons judged as positive towards the CHJ; = for the reasons considered neutral; and – for the reasons judged as negative. The third column of the table contains the mean value (average) of the perception of justice rating expressed, corresponding to each reason given; and the fourth column shows, as N, the number of people in the group giving one particular reason. The results do confirm that beneficiaries were consistent in the cause-effect connection they make between processes and results in the context of their perception of justice. Notice that the reasons considered positive or neutral correspond to the maximum average rating of 5.0 points given to results; while the negative reasons given correspond to mean values lower than 4.0 points. A lone exception to this consistency is the case of two respondents who expressed low perception of justice received (2.0 points) and gave a reason for this that can be considered as somewhat positive; namely: “we could solve the problem”. The mean values in the table were compared using an analysis of variance, which resulted in a very high statistical significance, F (13, 59) = 8.54, p = .000, with a very high effect size (f = .91) and a very high power (.99). 2.54 A salient fact in this analysis is that more than half of the individuals surveyed happened to indicate that “the problem was solved initially, but then persists”. This is a circumstance not strange to the processes associated with the restorative justice approach and the application of AMCR; i.e. the settlements reached through the approach’s prime procedure (mediation) are by definition voluntary agreements that, unfortunately, can later on be broken or not recognized any longer by one of the parties involved. Now, once understood that this is a perfectly possible outcome in the nature of mediation itself, the point to stress here is that while recognizing that their problems “persists” respondents do not hold that against the results facilitated or services provided by the CHJs, and still give a high average rate (above 4.0) to the solution achieved in terms of perception of justice. Arguably, this is evidence that the CHJ users’ understanding of issues goes beyond being consistent in the perceived connection between processes and results, and reveals a deeper comprehension of the nature and limitations of the restorative justice concepts and methods. 2.55 The reasons associated with the perception of justice were also analyzed by gender and age in order to assess any difference in such reasons due to the respondents’ belonging to the different groups. We omit the analysis by CHJ, because not all the reasons were mentioned in connection with all the houses. As before, no differences were detected either in the reasons for the expressed perception of justice, that were possibly due to the informants’ gender. In Figure 2.41 we can see that, as mentioned, the most important reason given was: “the problem was solved initially, but then persists” (average 60% between the male and female respondents). The same absence of statistically significant differences was established by the analysis of reasons given by age groups, as depicted in Figure 2.42. III. Analysis and conclusions Figure 2.41 Reasons for perception of justice by gender Figure 2.42 Reasons for perception of justice by age 22 3.1 The following text presents the analysis of the research findings discussed in the preceding chapter, and lay out the conclusions of the present study about the research questions it set out to answer at the start, concerning: (i) the Project’s observed development results; and (ii) the sustainability of those results and lessons learned from the Project execution. A. On the project development results 3.2 Within the conceptual framework proposed and adopted for the present study (see summary in paragraphs 1.3 - 1.6 above) the first two research questions posed in paragraph 1.2 semantically overlap. They do so in the general sense that both address the Project’s development results, and in the specific sense that they both address the development results in terms of access to justice. The first question inquires about how the Project’s “influenced the targeted communities in what concerns access to justice, respect for human rights and citizen security”. Within the study´s conceptual framework, this inquiry is understood to mean: what transformations induced by the project have had short-to-medium effects on the targeted community’s reality (i.e.: development results at the level of outcomes) concerning the 3 mentioned socio-institutional conditions. The second is also a question about development results, but here the inquiry appears to include not just the outcomes, but also results at the level of outputs (i.e.: project deliverables) and focuses exclusively on the issue of access to justice. The analysis that follows, therefore, addresses the Project’s observed development results in terms of both outputs and outcomes, in this order and as identified in the Project’s theory of change or expected results chain. (See Annex I, paragraph 1.1). The analysis is also aligned with the research premises herein adopted that: (i) the evaluation focuses first and foremost on determining whether the Project achieved or not its own expressed goals (see paragraph 1.2); and (ii) the evaluation applies appropriate metrics and measurement requisites as they were stipulated ex-ante, during project design (see Annex I, paragraphs 1.5 and 1.6). In this context, and given what was explained in the section D. Methods & limitations and referenced annexes, it must be clarified from the start that the only project outcome that could be properly assessed within these methodological premises is the one concerning access to justice. Objectives concerning respect for human rights and citizen security, although a rhetorical part of the project goals’ narrative, were not backed by appropriate indicators, nor by clear base line values or gauging procedures, that would make measurement feasible. The few indicators available on respect for human rights and citizen security refer to project training and orientation outputs deliverable by the Project, and they are reported on in the present evaluation; but no measurement of transformations that may have been induced concerning these issues was possible. However, other project results closely associated with the access to justice outcome, though not included in the originally identified metrics, were measured by the present study. The analysis and conclusions on this other results are herein presented. 1. Project performance in delivering expect outputs 3.3 As shown in detail on Chapter II, the project attained or exceeded most of its operational targets on deliverable products; especially in what concerned: (i) setting the organizational conditions (agreements, institutional alliances, co-opting of grassroots leadership, etc.) for a robust execution from Jun 2012 to July 2015 (expected output #1); and (ii) the expansion of coverage and diversification of individualized services to access alternative justice as well as launching collective activities for massive outreach (trainings, discussion meetings, door-to-door campaigns, etc.) on issues of human rights, protection against violence and citizen security (expected output #2). a. Image and credibility of the AMCR approach 3.4 High performance in these two respects has provided a strong footing for the consolidation of the CHJs image, and for the credibility achieved by the restorative justice methods in the Dominican society and Judicial System. As proof of this, there is abundant evidence of an active integration to the effort by the public sector, including all branches of the Judiciary and the municipalities involved, which have contributed with cash and personnel, as 23 well as by social and private organizations that have shown various degrees of commitment and modes of collaboration, specially in-kind contributions, pro bono work and functional complementation of services. An example of result from this collaboration that merits mentioning is the successful training of judicial officers and community leaders, which showcased the effectiveness of the multiplicative training strategy (training trainers) put forth by the Project. The success of this strategy also helps explain the relative speed at which the houses have become known and understood by residents in comparison to the ordinary Justice System, as well as the increasing number of grievances and conflict cases the houses have received and processed during the period studied. The positive outlook warranted by the above results is somewhat tempered by the performance in the single operational area in which the Project fell relatively short; namely: the creation of the necessary formal conditions for the institutional long term sustainability of results; conditions that were contemplated in expected development output #3. The implications of this last point are discussed further down in the present text. b. Unleashing of a pent-up demand for alternative justice 3.5 The delivery of operational outputs was not only high but also visibly accelerating by the Project midpoint, and so a considerable pro-activity on the part of the Executing Partners and field operatives is presumed behind these high performance results. The key and successful Project strategy of community penetration through interfacing with grassroots organizations and recruiting natural leaders in favor of the initiative, combined with important early efforts made to co-opt other interested NGOs, CSOs and private sector organization must also have played an important role in the acceleration of the Project’s performance and in surpassing the operational targets concerning collective activities of outreach. Yet, beyond the diligence of Project executors, it is difficult to explain the success of the outreach effort without recognizing the presence of demand factors and interest of the targeted population on the relevant underlying issues. The evidence seems to suggest that, in addition to the fact that the Project routinely prompted the collaboration of key organizations in launching the training and consciousness-raising activities, the high attendance to them could not only be a function of the promotion effectiveness, but also of the relevance that residents in the communities accorded the themes involved 3.6 On the face of it, the observed flow of women and youth largely exceeding the targets for participation in training or discussion meetings on domestic violence and delinquency might be at least partially suggesting that either: (i) the challenges of domestic violence and delinquency were gaining momentum as a concern in people´s daily life; or (ii) there was as large unaddressed need for orientation and support on those matters; or (iii) both factors were at work simultaneously. Pointedly, the evidence shows that the targets were also surpassed in the individualized services rendered (i.e. the case by case mediation, conciliation, legal orientation to persons, etc.) which was most likely a result of success in the collective activities of training and raising awareness. Now if, by virtue of their own promotion, the CHJs’ existence and mission become increasingly known and understood by residents in the communities and elsewhere, it would be reasonable to expect that a previously uncovered need and pent-up interest in the issues will keep showing up as increased demand for related services in the future as well. Even in the absence of outreach, the word-of-mouth spreading of the CHJs’ reputation -as shown, for instance, in focus groups- may become a self-feeding mechanism eliciting additional demand. So, whatever factors may be at work explaining the high performance herein, it follows that if the observed accelerating trend in activities keep up, as during the 2012 – 2015 period, the said effective demand might eventually run up against the CHJs scale, installed capacity and resource availability to respond appropriately. 2. Project influence on the target communities concerning access to justice. a. Response to demand for restorative justice. 3.7 As discussed previously in this report, the one indicator initially identified in the Project to measure the outcome: changes in access to justice had to be re-worked, to correct its definitional flaws, and eventually converted to a 24 measure of the number of cases of conflict processed by the CHJ through AMCR. Although when defined in this manner the indicator resembles more a measure of project output, the argument for treating the metric as an outcome indicator is that, the extent to which individuals appear before the CHJs´ with their grievances is a measure of their awareness of the existence of alternative ways to redress wrongdoings, beyond those of the ordinary justice system. Such awareness would also suggest knowledge and recognition of the CHJs functions themselves. Therefore, the indicator thus defined has been taken as an indirect or, at minimum, precursor measure of changes in attitudes of people towards conflict resolution through restorative justice means; which is a development transformation the Project attempted to induce. Despite that no change in the metric could be measured -even after the rewording- because no unambiguous base line value was provided, the measured indicator does show that the Project “response effect” was indeed present in absolute terms. b. Contribution to the general administration of justice 3.8 Help unburdening the ordinary justice system from cases more amenable to be processed through alternative justice means, but that still contribute to caseloads and gridlock in courts and prosecutors’ offices -and thus improving the administration of justice in general- was another major development outcome the Project sought. Now, despite the fact that the indicator originally designated for this measurement was also flawed and had to be re-written -to measure the % of cases processed by the CHJs that were derived by the ordinary justice branches- and despite the fact that there was three different ways of computing the same indicator, results show that the target set for the outcome (20% of cases processed being derived) was also attained and even marginally surpassed.6 The numbers that the three computational procedures yield for this rate, in the said order, are: 22%, 24% and 29%; and the Civil Courts appeared by much to be the main originators of cases or individuals referred during the period. This latter result was expected, since not individuals complaining about crimes but cases of a civil nature are more likely derived to mediation, conciliation and the other CHJs services. So, the available data does born out, at least indirectly, an incipient judicial decongestion effect induced by the Project; whether it refers to deriving individual persons, or deriving actual cases involving at least two each. Now, the ambiguity in the data caused by the alternative possible interpretations here is a reason for pause. The interpretation: “persons derived attended to” (22%, 24%) can imply social or psychological support, etc., which strictly speaking fall out of the “justice” realm. So the interpretation: “processed conflict cases which were derived” (29%) might be the better gage for a judicial decongestion effect due to the application of restorative justice. 3.9 The analysis of this indicator’s trend, however, reveals facts that qualify the conclusions about the decongestion effect. Specifically -and in the strict sense of “individuals derived”- the data demonstrate that during the first year of the Project the effect jumped noticeably above the reported base line and then fell steadily ever since, both in yearly and cumulative terms. And it did so despite the fact that one of the houses scored a flat trend and another scored an upward trend, in the rate. The very high early values (2012-2013) for the % of individuals derived from public institutions made it difficult for the tendency of this ratio to go in a direction other than down, later on. Yet, this does not seem to fully explain why the indicator reached its targets “from above”; i.e. only by descending from a high derivation rate in 2013. There is abundance of quantitative and qualitative evidence that the CHJs have been increasingly seen by the institutions of the Judiciary as a legitimate mechanism to hear cases that are less suited for the ordinary justice venues and, therefore, as an allied for the decongestion of their 6 The three ways of calculating the indicator come from: (i) the two narratives found for the metric (b and c indicators) as described in paragraph 2.34; and (ii) another possible interpretation (already alluded to in footnote 4 of the same paragraph) that springs from the manner in which the rate was actually constructed. In the CHJs data files the numbers registered as “cases” really refer to “persons”; while the way the rate was constructed by the Executing Partners count cases, not persons. The distinction is quantitatively important as well, because a derived person means exactly that: one individual who is referred to the CHJs; while a derived case refers to two or more individuals involved in a case (conflict). 25 workload. So, some decongestion of those venues must have taken place as a result of the Project, and proof of this is that the absolute number of individuals derived, as registered in the CHJs data files, never decreased during the period. On the contrary, it increased year by year. The fact, then, that the rate of individuals derived to total individuals attended decelerated must be clearly due to a more than proportional increase in the denominator of that rate. The conclusion is, therefore, that the downward sloping trend observed in the rate of derivation is a function not of the number of plaintiffs derived (numerator) declining, but of the total number of persons showing up -and being personally served one way or another by the CHJs (denominator)- increasing more rapidly: a result likely due to some combination of the outreach pro-activity of the Project and a newly released pent-up demand for the houses’ services. c. Contribution to conflicts resolution habits. 3.10 The fact, established with high statistical significance, that the CHJs made possible for at least half the cases treated with AMCR to reach a settlement is evidence of the Project´s conflict resolution effect. In all likelihood, this also contributed to the level of credibility the CHJs gained during the years of operation. Results from the field survey of beneficiaries do not refute the numbers provided by the Executing Partners on this resolution effect indicator, which is the best defined outcome metric in the Project and perhaps its clearest measure of its development effectiveness. Statistical tests also yield no significant differences on the results due to the gender or age of respondents, or to the particular house visited. Closely associated with this resolution effect are other indirect consequences observed in the communities; which are analyzed bellow. d. Induction of gray markets for legal practices around restorative justice 3.11 As an unintended effect, the Project may have promoted opportunities for “under-the-radar” legal practices around their operation. As confirmed in interviews and focus group, lawyers have increasingly brought their private practice cases to the CHJs for resolution. Although they are regularly advised that, by procedural rules, their presence have been prohibited in the CHJs sessions, this trend evidences changes in the professionals’ view of the AMCR from a source of business competition to a welcome alternative, that can increase their effectiveness as counselors and help their practices. Largely attributable to the Project’s awareness raising effort within the profession, this trend implies a significant, albeit incipient, departure from the “appetite for litigation” reportedly fostered in lawyers by their academic training. Lawyers in poor communities mostly operate informal, low cost counseling practices; so the trend suggests that some underground economy is evolving around it, as they steer cases towards the CHJs for a fee. Now, any private charges by lawyers for helping solve cases through the houses´ mediation, conciliation and orientation in general, can still result in savings as well as in faster and more satisfying results for most people. Thus, poor plaintiffs might likely view such charges as acceptable opportunity costs, in view of the wait time and expenses they may incur in the ordinary justice processes. So, provided that -as is and has been the case consistently- the CHJs services are themselves free and seen by users as incorruptible and distinct from the more inefficient and less open services of the ordinary justice, private arrangements in poor communities between lawyers and plaintiffs can be viewed as win-win solutions, benefiting alternative justice methods while producing Pareto-optimal satisfaction for all parties involved. e. Stimulation of changes in attributes of beneficiaries 3.12 Another perhaps unintended Project result that has been observed is a trend change in the population served by the CHJs. As word of the houses spread out, and because they are de facto institutions of a special jurisdiction - assigned to a particular territory but not precluded from hearing cases from outside that demarcation- individuals have started to come from areas far from the ones “assigned”. This trend may add to the mentioned effective demand pressure on the capacity to provide services. Another trend observed concerns a more salient qualitative change in the beneficiary population. Conceptually, the CHJs focus on the most vulnerable populations and this is 26 the one regularly targeted by the Project collective activities and personalized services. Yet, focus groups have reported the processing of cases that do not necessarily concern poor women, children, senior citizens, the physically disabled and other such vulnerable groups. Real estate brokers, business people etc. also regularly bring cases for resolution through alternative justice methods. Because they are residents in poor communities, they tend to be small business owners or relatively marginal operators who appropriately belong in the CHJs target population. Yet their increasing reliance on AMCR does diversify the “market” for the houses’ services. f. Stimulation of changes in the countenance of law enforcement 3.13 Citizen security is one of the issues prominently included in the wording of the Project’s objectives, but on which actual changes could not be really measured for lack of adequate outcome metrics. Yet, in interviews, community leaders pointed to a trend toward raised awareness on issues of citizen security not only among residents but also, remarkably, among police chiefs who have been invited to partake in CHJs´ activities, such as trainings on related subject matters. In the opinion of some grassroots leaders, local law-enforcement commanders have become increasingly aware that residents can take grievances relating to security to the houses where their complaints will be heard. This, in their view, is behind some outreach initiatives taken recently by police to make officers be seen by citizens as public servants, rather than purely repressive enforcers. 3. Project influence on the target communities in other related aspects 3.14 As part of the effort to gain intelligence on project outcomes not initially included in the expected results chain, the study researched other aspects of the resolution effect concerning not the quantity of cases resolved but the quality of settlements achieved. The idea was to provide additional context and contrast to the findings on the project conflict resolution outcomes; specifically in aspects such as the nature and variety of cases; the level of user satisfaction; the degree of difficulty in settling conflicts; and the perception of justice by plaintiffs. a. User satisfaction 3.15 Because individuals may be content with the end results of mediation or conciliation but still feel uneasy about the quality of the process itself, because of treatment received, attitudes or behavior of CHJ officers, etc. -or vice versa- this evaluation made specific inquiries about the satisfaction with the services received, and found a highly positive response thereon. In fact the percentage of respondents who gave high marks to services on the 4-level semantic scale used (NONE, LOW, MEDIUM or HIGH satisfaction) was notably higher than the registered resolution rate, and there were no statistical differences in answers due to the gender or age of respondents, or to location of houses. The education of individuals was even used as independent variable and no significant differences in answers were found due to this factor either. The satisfaction appeared particularly high with the mediation services and it was not expressed just about relatively common processes or minor grievances, such as those in rent/tenant problems or overdue alimony payments, but about the whole gamut of conflicts, to include even graver cases as rape or domestic violence. It is also remarkable that the satisfaction did not appear related to the degree of success of the process. The cases brought by individuals surveyed also concerned a full variety of problems, including the more difficult to solve (i.e. those with low or null resolution rate) such as cases of division, damage or loss of property. These results suggest that beneficiaries generally tended to be satisfied with the treatment or services received, regardless of end results. This conclusion was further validated when users were asked about their sense of having received justice or not, and on what they based this sentiment. b. Perception of justice 3.16 The study recognizes that even when settlements were reached and individuals were satisfied with the process, a separate issue in their minds might be whether they felt that “justice” was rendered. In assessing this perception, 27 the study did not aim to measure people´s conceptual understanding of the difference between ordinary and alternative justice, but to assess their general sense of fairness in the solution reached. Based on a semantic scale agree-disagree that justice was rendered, with a numeric correlate for each degree (1= maximum disagreement; 5=maximum agreement) the average response was consistently above 4 in all groups surveyed, with no statistically significant differences in the responses, due to independent variables used in the study. c. Process recognition 3.17 Further analysis of the Project effect on this perception of justice focused on whether users made a logical connection between the nature of processes undertaken and the quality of results achieved. A first remarkable outcome on this respect is that people do appreciate the clarification benefits of the AMCR processes, even if end results are not completely satisfactory. As confirmed by interviews and focus groups, people do not necessarily recognize at first the nature of the problems they face, and often take them to be bigger than they truly are. So the mere discovery of, for instance, what legally constitute theft or threat often is a benefit welcome by plaintiffs who might, as a result, adjust their views and expectations concerning the steps, time and costs involved in the redressing of their grievances. This “discovery” is also appreciated by lawyers, who can better anchor further arguments about their cases on this clarification process. It also contributes to the promotion of peace in general by providing a filter to avoid the frequent escalation of conflicts into bigger and unnecessary confrontations. 3.18 The inquiry about the reasons respondents had for their perception of justice confirmed that users were consistent in the cause-effect connection they made between process and results as basis for said perception of justice. Remarkably, 50% of users expressed that “the problem was solved initially, but then persists; yet, also curiously enough, the mentioning of this circumstance provides one of the strongest arguments for the quality of observed project outcomes, given the high level of perceived justice. However unwanted, the sequel that “the problem was solved initially, but then persists” -which generally leads to resumption of mediation or re￾conduction to conciliation or to other legal courses- is not strange to the restorative justice approach, since agreements reached through mediation are by definition, voluntary, and can later get broken or unheeded by one of the parties. So, the remarkable result here is that, even when this negative circumstance arises, respondents do not hold that against the process or services rendered by the CHJs; on the contrary: the average rate given in terms of perception of justice was still consistently 5.0 in this groups. This arguably shows that CHJ users are not only consistent in their perceived process-results connection, but that they also have reached a somewhat deeper understanding of the character and restrictions of the restorative justice concepts and methods. This, in turn, attests to the effectiveness of the Project training and awareness-raising efforts. B. On the Project’s progress toward self-sustainability 3.19 As discussed in the appropriate sections, the fact that sufficient funds were raised domestically -including cash and in-kind contributions - to allow CHJs to operate without interruption during the period evaluated, is evidence of the Project immediate sustainability. Yet, the study has stressed the importance not to confuse the effort to sustain the Project -a commendable and clearly successful one so far- with the effort to achieve self-sustainability for the CHJs themselves; which is the strategic long term goal here. Now, the fact also discussed in the present report that the available data on the matter do not come from an accounting audit of funds flows and financial results statements for the houses themselves, do not permit definitive conclusions, especially in what concerns their self-sustainability over the long run, when the CHJs would have to fend for themselves without the Project or its executing partners. Also, beyond the required audit review of costs accounting and cash and fund flow records to properly establish financial self-sustainability outlook, the study has stressed the necessity to recognize that the key conditions for the CHJs´ long term financial sustainability, rest on its long term institutional sustainability; i.e. the chance that they evolve into a fully sanctioned branch of the judicial establishment, permanently required to function as instrument of public policy in the administration of justice. While social 28 ownership and private empathy toward CHJs are important for their maintenance, its long term financial viability is a function not of the fetching of funds out the willingness of CSO or private sponsors to nurture the restorative justice approach in the country, but ultimately depends on a State decision to make the CHJs part and parcel of its official structure and routine operation, for which funds must be allocated as a matter of course in the regular national budgeting process. Given this premises, the study concludes that the Project sustainability was achieved, but the institutional consolidation deliverables -a determinant of long term financial self-sustainability- were not. C. On how USAID/DR can build on the Project’s lessons 3.20 The experience with this project affords an array of lessons that can be put to use in the design and execution of other initiatives or in future stages of the present one. The suggestions to be derived from such lessons, discussed in the Recommendations chapter, can strengthen further efforts to promote restorative justice systems, both in terms of general strategic thinking forward and in terms of operational details for field execution and added effectiveness and quality/sustainability of results. These lessons, discussed below, point to ways of building upon the current experience and may be useful in shielding future ones from identifiable risk that may threaten, diminish or otherwise compromise success, but that could be managed and mitigated. 1. Role of the social penetration strategy 3.21 The strategy of “leading with community mobilization”, i.e. integrating first the grassroots organizations and natural community leaders into to project, very early in its implementation, seems to have paid off in high operational performance and quality of results, both in the currently phase and for the whole effort. When combined with the early co-opting of other CSOs and NGOs involved in social development, this lead-off game plan may largely explain how deeply the Project was able to penetrate the communities and draw the favorable disposition of residents to engage in activities that appears to be behind the constant surpassing of operational targets during execution. Also, as evidenced in the attitudinal outcomes discussed above about user satisfaction, perception of justice and appreciation of the nature of processes involved, the level of receptiveness to the Project sowed early among the targeted population appears to have played a role in the depth people exhibit in their grasp of the underlying concepts of restorative justice and alternative ways to resolve confrontations. 2. Communities best endowed for success 3.22 Before the start of the present study, USAID local authorities expressed interest in determining what type of communities were more amenable for implementation or success of this kind of project; e.g. small town communities Vs. large urban ones, etc. Now, if there are characteristics in communities to make a restorative justice experiment more or less likely to succeed in them, they appear to relate less to their size or location than to other more qualitative features. Concerning peoples’ disposition towards the Project, based on opinions heard in focus groups and interviews with grassroots leaders -as well as from national observers in a position to see the “big picture”- the contrast between the level of conflict perceived in the communities before and after the CHJs appears to be a top-of-mind issue in the stakeholders’ recollection of the experience. This suggests that, provided a lead-off social and organizational penetration strategy is adequately applied, and sufficient pro￾activity is displayed, communities with high strife levels stand a better chance to reap earlier and deeper-rooted success and resident appreciation. Arguably, this would be a function of what we have referred to as the demonstration effect of results from the houses’ activities providing a sharper contrast with the ex-ante reality in communities with previous high conflict levels than in ones with lesser conflict levels. The easier this contrast can be made visible, the stronger this demonstration effect may be achieved as a social leverage to promote success of the initiative. Now, the key in this line of argumentation is the perception of conflict rather than the reality of it. This study has found circumstantial evidence that people often perceive that the problems confronting them are graver than they really are, and that the mediation and conciliation process provide a filter to avoid 29 unnecessary escalation of conflicts. Consequently, the higher the conflicts are initially perceived, the more welcome its reduction may appear afterwards. So, the quality of results may turn out to be equivalent, no matter the communities, but they might prove to be more impressive or satisfactory, and therefore “saleable” to society at large when the effort is initially focused in localities with high perceived conflict or crime levels. 3. Determinants of long term self-sustainability 3.23 The experience proves that the pro-activity of non government stakeholders and the voluntary commitment of key public sector institutions and opinion leaders and politicians are critical for the success of a project of this kind, and can co-opt enough public support and sufficient domestic resources to sustain it financially in the short term. Yet, insofar as this commitment is the only leverage to maintain the CHJs, their long term self-sustainability is bound to remain bogged down in uncertainty. If eliciting social and private grassroots mobilization in favor of the CHJs is a proven strategy for success and a requisite for their adequate operation, the long term vocation of any restorative justice system cannot be other than to become a formal, permanent component of the official Judicial System of a country, whose sustenance is not a matter of pro bono commitment, but of a regular budget allocation to a state branch in charge of regular public functions, as any other judicial venue. Since sanctioning such formal branch falls entirely into the curfew of the State, the long term sustainability of the CHJs ultimately depends on a State decision to promulgating the requisite Law and attendant official regulations. 4. Role of a Law formally establishing a Restorative Justice System 3.24 An initial experiment with restorative justice methods extended for some time and based on the active involvement of social and private stakeholders is in all likelihood necessary, not only to accumulate valuable lessons for field operation, but also to generate enough positive demonstration effect about the CHJs and build a convincing case, to overcome attitudes that might be prevailing in opposition or resistance.7 But, in a parallel manner or once this initial thrust has been given to the transformation process, a diligent effort seems advisable to secure the above mentioned formal sanctioning of the system, if its long term sustenance is to be attained. This sustenance requisite would be especially urgent if during the experiment, as it appears to be the case with the present project, a pent-up need for restorative justice is unleashed and turned into effective demand for the related services, and that demand challenges the limits of the houses’ installed capacity and resources available to respond in an appropriate, posing the risk that their own success turn counterproductive. 5. Ways to strengthen the arguments for the CHJs 3.25 From the point of view of social desirability, the Project’s positive demonstration effect has given weight to the argument in favor of formally sanctioning a restorative justice system in the D.R. It appears, however, that an equally strong argument has not been made from the financial feasibility point of view. And another important lesson to be gleaned from the present experience is that the lingering doubts about the sustainability of such a system cannot be assuaged unless and until that argument is made forcefully and based on figures and the provable facts that: (i) funding of the system can be permanently assured as part of the annual public budget allocation exercise; (ii) funding of the system will not necessarily require additional requests for resources, but 7 It should be born in mind that the demonstration effect can be positive, but just as easily can be negative too; therefore hindering, instead of helping, the path to success. On this respect, a note of caution should come out of the Project´s failed experience with the gender violence unit in the Cienfuegos CHJ; namely that: if an initiative is launched and then it is not endowed with proper resources and mandate to accomplish the mission giving to it in theory, a grave risk is incurred of producing an expectations shock in the attendant population, with the ensuing diminished credibility that makes it harder to re-launch the initiative at other times and places. 30 mostly a resource allocation exercise, as the system would take over part of the current caseload and ease the gridlock of the ordinary justice system; and (iii) such resource redistribution exercise will prove to be cost￾effective for the entire judicial system, because the alternative justice processes are considerably less expensive in terms of monies and time required, than those of the ordinary justice system. 6. Risks of over-regulation and politicization 3.26 It appears clear that the adopting the normative to establish a formal restorative justice system must avoid two equally pernicious threats that may crop up in the process; namely: (i) an excessive legal regulation of the system that would make it to resemble so much the ordinary justice system, that people could not appreciate its relative advantages; and (ii) a possible dependence of the restorative justice system on funding decisions made by the executive branch of government. That branch should have no influence whatever on the actual operation of the CHJs, especially in what concerns funding decisions in a country where a President-centered power culture is not only prevalent by easily applied for political aims, especially at the community grassroots level. 7. Role of discovery in house prosecution services 3.27 In the experience of CHJs’ officers and users expressed in interviews and focus groups, the odds of achieving any solution agreeable to all individuals involved in a conflict highly improve when the accused party heeds the CHJs’ summon and attends a hearing before a mediator or prosecutor. Therefore, users express their wish that the CHJ had more “coercive power” to enforce such appearance and tend to regret its lack of means to do so. They also see and appreciate advantages in this concern when house prosecutors intervene, as compared with pure mediators. This opinion is expressed from a position of sympathy with, and not criticism of, the CHJs; yet do not necessarily reflect an understanding that, by the nature of the restorative justice approach, no cowing or intimidation can be exerted by the CHJs in discharging their functions. Yet, a fine line is often threaded in practice that take advantage of certain “grey area” in the cultural perception of authority to help improve the “fifty-fifty” success record of the CHJs in promoting peace (conflict resolution rate). For instance, the regular house users are community residents of limited education but, by virtue of the CHJs’ own disclosure efforts, the majority tend to understand fairly well that while house mediators are there only to promote agreements facilitating a dialogue between the parties, the house prosecutors are judicial officers with the authority to typify criminal infringements; identify and recommend further legal steps to be taken if settlements are not reached; intervene in cases of violence; and generally have more sway with people that brake agreements or do not respond to summons. For this reason, summoning acts tend to be more effective in getting people to appear before the houses if the subpoena is served by a house prosecutor, even if its purpose is only to start a mediation process. 3.28 This difference in people’s reaction between facing a house prosecutor and facing a house mediator is due in no small measure to the cultural perception that “prosecutors can have you arrested, while mediators cannot”. This perception is not quite correct, but can be put to good use in promoting peace in high-conflict communities, if it makes wrongdoers more likely to appear before the CHJs for mediation or conciliation. In the words of a judicial authority interviewed for the present study: “house prosecutors cannot really have you arrested, but people not necessarily know that, either”; so creatively managing that social nuance, on a case by case basis and provided it is done within the legitimate limits of proper procedure, can help improve the attendance of people subpoenaed by the houses and, arguably, the CHJs’ resolution rate as well. Especial care must be taken to avoid sending the message that the function of the House prosecutor is not different from that of ordinary justice prosecutors; but, reportedly, subpoenas by house prosecutors for mediation can be done without violating proper procedures, because there would be no legal impediment for a house prosecutor to refer a case to mediation, once the summoned individuals are present in the house. Other proper form of managing cultural “grey areas” in the perception of authority to favor restorative justice solutions to conflicts is the house prosecutors´ “descents” to the theater of conflicts as part of their responsibility to corroborate facts in the field. Judicial authorities 31 interviewed for this study recognize the importance, and favor, that local prosecutors discharge their discovery function through such fact-finding visits, provided they are done with no show of force or imply no intimidation whatever of the individuals involved in the conflicts. So, for instance, the use of non-descript vehicles and plain￾clothes police officers accompanying prosecutors in their “descents” are encouraged practices to convey the proper message to the population and contribute to solve confrontations within the confines of the AMCR. 8. Role of human capital formation and extra intelligence about population needs 3.29 A key restriction on the capacity of houses to respond to an increasing effective demand for services appears to be the availability of personnel adequately trained. If, as it seems to be the case with the present experience, a previously uncovered need for alternative justice services is found in the population, or the houses start to receive plaintiffs not previously targeted -like medium income people, medium-sized business owners, etc- the rapid unleash of that pent-up demand would probably tax the CHJs capacity the hardest in that area of human capital with adequate professional competencies. The mentioned possible diversification of the houses “clientele” appears to suggest the need to also expand the knowledge of the “market” over the medium term, by studying the needs of people in the municipalities as a whole, and not only those of the vulnerable population. 9. Requirements for alliances with the private sector 3.30 If the private sector tends to lose every time people resort to the ordinary justice system, businesses have a legitimate interest in promoting alternative justice venues. By the same token, initiatives like this project stand to gain from alliances with the business community. For the reasons amply discussed in the qualitative case study (Annex IV) private sector funding comports conflict of interest risks, so it should be managed with caution. The private sector might prove a powerful ally for promoting the approval of the law required to sanction a restorative justice system, as well as in co-opting other social support for the initiative. The D.R. is relatively new to the initiative, but the experience in countries like Chile or Colombia suggests ways in which private sector support to AMCR institutions can avoid or significantly mitigate the mentioned risk. For instance: if the support comes from business organizations rather than from particular businesses; or if said support indirectly benefits the effort but does not relate directly to sustaining any particular function, process or case, proper safeguards are in place to prevent conflicts of interest. Reportedly in Chile the practice to make public the information about the sponsors of academic activities, launching of books, etc. that promote restorative justice, has been adopted. In Colombia, businesses, especially those physically located within the communities where the CHJs operate, fund the training of mediators as well as community leaders and residents on the relevant subject matters. 10. Pros and cons of lawyers’ underground economy around the CHJs 3.31 As discussed in the present report the nascent grey markets for private legal practices around the operation of the CHJs, for which indirect evidence exists in the targeted communities, might be not only an inevitable development, but also a relatively welcome one as long as: (i) users see this informal practice as cost-effective alternative to the wait time and expenses associated with the ordinary justice processes, and (ii) user perceive such processes as different and apart from the CHJs services which are themselves free and incorruptible. So as long as proper measures are in place to impede the contagion of the CHJs system from the workings of the mentioned legal practice underground economy, private arrangements between lawyers and plaintiffs can represent win-win, Pareto-optimal solutions from the point of view of satisfaction of all parties involved 11. Role of pro bono legal aid for vulnerable plaintiffs 32 3.32 There are some indications that the availability of free legal aid -beyond the one provided by CHJ- like the help that term law students or interns can give may have positive effect of helping mitigate the need for lawyers’ informal practice around the CHJs process and thus prevent undue influence or interference with said processes. IV. Recommendations 4.1 Based on the lessons of the experience identified by this study, the following are recommendations for the design and execution of other like initiatives or for future stages of the present project. Recommendation # 1: Lead the implementation of CHJ projects in new locations with an early social penetration strategy specifically designed to (i) enlist the allegiance to the project of natural leaders and grassroots organizations in the targeted communities, as well as (ii) to co-opt de support of like-minded CSOs and NGOs engaged in like-minded community and social development issues and efforts. Recommendation #2: In choosing areas for new CHJ projects in the future, give priority to communities with high conflict levels, for maximum possible demonstration effect over the short and medium term. Recommendation #3: In parallel to, or shortly after, an initial phase of experiment to accumulate valuable lessons for effectiveness in the field, and to build up a convincing case for a restorative justice system, give priority to the formal sanctioning of such system through the requisite law and regulations. To the extent possible, avoid negative demonstration effects during the experimental phase due to lack of adequate endowment of resources and mandate, that might diminish the initiative´s credibility as in the failed experience with the Cienfuegos CHJ´s violence unit. Recommendation #4: In parallel to an initial phase of experimentation to build up a convincing case for the social desirability of a restorative justice system, or shortly thereafter, make a quantitative study on the feasibility of funding the system based on a cost-effective redirecting of budget funds traditionally allocated to ordinary justice venues. Recommendation #5: In developing a draft Law and its By-laws to sanction a restorative justice system avoid: (i) excessive regulations that might make its procedures resemble too much those of the ordinary justice system, and (ii) any possible dependence of the system on funding decisions made by the executive branch of government Recommendation #6: Whenever possible, provide the CHJs projects with resources to finance vehicles for the exclusively official use of house prosecutors to discharge their discovery function through fact-finding visits to the theater of conflicts, done with no show of force or intimidation of the individuals involved Recommendation #7: Give priority to the formation of human capital with the adequate professional competencies required to properly attend and process the demand for alternate justice services, and expand the knowledge of populations beyond the project target communities that might get enticed to demand those services in the future. Recommendation #8: Support alliances with, and co-opt the support of, the private sector for promoting a restorative justice systems, provided that the support comes from business organizations rather than from particular businesses; and safeguards are in place to avoid conflict of interest, such as that said support indirectly benefits the project but does sustaining any particular CHJ hearing process Recommendation #9 33 Make sure that proper measures are in place to impede the contagion of the CHJs system by the workings of any underground economy that might crop up in lawyer’s private practices around the operation of CHJs, including the addition of legal orientation capabilities, through pro bono professional work or voluntary help from term law students or interns. Technical references A. Weis, C.H. (1998) “Evaluation: Methods for studying programs and policies.” Prentice Hall, NJ. B. Campbell, D.T, & Stanley, J.C. (1963). Experimental and Quasi-experimental Design for Research. Chicago, Rand McNally. C. Cohen, J. (1992). "A power primer". Psychological Bulletin. 112 (1): 155–159. doi:10.1037/0033- 2909.112.1.155. PMID 19565683). D. Faul, F. (2012) GPower Version 3.1.4, http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register E.Preacher, K. J. (2001, April). Calculation for the chi-square test: An interactive calculation tool for chi￾square tests of goodness of fit and independence [Computer software]. Available from http://quantpsy.org.%