22 January 2014 This publication was produced for review by the United States Agency for International Development. It was prepared by Development & Training Services, Inc. (dTS). FINAL EVALUATION PROJUSTICE PROGRAM IN THE DEMOCRATIC REPUBLIC OF THE CONGO Prepared for the United States Agency for International Development, USAID Contract Number AID-RAN-I-00- 09-00015, Task Order Number AID-660-TO-13-00001, Evaluation of the ProJustice Program in the DRC: Evaluation Report Implemented by: Development & Training Services, Inc. (dTS) 4600 North Fairfax Drive, Suite 402 Arlington, VA 22203 Phone: +1 703-465-9388 Fax: +1 703-465-9344 www.onlinedts.com FINAL EVALUATION PROJUSTICE PROGRAM IN THE DEMOCRATIC REPUBLIC OF THE CONGO 22 January 2014 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect those of the United States Agency for International Development or the United States Government. ACKNOWLEDGMENTS The dTS team benefitted from the support and contributions of many individuals in the form of time, effort, and logistical support. We thank all the ProJustice staff in the DRC who contributed to organizing travel, interviews, and lodging during the field visits to various program sites, and above all for answering questions about the program’s components. We also thank USAID staff in the DRC for their responsiveness and willingness to meet with the evaluation team and discuss the program and their expectations. Finally, the team thanks the staff of the courts, the community members, and other development partners who showed interest in and dedicated their time to discussions with the evaluators. ` Final Evaluation: ProJustice Program in the DRC i CONTENTS ACRONYMS ................................................................................................................................................ ii EXECUTIVE SUMMARY ........................................................................................................................... iii 1. INTRODUCTION ............................................................................................................................... 1 2. EVALUATION PURPOSE AND METHODOLOGY ........................................................................ 2 3. FINDINGS ............................................................................................................................................ 9 4. DISCUSSION AND CONCLUSIONS ............................................................................................. 31 5. RECOMMENDATIONS .................................................................................................................... 46 APPENDICES APPENDIX 1: COURT USER SURVEY BY PROVINCE .............................................................................. A-1 APPENDIX 2: HOUSEHOLD SURVEY—DEMOGRAPHIC DATA ................................................................ A-2 APPENDIX 3: UPDATED PROJUSTICE M&E PLAN AT THE TIME OF THIS FINAL EVALUATION .......................... A-8 APPENDIX 4: EVALUATION OF THE SCOPE OF WORK ....................................................................... A-21 APPENDIX 5: FINAL EVALUATION PLAN ........................................................................................ A-25 APPENDIX 6: EVALUATION TOOLS .............................................................................................. A-26 APPENDIX 7: HOUSEHOLD QUESTIONNAIRE................................................................................... A-32 APPENDIX 8: INFORMATION SOURCES AND CONTACTS .................................................................... A-35 ` Final Evaluation: ProJustice Program in the DRC ii ACRONYMS CBO Community-based organization CoP Chief of party CSO Civil society organization CSM Conseil Supérieur de la Magistrature DG Democracy and governance DRC The Democratic Republic of the Congo dTS Development and Training Services EFPRJ School of Training for Judiciary Personnel EU European Union FGD Focus group discussion GBV Gender-based violence GoDRC Government of the DRC JRP Justice Reform Plan M&E Monitoring and evaluation MoJ Ministry of Justice MONUSCO United Nations Organization Stabilization Mission in the Democratic Republic of the Congo MoUs Memoranda of Understanding NGO Nongovernmental organization PARJ Programme d’Appui à la Reforme de la Justice PACT Program Action Plans Trademark PMP Performance Management Plan PMT Project Management Team RoL Rule of law TGI Tribunal de Grande Instance UN The United Nations UNDP United Nations Development Program UNJHRO The United Nations Joint Human Rights Office USAID United States Agency for International Development ` Final Evaluation: ProJustice Program in the DRC iii EXECUTIVE SUMMARY The ProJustice Program was funded by the United States Agency for International Development (USAID) to address challenges faced by the Congolese judiciary while supporting reforms initiated by the Government of the Democratic Republic of the Congo (GoDRC), as set forth in the 2006 constitution. Tetra Tech DPK implemented the ProJustice Program over the course of five years (September 2008–September 2013) with a budget of US $20,484,410. The main objective of the program was to develop an accessible and empowered justice system through improved judicial management, lower costs of access to justice, and a more informed and engaged public. ProJustice was designed around four main components, each with a corresponding core objective: • Supporting the establishment of new judicial institutions • More effective and transparent management of the judiciary and Ministry of Justice (MoJ) • More effective, transparent, and accessible court operations in pilot jurisdictions • Increasing access to justice for vulnerable populations. Development and Training Services (dTS) was contracted to conduct the final performance evaluation of the USAID/DRC-funded ProJustice Program and publish its findings in a report. Field research and data collection for the evaluation was conducted from 15 August to 25 September 2013. This final performance evaluation helps to assess the validity of the program design and assumptions as well as the overall performance of the ProJustice Program, and to gauge the present landscape of the judiciary in the DRC. The specific objectives of this evaluation exercise are to: • Make an overall independent assessment of the past performance of the ProJustice Program, with particular attention to the impact of program actions against its pre-established goals and objectives. • Identify key lessons learned and propose practical recommendations for the program and USAID/DRC for future rule of law (RoL) strategies. EVALUATION METHODOLOGY The evaluation methodology used both qualitative and quantitative data collection and analysis techniques. A stratified sampling strategy was used to select judiciary courts for examination. The evaluation team collected data in three out of four provinces of the DRC where the program was implemented: South Kivu, Bandundu, and Katanga. Data were collected in both urban and rural communities served by 10 courts (eight treatment and two control).1 Two quantitative data collection tools were used: the household survey and the court user survey. Both tools provided the data with which this evaluation report examines community-level satisfaction and confidence in the justice sector in the DRC. 1 Because of logistical issues, no control court was visited in Bandundu. The sample size was later revised to overcome that issue. ` Final Evaluation: ProJustice Program in the DRC iv Key informant interviews and focus group discussions were used to collect qualitative data in the communities that the team visited. The evaluation team used the combined data collected from key stakeholders, community members, and court users to assess the program’s performance. FINDINGS AND CONCLUSIONS The approach used by the program on both the supply and the demand sides of the justice system appears to have had a positive effect on the Congolese judicial institutions. Following an organizational audit, ProJustice proposed that the MoJ be reorganized and RoL be administered in the DRC. To execute this restructuring, several MoJ leaders were trained and coached. ProJustice worked to provide better transparency in the Congolese judiciary by allowing the Conseil Superieur de la Magistrature (CSM) to have more authority on decisions that rule the Congolese magistracy. The program supported the CSM in the revitalization of the disciplinary chambers in all four target provinces. Unlike the situation that prevailed prior to the ProJustice Program, CSM was provided with a functional office outside the MoJ and has now begun to be the recognized voice of the Congolese judiciary. Capacity building and support to the CSM were critical to improving management of the judiciary. With support from ProJustice, CSM led the revitalization of the disciplinary chambers, a process that has increased judicial transparency and the confidence of Congolese citizens in RoL. However, although the support was lauded, several informants expressed concern about the ability of the CSM to continue to support the chambers without external funding. With the exception of activities conducted under a fifth component, “Windows of Opportunity,” all four program components have been deemed appropriate, based upon data emerging from the evaluation. Several adjustments have been proposed to better support the justice sector and provide target communities with access to fair and transparent justice. The evaluation exercise and data confirm that the approach taken by the program was relevant and appropriate, addressing the needs and critical issues that the program set out to achieve. Although there were opportunities for improvement, overall the ProJustice project was successful in meeting its objectives. Court administration and management. Respondents in the pilot courts noted (1) improvement in the filing system, (2) increased capacity as a result of trainings, and (3) logistical support received as positive outcomes of the program. At the end of the program, transparency and accessibility of court operations remain a challenge. In addition, according to the methodology adopted with the Program Action Plans Trademark (PACT), pilot courts should clearly display their fees so that information is accessible to all users. Unfortunately, the majority of courts visited during the evaluation were not in compliance with this policy. Gender and vulnerable people in the ProJustice Program. After the government appointed 660 female magistrates during the 2011 recruitment period, the program worked with the CSM to have senior female magistrates promoted to higher positions within the system. In addition, the program worked with a consortium of actors to facilitate access to justice for vulnerable community members. While the partnership with civil society organizations proved effective in reaching the most people possible, the criteria used to detect these vulnerable people were not robust enough to detect all vulnerable segments of the target communities. Program staffing and implementation. The staffing of the program at the central level (Kinshasa) was determined to be appropriate; however, the staffing approach used one or two people to conduct ` Final Evaluation: ProJustice Program in the DRC v all the planning and monitoring of activities per province. It was difficult to maintain a certain level of rigor and pace given the size of the provinces and the number of staff to lead implementation. Most of the planned activities were achieved except for the support given to the constitutional courts. The program has managed to implement all planned activities under its first four components. However, component 5 (Windows of Opportunity) was not implemented as conceived due to a combination of factors including a lack of pre-defined objectives and limited understanding on the part of legal and MOJ staff of the process for accessing funds under this program component. The results during the five-year period of implementation were efficient as the wide majority of activities were achieved. Also (as discussed in the recommendations section in detail), some changes in the design of the program could improve its multiplier effect and further facilitate its sustainability. Despite all the efforts that have been made by this pilot program, there is still a need for continued RoL programming in the DRC. The programming should seek to continue to institutionalize and expand the reach of justice in the DRC while continuing to increase access to justice. As a pilot program, ProJustice could not meet all needs; hence, it is clear that the program approach is still relevant as a means of continuing to support the institutionalization of RoL in the DRC. RECOMMENDATIONS The evaluation team proffers several recommendations that reflect the most important gaps and opportunities that would strengthen existing and future programming efforts in the DRC. • Future RoL programs seek pre-, peri-, and post-government budgetary commitments, in order to ensure proper resourcing of the justice sector and sustainability of program outcomes. On the basis of the large portion of justice sector informants, the justice sector is not adequately funded and most courts do not receive budgetary support. • Future RoL programs should continue to support the constitutional court and the other newly established judicial institutions as they will play a critical role in the judicial landscape. RoL programs should continue to provide capacity building and both technical and budgetary support to these emerging pillars of Congolese justice, to help institutionalize and formalize their functions. • Future RoL programs should extend work with local organizations to strengthen networks of legal defenders, who can provide legal support at lower cost to members of their communities. • The filing system promoted by the program in the pilot courts should be replicated in as many courts as possible. While the system promoted by the program is not fully operational, it has already started to yield results by increasing the efficiency of the staff in pilot courts. • Future RoL programs should improve program design to ensure that performance-based milestones are clearly identified and reinforced, and to incentivize stakeholders’ adherence to future programs. The lack of effective disincentives or any significant consequence for poor performance fostered an environment of relative mediocrity within pilot jurisdictions of the ProJustice Program. The PACT process failed to encourage counterparts to implement all aspects of the program. • The mixed justice committees should be revitalized for better collaboration between RoL actors in the DRC. The mixed justice committees allow better coordination of activities in the ` Final Evaluation: ProJustice Program in the DRC vi country and limited inefficiency. It is important to maintain coordination with other actors to avoid duplication of effort and mismanagement of resources for future RoL programs. ` Final Evaluation: ProJustice Program in the DRC 1 1. INTRODUCTION ProJustice was a three-year program with two 1-year extensions, funded by the United States Agency for International Development (USAID) and implemented by Tetra Tech DPK from 2008 to 2013 in the Democratic Republic of the Congo (DRC). Its goal was to promote and strengthen justice in the DRC ProJustice selected Katanga, Bandundu, Maniema, and South Kivu as target sites. The program was based in Kinshasa, where broader policy issues would be raised, particularly around providing resources for and independence of the judicial sector. The program was meant to support the Congolese judiciary and facilitate access to a transparent and effective justice system for all Congolese citizens. ProJustice was developed within the context of the DRC Justice Reform Plan (JRP) and the USAID Democracy and Governance (DG) assessment conducted in 2008. The JRP content came about as a by￾product from popular consultations conducted around the “Strategic Document for Growth and Poverty Reduction” in the early 2000s and from various judicial sector audits carried out in 2003–2004. The JRP defined various priorities, some of which the ProJustice Program eventually targeted: • Need to improve the population’s access to justice • Need to offer the justice system better infrastructure and equipment • Urgent need to eliminate corruption and impunity within the judicial profession. The plan also specified very important conditions to achieve the identified priorities, including the following: (1) buy-in from all justice sector stakeholders; (2) financial commitment from the Government of the DRC (GoDRC) to support reform and increase resources to the judicial sector; (3) peace and stability with neighboring countries and domestically; (4) disciplinary actions taken, when needed, against unethical behaviors; (5) a growth in trust toward the judicial sector by the population; and (6) long-term support on logistical and technical fronts from partners (donor community). In each province, the program concentrated on the capital city but also implemented activities on the outskirts of cities and in surrounding rural areas. While in Bandundu, the capital city was targeted together with Bagata city, a rural site. The capital cities were also used as assembly points when trainings were organized for significant numbers of personnel in the provinces. The program description of ProJustice started with a preamble that emphasized the relative absence of the justice system across most of the country. The program also stressed the fact that vulnerable populations were the most affected by weaknesses in the judicial system, alluding particularly to children and to women who were victims of sexual violence, gender-based violence (GBV). The program was thereafter developed with four main components representing core objectives: 1. Supporting the establishment of new judicial institutions 2. Triggering the existence of more effective and transparent management of the judiciary system and the Ministry of Justice (MoJ) 3. Catalyzing more effective, transparent, and accessible court operations in pilot jurisdictions 4. Working to increase accessibility to justice for vulnerable people. ` Final Evaluation: ProJustice Program in the DRC 2 2. EVALUATION PURPOSE AND METHODOLOGY Development and Training Services (dTS) conducted an independent final performance evaluation of the USAID/DRC-funded ProJustice Program. Field research and data collection for the final evaluation was conducted in the DRC from 15 August to 25 September 2013. The results of the evaluation will enable the Mission to assess the validity of the program’s design assumptions as well as its overall performance, and to gauge the present landscape of the judiciary in the DRC. The evaluation findings aim to guide USAID’s strategy and decision-making regarding future rule of law (RoL) programming. The specific objectives of the assignment were to: • Make an overall independent assessment about the past performance of the ProJustice Program, paying particular attention to the impact of program actions against its pre-established goals and objectives. • Identify key lessons learned and offer practical recommendations for the program and USAID/DRC for future RoL strategies. In keeping with best practices, the evaluation team prioritized best practices and impact assessment methodologies as a measure of ProJustice performance. Output measurements are included to provide evidence for gauging the overall relationship between program assumptions, program activities, and program sustainability. Challenges and limitations are presented in addition to findings to justify the evaluation approach as the best possible methodology, given the reality on the ground. The intended audiences of this evaluation are the USAID/DRC Mission, the GoDRC and its affiliated bodies, the Conseil Superieur de la Magistrature (CSM), Tetra Tech DPK, and any development actor who may be interested in RoL programming in the DRC. Six evaluation criteria were set forth in the scope of work for this evaluation. For each criterion, a list of associated key questions is presented below: 1. Program Approach • Is the program approach still relevant to developing, reinforcing, and empowering Congolese judicial institutions? • Are the program’s results applicable in the current context of DG and justice sector reform needs and objectives? 2. Technical Effectiveness • What is the effectiveness of each of the program-expected results? • Did activities achieve the programs’ expected goals and results? • What factors have contributed to or militated against ProJustice performance? ` Final Evaluation: ProJustice Program in the DRC 3 • Do community members in supported courts see ProJustice activities as effective? • How have men and women benefited from ProJustice, and how has this program impacted them differently? 3. Program Management • Was the program designed and staffed to work collaboratively and effectively across all component areas? • Are the monitoring, evaluation, and reporting systems adequate, especially protocols for collecting and utilizing data on court management processes? • Are there programmatic or organizational gaps that hinder the achievement of results? • Was information effectively captured and used internally and externally? • How effective was ProJustice in engaging, coordinating, and/or supporting legal institutions, government entities, civil society, the larger donor community, and other stakeholders? 4. Program Impact • What (if any) has been the substantive impact in the program areas attributable to program activities? • How (if at all) are targeted civil society organizations (CSOs) better able to effectively advocate for reforms, monitor government activities, and provide critical services to their constituents? • How (if at all) are CSOs, community-based organizations (CBOs), local peace structures, and the GoDRC better able to improve court management processes and respond to the needs of their citizens in seeking justice? • How (if at all) has access to justice changed for vulnerable populations, especially women? 5. Sustainability • What are the elements of the ProJustice Program that will remain after its completion? 6. Lessons Learned and Recommendations • What are the lessons learned from program design, implementation, and monitoring? • What could ProJustice have done better to ensure that results were achieved and sustained? • Are there program successes that deserve replication? • What measures can future programs take to increase access to justice for vulnerable populations? 2.1 SAMPLING The dTS evaluation team visited nine (out of 12) treatment courts and two (out of four) control courts in three (out of four) pilot provinces and included community survey work in eight distinct localities. To complement field research and data collection on the ProJustice program implementation in the ` Final Evaluation: ProJustice Program in the DRC 4 provinces, the evaluation team also held key informant interviews with magistrates and judicial personnel at the CSM in Kinshasa. The evaluation sampling criteria were based on accessibility, evaluation relevance, and standard best practices. The evaluation endeavored to cover all ProJustice Program provinces and courthouses. Adjustments were made early in the evaluation sampling process that reflected the unique conditions for collecting data in the DRC and the specific program interventions by ProJustice: • The evaluation team expanded the data collection efforts in South Kivu Province to reflect the additional program focus and funding by ProJustice to courts in the east of the country. • Owing to logistical challenges, the evaluation team was unable to visit courts in Maniema Province. Two additional survey communities in South Kivu were added to maintain adequate observation numbers in the household survey. • The evaluators determined the quality of the comparison courts to be insufficient for a statistically rigorous quantitative control at the court level; however, the adjoining communities demonstrated demographic characteristics suitable for comparison purposes at the household level. • The substantial differences between treatment and control courts resulted in focused qualitative methods being administered specifically to court personnel. Quantitative data focused on broad community-based surveys in corresponding treatment or control localities. • For anecdotal purposes, short surveys of court users were conducted at all courts and provide valuable insight into court performance operations. • Both qualitative and quantitative data collection strategies provided the necessary flexibility to maintain an adequate sample size for the purpose of this evaluation. It is important to note that the evaluation methodology follows a stratified approach, which allows for variation in the sampling between different evaluation tools. The internal variation between treatment and control courts rendered quantitative methodology unreliable in the surveys specific to court users. The qualitative approach, however, provided useful insight into the general atmosphere of the court experience and the specific outcomes for court personnel within the ProJustice Program. The carefully selected localities in the quantitative household survey sample reflected individual demographics suitable for comparison to external factors, such as the program intervention. The sampling limitations associated with the court-user surveys—specifically, the poor selection of treatment and control courts—had virtually no impact on the statistical quality of the data associated with the sampling methods used in the household surveys. The sole sampling limitation shared between the two different quantitative tools is attributed to exogenous differences associated with rural and urban communities. The evaluation team found these differences to be manageable within each specific province. The assumption was made that communities within the surrounding areas of treatment courts would reflect program impacts at a greater level than those communities outside the jurisdiction of ProJustice courts. Where the outcomes of household survey questions capture the program intervention activities, this evaluation attributes some of this impact to the ProJustice Program. ` Final Evaluation: ProJustice Program in the DRC 5 2.2 DEFINITIONS AND DESCRIPTIONS To maintain continuity, the evaluation strategy mirrored the program definitions used by ProJustice: • Treatment courts were those that benefited from the ProJustice intervention. The communities served by these courts were expected to have better access to the judicial system in comparison to those where control courts were located. • Control courts were selected to be as similar to the ProJustice treatment courts as possible, with the main objective difference being program intervention. In general, control courts were geographically close to the treatment courts to ensure that they shared similar geographic characteristics and socio-political situations. In practice, however, the control courts failed to provide suitably systematic controls for a statistically rigorous evaluation. 2.3 LIMITATIONS AND CHALLENGES Several limitations and challenges emerged over the course of the evaluation. Every effort was made to maintain evaluation best practices in identifying solutions that met time, budgetary, and logistical constraints. • ProJustice control courts differed significantly from treatment courts in a variety of measures outside of the program intervention. Field observations determined that these courts lacked the most basic of resources and provided services to communities facing significantly more legal disadvantages due to the rural nature of their locations. All but one of the treatment courts, Bagata, had the benefit of being located in central urban districts. The control court in Kasumbalesa did not have a courtroom for magistrates to hear cases (e.g., trials were held under a mango tree behind the courthouse building). As a result of these disparities, the evaluation team focused its court evaluation effort on qualitative tools designed to address program assumptions and performance with the court personnel. This approach allowed the evaluation to identify specific activities, regulations, and events that benefited the treatment courts while identifying areas of greatest need for future programming. • Courts serving people from the main city of the province were classified as urban courts; those primarily serving people from outside the main city of the province were classified as rural courts. These rural and urban courts were selected based on the list of courts supported by ProJustice. Note that in addition to the previously described issues in control court selection, the location of the court played a substantive role in describing court features. ProJustice primarily served courts in urban centers, where courts were significantly better resourced and staffed due to their proximity to power centers in each province. Despite this, the comparison of treatment and control communities provides valuable insight, given the comparable composition of community members and the exogenous nature of the judicial disparities this evaluation seeks to identify. This evaluation assumes that communities that benefited from greater proximity to treatment courts would have benefited from “spill-over” effects made to ProJustice improvements in the judiciary. • The judicial vacation period overlapped with the data collection period, resulting in a complete shutdown of the judiciary in certain areas and a reduced number of cases being heard at the majority of courthouses surveyed. This did not impact the qualitative evaluation in any meaningful way, as the requisite number and level of court personnel were ultimately made available. However, this reduced the effectiveness of the quantitative court user satisfaction survey. ` Final Evaluation: ProJustice Program in the DRC 6 • Courthouses had much lower caseload activity than originally suggested by Tetra Tech DPK. During the first week of field activity, the ProJustice staff provided the evaluation team with estimates of the total number of daily cases processed by the courts. In actuality, all courts exhibited substantially lower turnover rates. This is partially explained by the judicial holiday; however, interviews conducted with court staff suggested the estimates were high for “normal” judicial activity days. Control courts exhibited such low levels of activity that only one of the two control courts visited had any court activity to assess. This evaluation notes that any differences between the treatment and control courts are unreliable in determining the likelihood that the findings can be directly attributed to the ProJustice Program alone. But this does not undervalue the importance of the various data collection tools as an effective means of describing the judicial picture on the ground. The qualitative tools provide data to discuss specific programmatic outcomes, whereas the focus of quantitative tools at the community level in the form of household surveys allows for extrapolation of ProJustice results through comparative analysis related to tribunal proximity. 2.4 QUANTITATIVE TOOLS Court User Survey. This evaluation aimed to capture the public court experience through court user surveys. Individual court users were approached as they departed the courthouse and asked to share their experience that day by completing a short survey. The survey was designed to address five key areas for evaluation: (1) access to the judicial system, (2) public legal outreach activities, (3) effectiveness and transparency of court management activities, (4) effectiveness and transparency of court personnel, and (5) overall court user opinion of court performance. In total, 157 individual court users were surveyed at seven treatment courts in all three provinces, and one control court in South Kivu, during the evaluation period. Owing to the reduced number of cases being heard by the courts during the evaluation period, the field team was unable to collect the targeted number of court user surveys. The court user survey, for this reason, is not used as a measure of direct beneficiary comparison of the control and treatment courts activities. Instead, the court user survey provides demographic information on the types of individuals who frequented the courts during the evaluation period, the purpose of their court visit, and their specific experience during their visit. This information, in tandem with the complete set of evaluation tools, contributed to providing a thorough picture of the judiciary during the evaluation period. The demographic data (Table 1, standard deviations in parenthesis) in the court user survey provide a snapshot of the gender and age of court users surveyed. TABLE 1. COURT USER SURVEY DEMOGRAPHICS BY PROVINCE Treatment Courts Control Courts Bandundu Katanga South Kivu South Kivu Average Age 34.7 44.0 37.0 44.3 (10.89) (17.81) (11.69) (12.57) Court Users by Gender (%) Male 100 67.9 67.6 46.2 Female 0 32.1 32.4 53.8 Total Observations 20 56 68 13 ` Final Evaluation: ProJustice Program in the DRC 7 Community/Household Survey. A household survey was conducted in communities served by both treatment and control courts to determine the impact of the ProJustice Program and gauge the public’s overall experience and perception of the Congolese judiciary. Data collectors from local university law faculties were hired to ensure that cultural and language sensitivities were adequately addressed in each provincial community. Data collectors received introductory training on survey methodology and in field instruction. Survey teams were deployed to high-traffic market areas in all three provinces to collect randomly selected individuals for the household surveys, 25 August to 18 September 2013 (Table 2). TABLE 2. HOUSEHOLD SURVEY SCHEDULE Province Treatment Community Dates Control Community/Observations Dates Bandundu Bandundu-ville 29 Aug. Owing to unexpected limitations in Bandundu, no control community was surveyed. Bagata 30 Aug. Katanga Lubumbashi-ville 4–6 Sept. Kasumbalesa 5 Sept. South Kivu Bukavu 18 Sept. Kavumu 16 Sept. Kadutu 17 Sept. Mudaka 15 Sept. In total, 620 individual community members in seven communities across three provinces participated in the study. Despite limitations associated with rural versus urban localities,2 a diversified sampling in sex, age, and household standing within the local population provided for a randomized and representative survey sample usable for comparative purposes. Adjustments were made accordingly during the survey to ensure a broad and random sample of household respondents, particularly as it pertained to gender and age. Table 3 summarizes the household demographic data for this evaluation. TABLE 3. HOUSEHOLD SURVEY DEMOGRAPHIC DATA Variable Total Treatment Communities Control Communities Household Standing (%) Head of Household 47.1 44.7 53.1 Spouse 36.0 35.2 38.0 Other 16.9 20.2 8.9 Sex (%) Male 53.2 53.7 52.0 Female 46.8 46.3 48.0 Average Age (standard deviations for gender in parenthesis) Total Sample 38.27 37.94 39.11 (12.67) (12.49) (13.10) Male 38.73 38.11 40.33 (13.04) (12.55) (14.17) 2 Location, rural or urban, was not a significant variable in determining significance for comparison of communities, particularly when analysis was done at the provincial level between treatment and control communities. This can largely be explained by the overall low levels of development across DRC, which affects analysis at provincial level differences in the study. The market population in Kasumbalesa did not demonstrate substantially different characteristics than those surveyed in Lubumbashi. Indeed, Kasumbalesa had a healthy level of trade and commerce due to its location in Katanga province at the border with Zambia. Similarly in Bukavu Province, the control communities of Kavumu and Mudaka demonstrated comparable levels of mobility and relative commerce compared with the treatment communities in the city center at Bukavu and Kadutu. All communities within the provincial level experienced suitable characteristics for comparison, regardless of any court presence in the respective area, allowing for comparative use. ` Final Evaluation: ProJustice Program in the DRC 8 TABLE 3. HOUSEHOLD SURVEY DEMOGRAPHIC DATA Variable Total Treatment Communities Control Communities Female 37.74 37.75 37.73 (12.22) (12.46) (11.68) Literacy Rates (%) Total Sample 85.5 90.7 72.6 Male 91.5 94.1 85.0 Female 78.3 86.3 59.3 Level of Education (%) Basic literacy 6.0 5.4 7.3 Primary 11.0 6.6 21.8 Secondary 47.3 51.3 37.4 University 23.2 27.9 11.7 No formal education 12.6 8.8 21.8 Principal Employment (%) Farmer 7.9 5.7 13.4 Shopkeeper 47.4 48.8 44.1 Civil servant 18.4 19.5 15.6 Private sector 13.6 13.8 12.9 Other 10.7 10.4 11.2 Unemployed 2.1 1.8 2.8 Total No. of Observations by Province Bandundu 118 118 0 Katanga 221 171 50 South Kivu 281 152 129 2.5 QUALITATIVE TOOLS Key Informant Interviews. Key informant interviews were conducted from 15 August to 25 September 2013, in all program areas (Table 4). Individuals were selected based on their relevance to the ProJustice activities in both treatment and control areas and availability during the data collection period in each province. Interviews included ProJustice staff, MoJ officials, representatives of the CSM, representatives of PARJ (Programme d’Appui a la Reforme de la Justice), funded by the European Union (EU) presidents of the tribunals, lawyers, and CSO representatives. Fifty individuals were interviewed on their experience with the ProJustice Program. TABLE 4. KEY INFORMANTS INTERVIEWS PER PROVINCE Province Total Number of Interviews Kinshasa 11 Bandundu 14 Katanga 16 South Kivu 9 ` Final Evaluation: ProJustice Program in the DRC 9 Focus Group Discussions. Focus group discussions (FGD) were held with eight ProJustice beneficiary groups in all three evaluation provinces (29 Aug.–19 Sept. 2013). The groups represented a variety of program beneficiaries and included individual lawyers, one Bar association, CSO representatives, and individual community members associated with ProJustice-supported CSOs. In instances of vulnerable populations, the evaluators endeavored to organize the focus groups specifically by gender. Table 5 provides a breakdown of the FGD data collection activities. TABLE 5. FGD SCHEDULE Province Date Organization Comments Bandundu 29 Aug. Ceprosoc CSO, mixed group, including vulnerable populations 29 Aug. ASEARO (Radio Okapi Listeners’ club in Bandundu CSO, male only Katanga 6 Sept. Dynamique des Femmes pour le Développement du Congo CSO, mixed gender focus group 7 Sept. Lubumbashi Bar Association Mostly male lawyers South Kivu 18 Sept. OCET Bukavu CSO, female only, vulnerable populations 18 Sept. OCET Bukavu CSO, male only 19 Sept. NAPO CSO, female only, vulnerable populations 19 Sept. NAPO CSO, male only 3. FINDINGS This section presents the evaluation findings for each of the main evaluation questions. Where necessary, subsections have been introduced to clarify the findings. 3.1 IS THE PROGRAM APPROACH STILL RELEVANT TO DEVELOPING, REINFORCING, AND EMPOWERING CONGOLESE JUDICIAL INSTITUTIONS? The program approach in treatment courts was designed to address both the supply- and demand-side aspects of the judicial system. Supply-side factors are related to competencies, capacity building, and availability of human resources within the judicial sector; working conditions of magistrates and MoJ staff; and the logistics of the sector. The logistics would cover elements such as office supplies for courthouses as well as transport and travel of judges and court clerks while performing their duties. Demand-side factors are related to issues affecting court and tribunal users (i.e., those that would benefit from judicial services). It includes things such as availability of information to court users, affordability and transparency of costs related to judicial proceedings, and the existence of legal assistance to vulnerable people. ` Final Evaluation: ProJustice Program in the DRC 10 3.1.1 ProJustice work on the supply side Several magistrates interviewed during this evaluation have recognized the benefits and relevance of the much needed trainings received through the program.3 The program compiled, printed, and distributed to magistrates and court clerks, copies of key legal documents, such as the Congolese penal code and legal procedures. Magistrates and non-magistrate staff were introduced to various training modules that were designed to improve the quality of hearings, court proceedings, archiving, and time spent on cases. One key informant in Bandundu mentioned that as a result of the trainings, judges from the Tripaix stopped ruling on cases related to lease conflicts that are, by law, the prerogative of the Tribunal de Grande Instance (TGI). He also stated that ProJustice arranged for them to meet with judges in Bandundu to discuss their work and improve their understanding of the current laws of the DRC.4 The prosecutor of the Bandundu TGI said that the support received from ProJustice made it possible for the tribunal to effectively process over 100 cases in a single quarter for the first time in its recent history. He also noted that the last time such a performance occurred was well before Mobutu’s regime started collapsing. Apart from the trainings and reading material mentioned above, ProJustice provided logistical support to the TGI of Bandundu, Katanga, and Bukavu. This logistical support made it possible to hold mobile courts that helped process cases delayed due to the absence of judges in specific remote sites, where people were kept in prison cells. According to the lead and deputy prosecutors of the Bandundu TGI, this logistical support was critical in avoiding the prolonged detention of prisoners.5 Visits to prisons supported by ProJustice facilitated the review of such cases and prisoner release granted as prescribed by law.6 The court clerk from Bagata Tripaix in Bandundu Province was provided with a motorbike so that he could travel to remote police stations, distribute court decisions, and reach people with cases pending at the court. Similarly, the court clerk of the Bukavu TGI stated that the transportation of magistrates to prisons, in partnership with the United Nations Organization Stabilization Mission in the Democratic Republic of the Congo (MONUSCO), helped to expedite the handling of cases, especially when judges could be flown to prisons where no magistrate were available. The ProJustice program strategy aimed to go beyond capacity building and increase the financial resources available to the judicial sector. The president of the appellate court in Bandundu and the training officer of the Permanent Secretariat of the CSM declared that, following support received from the program, they participated in trainings on budget preparations and eventually worked on budget submissions to Parliament. Research on the ground brought to light testimonies from magistrates stating that funds is not reaching courts and tribunals. As an illustration, one magistrate in charge of training at the CSM Permanent Secretariat reported that he had not received funding to support training activities in three years.7 During the interview, he received calls from magistrates struggling to get their air tickets to reach the 3 Among the magistrates that emphasized the relevance of the trainings we have the President of the appellate court in Bandundu interviewed on 28 August in Bandundu-ville, the lead and deputy prosecutor of Bandundu TGI to whom the evaluation team spoke separately on 29 August, the CSM magistrate in charge of trainings at the Permanent Secretariat met on 5 September, and the President of the appellate court of Lubumbashi. 4 Information came from the appellate court president in Bandundu-ville whom the evaluation team met on 28 August. 5 Lead and deputy prosecutors of Bandundu TGI were interviewed on 29 August. 6 Ibid. 7 He said that he would be ready to defend his position publicly if necessary. He was very vocal on this topic—he spent almost 15 minutes during the interview on it. He repeatedly emphasized the lack of support from the State. ` Final Evaluation: ProJustice Program in the DRC 11 capital city of Kinshasa, where trainings were planned in the following week. In continuing his description of insufficient funding from the government, the court clerk of Bukavu TGI declared, “Since funds or donations from the state are not forthcoming, the end of ProJustice in Bukavu also means that stationery would no longer be provided to the TGI.” He added that the court would have no choice but to revert to the old system of asking for additional fees from court users to cover the costs of papers and other supplies. The finance and budget officer of the Permanent Secretariat8 also mentioned the fact that official figures disclosed by the state reported an increase in government funds spent on the judicial system overall as compared to the situations in 2008 and 2009, with increases being gradual over the years. He, however, cautioned that these figures did not reflect the situation prevailing in the offices, where funding levels remained very low and not increased as officially reported.9 This issue is explored further in the next section on discussions. Supply factors are vital to the development of judicial institutions. They provide useful resources to magistrates and non-magistrates, enabling them to fulfill their responsibilities as members of the judicial system. An important factor on the supply side is the human resources aspect within courts and tribunals in treatment sites. On this front, ProJustice identified the absence of relevant legal texts as an obstacle to enabling magistrates and court clerks to work using relevant documents. The program thus developed activities that would ensure that CSM and MoJ staff working in courts would have up-to-date legal texts. Plans were also made to hold several trainings on legal procedures, ethics in the judiciary, budget planning, and management. Still on the human resources issue, ProJustice aimed to reinforce the capacities of court clerks in office management skills as these were seen as an important factor in improving the management of the courts and tribunals. Another factor on the supply side was the logistics of running courts and tribunal activities. ProJustice had provided the courts with printers, computers, and office furniture.10 The program also identified the logistical support needed for magistrates to visit several prisons in order to participate in mobile courts, which is critical to reducing delays in the handling of cases. 3.1.2 ProJustice work on the demand side In Bandundu, the evaluation team met with three victims of sexual violence who had received support from Ceprosoc, a local NGO that benefited from two grants in Bandundu city during program implementation. Two daughters of one family in Bandundu city were raped by their landlord. The victims’ father initially started prosecution formalities with an independent lawyer but could not afford to complete them. After being issued with a certificate stating that he was a “vulnerable” person together with his family, 11 he received support from Ceprosoc pro bono lawyers, who succeeded in getting the wrongdoer sentenced to 7 years’ imprisonment at the TGI level, then15 years at the appellate level, and fined 8 This magistrate made this point on 19 December 2013, as we asked him to clarify the ambiguity between reported high figures of the judicial system and continued underfunding by the government. 9 This topic is discussed further in the Conclusions and Recommendations sections (sections 4 and 5). 10 Prosecutor of Bandundu TGI reported that his court had received two printers that he managed to exchange for one at a local store that consumed less ink. The exchange was authorized by ProJustice, he added. 11 This certificate is issued by the mayor or governor and classifies a person as vulnerable from a social point of view. The certificate grants the bearer the right to receive pro bono services from a lawyer. ` Final Evaluation: ProJustice Program in the DRC 12 $3,000. The two victims were 12 and 6 years of age at the time of the rape. During his interview the father declared that without the help of the program, he would never have won the case because he could not afford the legal fees and the money needed to pay an independent lawyer. He said prior to receiving legal assistance from the NGO, he had spent two months suing the landlord but was unable to make any progress due to a lack of sufficient resources.12 Still in Bandundu city, a 16-year-old girl was impregnated by her boyfriend, who was over 18 years old. A friend of the boyfriend tried to perform an abortion on the minor. The DRC law against sexual violence criminalizes sexual contact with a minor, so the family of the victim sued both men with the help of Ceprosoc lawyers who benefited from a ProJustice grant. The boyfriend and his friend were sentenced to four years’ imprisonment. The family of the victim also said that the conviction of the two men would not have been possible without Ceprosoc’s legal assistance, using a lawyer from the CSO Promotion de la Santé de L'enfant et de la Femme (PROSADEF).13 In both cases, owing to the limit of the grant duration to three months, PROSADEF ran out of resources to take the cases to the point where the victims could be compensated for damages by the assailants. The lawyers complained about the fees necessary to get copies of the judgment (approximately $10/page), which made it impossible for them to deliver copies to the victims for their use in the future to undertake proceedings to claim damages and compensation as the law allows.14 During the interview with PROSADEF, lawyers supported in Bandundu city by ProJustice expressed a wish to see the grant period extended to cover costs through the pronouncement of judgment in these cases. Two NGOs, one in Katanga and another in Bandundu city, stated that through ProJustice support, they managed to advance, with the support of legal defenders, 22 and 15 legal cases, respectively, of vulnerable people who were experiencing administrative delays. In Bandundu the NGO Ceprosoc also created a network of legal defenders. These legal defenders were volunteers who worked in legal clinics and were trained to provide initial legal assistance to residents at the community level. This network continued to function even after ProJustice support was discontinued a year before. A Ceprosoc15 representative stated that both the legal defenders and the lawyers are motivated to continue supporting vulnerable people out of compassion and their principle of offering pro bono services. The program approach in the treatment sites and tribunals was to address both the supply- and demand￾side aspects of the judicial system. The concrete results expected from this approach were to enable better trained and more effective staff; speedier treatment of cases (fewer backlogs); a justice sector less dependent on the executive (government); better informed communities regarding judicial proceedings at courts and tribunals; and lawyers who are more supportive of judicial cases affecting vulnerable citizens. 3.1.3 Results of the program approach on the judicial institutions The approach used by the ProJustice program on both the supply and the demand sides seem to have impacted Congolese judicial institutions. Institutional management of the MOJ was strengthened, 12 Interview with the father in front of the elder daughter was held on August 29 in Bandundu-ville. The corresponding court case was RP 7623. 13 PROSADEF is a local NGO that benefited from the grant given to Ceprosoc. It is a group of independent lawyer that provide legal assistance to vulnerable population. The case number here was RP 7622 14 According to PROSADEF lawyers interviewed on August 29 in Bandundu-ville, some judgments could take as much as 10 pages, while the value of the grant per victim was $170. Efforts and focus were therefore getting the judgments passed prior to securing copies for the victims. 15 This person was interviewed on 28 August in Bandundu-ville. ` Final Evaluation: ProJustice Program in the DRC 13 disciplinary chambers were established in four target provinces and court operations changes as a result of ProJustice programming. Management of the MoJ. According to the director of cabinet of the MoJ, the organizational audit conducted with ProJustice support has helped the ministry have a better picture of its structure and define the roles and responsibilities of its staff. Additionally, their capacity to manage the judiciary has been increased as a result of the equipment and the various trainings they have received in budgeting, human resource management, and the like. Regarding the CSM. Unlike the situation prevailing prior to the ProJustice Program, the CSM was provided with a functional office outside the MoJ and could hold disciplinary chambers in the four target provinces. According to the general prosecutor, ProJustice is responsible for the establishment of the disciplinary chambers in Bandundu, as there was no other development partner working on this in the province. In an in-depth interview the permanent secretary of the CSM indicated that although significant steps had been made to strengthen the CSM, the structure still needed to be supported during the nascent stages of its existence in order to ensure that it is consolidated and independent. Pilot Courts. Respondents noted the improvement made in the filing system, the increased capacity that resulted from the trainings, and the logistical support received from the program as positive results stemming from ProJustice. In addition, the representatives of several local organizations who met during the data collection phase mentioned that court operations had changed as a result of some of their activities. For example, Ceprosoc declared that the “open days” organized by the program had led to the adoption of a fixed financial fee paid by users to file complaints within courts and tribunals in the city. That activity was part of a much larger package disseminated by the program in its attempt to increase effectiveness and transparency of court operations. But several magistrates, users of pilot courts, and CSO representatives reported that much work still needs to be done to ensure that justice is fair and equitable at the court level. 3.2 ARE THE PROGRAM’S RESULTS APPLICABLE IN THE CURRENT CONTEXT OF DG AND JUSTICE SECTOR REFORM NEEDS AND OBJECTIVES? 3.2.1 ProJustice results in the management of the judiciary ProJustice supported the establishment of disciplinary chambers at the national level and four in the provincial capitals of treatment court sites.16 Tetra Tech DPK management also informed the evaluation team that the creation of provincial chambers in Bandundu, Lubumbashi, Bukavu, and Kindu triggered the establishment of others by the United Nations Development Program (UNDP) in the rest of the DRC provinces. The information submitted by the ProJustice team was corroborated in Bandundu by the TGI lead prosecutor, who stated that the program was the only institution that worked to set up a disciplinary chamber there. ProJustice provided technical and financial support to the chambers, and 16 Information recorded after a discussion with Tetra Tech DPK management. ` Final Evaluation: ProJustice Program in the DRC 14 Tetra Tech DPK expected that the state would take over these particular responsibilities after the close of the program, in order to ensure that the chambers continue to function in the future.17 3.2.2 ProJustice and the CSM On another front, according to the permanent secretary of the CSM, one of the most important results that ProJustice achieved with regard to supporting institutions is the reinforcement of the CSM. Apart from providing a functional office to the CSM, the program also organized trainings for members of the CSM and MoJ and helped them put in place procedures and policy documents to run their institutions. 3.2.3 ProJustice and the training of court personnel There is an observable compound effect from the training organized by ProJustice. In fact, the program developed a training the trainers’ module that functioned in pilot sites. Positive outcomes of these efforts included the hiring of some of these trainers by other programs such as PARJ, according to one PARJ staff member interviewed.18 He said that magistrates that had received ProJustice trainings were able to organize and run trainings with very little support from PARJ thanks to the skills acquired on the USAID￾funded project. Another aspect of the positive effect of this capacity building is in pilot sites the trainers are field based and no longer need to come from Kinshasa. These are what are referred to under the project as “resident trainers” (i.e., “field-based trainers”).19 Lastly, another positive outcome is increased coordination with other donors regarding training modules. According to the project’s former chief of party (CoP), the work of ProJustice has received recognition for the use of its modules by other donors. 3.2.4 ProJustice results at the community level At the community level, the program sought to increase transparency and effectiveness of court operations using a variety of approaches. Local CSOs were subcontracted to provide legal assistance to vulnerable people within their communities. In addition, ProJustice influenced the attitude of communities towards the justice system through its “open days.” These events offered community members the opportunity to gain clarity from magistrates on various issues related to judicial procedures and to raise questions about ongoing trials or prosecutions. The Ceprosoc administrative assistant reported that in one instance, a community member who had attended an open day session accused a magistrate of misconduct. The accused was subsequently removed from the case on which he was working.20 Several needs still unmet despite positive results Despite the positive results achieved by ProJustice, several needs in the justice sector are still unmet. One magistrate from the permanent secretary of the CSM21 stated that there was a need to continue working on the independence of the judicial system from the MoJ. He said that the MoJ still wanted to 17 According to Tetra Tech DPK the operation of chambers at provincial level falls under a $10,000 budget to be funded by the central government. 18 The PARJ staff member stated that PARJ inherited trained trainers from ProJustice. He was interviewed. 19 This information came from Tetra Tech DPK management. 20 Information around these activities comes essentially from Ceprosoc and Partner NGOs such as PROSADEF. The evaluation team also spoke with the president of the CSO in Bagata. 21This magistrate was interviewed on 5 September. ` Final Evaluation: ProJustice Program in the DRC 15 have magistrates dependent on the justice minister as court clerks. He advocates for dual tutelage by the CSM and MoJ for magistrates and court clerks, respectively. He said that the MoJ was using article 25 of the Law on Public Establishment to try to force its tutelage upon the CSM. He further argued that the autonomy of the judiciary had been declared by Parliament through a legislative text with a higher scope than article 25, which organizes activities of Public Establishments (such as courts). He said that ProJustice had provided offices and training to the CSM and that this was useful, but a more sustainable solution would be for the state to fund the judicial system properly. According to him, greater collaboration was needed between the judiciary and the government in order to reduce its reluctance to fund the judicial system. Such collaboration would increase the power of the judicial system. A PARJ staff added that the magistrates still needed to free themselves from the supposed strength of the executive power as they still viewed themselves as “subordinate” to the government. To promote the independence of the judiciary system, the next steps for the CSM included reinforcing its organizational structure. He said that in the coming months, PARJ would be building the headquarters for the CSM, but the structuring of the office and roles and responsibilities could be supported by a project like ProJustice. 3.3 WHAT IS THE EFFECTIVENESS OF EACH OF THE PROGRAM-EXPECTED RESULTS? 3.3.1 Component 1: Support the establishment of new judicial institutions ProJustice helped strengthen the CSM. When the program was launched in 2008, the CSM was a nascent body within the judicial system with no functional office. At that time, a law was drafted to create the CSM, but no actions had been taken. ProJustice enabled the CSM to set up a functional office, achieved immediately after the program was launched. In addition, ProJustice supported board members with equipment and capacity-building activities. According to the first president of the appellate court of Bandundu, ProJustice is the sole institution that has successfully and effectively worked to establish the CSM at the national level with benefits flowing to the provinces. At the provincial level, the program helped provide trainings and technical documentation to improve the capacity of magistrates and court personnel and transformed the filing practice within the courts’ administration. The president of the appeals court of Lubumbashi stated that ProJustice had greatly contributed to the capacity building of magistrates and other personnel through trainings and distribution of several documents on DRC laws. He suggested that the trainings and the documents be offered in other provinces as well, and that libraries be set up to ensure the permanent capacity building of judicial and MoJ staff and lawyers. ProJustice worked with CSM board members to develop and adopt rules and procedures for the organization, which were adopted almost a year after the program began. After these rules and procedures were adopted, ProJustice continued to work for the election of a permanent secretariat for the CSM, and following a call for applications, the permanent secretariat was elected. The CSM is now a functional body recognized by the government, according to its permanent secretary and to the former CoP of ProJustice. It is now responsible for the appointment of judges.22 The CSM also prosecutes 22 In practice, the CSM makes suggestion to the president, who appoints the judges. This information was corroborated by one CSM magistrate, who said that there was an acceptable match between suggestions of appointment from the CSM and effective nominations by the president. ` Final Evaluation: ProJustice Program in the DRC 16 judges who break the rules and speaks on their behalf across the country. However, according to the CoP it still does not have fully independent decision-making powers. After the CSM was established, ProJustice helped it set up disciplinary chambers and supported CSM’s implementation of disciplinary action against magistrates. Disciplinary chambers functioned with ProJustice support in the four provinces where the program was implemented. These disciplinary chambers helped boost the credibility of the judiciary because magistrates accused of wrongdoing were tried and sentenced, if found guilty. In Lubumbashi, a local CSO member interviewed reported that a magistrate had been arrested and then suspended after being found flagrantly harassing a businessman’s truck driver for illegal fees. (The truck was released after investigation.) In Bandundu, a magistrate was removed from a case following a complaint from a community member during an open day session. Another was suspended after being found guilty of unlawfully prosecuting several CSO members under the pressure of the governor of Bandundu.23 Interventions planned with the constitutional court and other newly established bodies in the judiciary did not take place because the court and these bodies had not yet been established. As a result, over the five years and by the end of September 2013, the CSM was the only new institution with which the program worked. Overall, it is clear that the establishment and operation of the CSM have played an important role in the independence of the judiciary in the DRC in recent years. 3.3.2 Component 2: More effective and transparent management of the judiciary and the MoJ According to the director of the cabinet of the MoJ, ProJustice supported the development of the MoJ organizational structure by enabling five MoJ officials to be trained in human resources management. Additional activities undertaken by ProJustice with the MoJ included training staff in budgeting and conducting advocacy sessions to increase the budget of the judiciary. The MoJ has specifically benefitted from trips organized by the program to the United States and to Benin. The objectives of both trips were to show participants success stories and new procedures used elsewhere to manage the judiciary. On the budgetary front, Tetra Tech DPK informed the team during an interview that the judiciary budget had increased by over 300% as a result of ProJustice, which had trained magistrates and some court clerks on budget preparation. During a field interview, the principal court clerk of the appeals court of Bandundu reported that in July 2013 the government sent 240,000 Congolese francs to the court, for the first time since the beginning of the year, for the running of the office. However, he added that the government disburses very few funds with respect to judiciary needs, which translates into insufficient performance and accountability on the part of court staff. The principal court clerk of the TGI of Bukavu stated that ProJustice had provided the tribunal with storage boxes to file cases and with shelves and office stationery. These donations were made alongside similar gifts in kind from other donors including MONUSCO and the American Bar Association. He also stated that the ProJustice support helped bring magistrates from Kinshasa to Bukavu during trainings, which had helped to increase technical expertise. The program also provided trainings for court clerks and public prosecutors and had given them various handouts on various legal issues. The court clerk also stated that the School of Training for Judiciary Personnel had been closed for many years and had 23 Both of these facts were reported by Ceprosoc staff members in two different interviews. ` Final Evaluation: ProJustice Program in the DRC 17 reopened under the ProJustice Program. He added that in the medium to long run, the school will help increase the capacity of the judicial staff as well as reduce the deficit of personnel in the judiciary. Lastly, the ProJustice program helped increase the speed of case treatment, as sometimes judges are transported (often with MONUSCO’s help) to enable hearings taking place at prisons. 3.3.3 Component 3: Catalyzing more effective, transparent, and accessible court operations in pilot jurisdictions Court user satisfaction surveys, household surveys, and beneficiary FGD revealed public opinion on the effectiveness, transparency, and accessibility of the courts served by the ProJustice program in the DRC (Table 6). Results of this analysis of this data suggest that variation in these three areas was greater between provinces than between treatment (pilot) and control courts. This is likely driven in part by the ProJustice intervention, in combination with external factors related to the specific localities. TABLE 6. EFFECTIVENESS—COURT USER SATISFACTION BY PROVINCE Percentages in the Affirmative Treatment Courts Control Courts Bandundu Katanga South Kivu South Kivu It was easy to get the information I needed when I came to the courthouse. 35 63 46 85 I felt comfortable coming to the courthouse because the court fees were clearly posted. 10 2 24 38 Court personnel had all the records I needed to be seen by the judge. 80 66 44 54 Court personnel treated me with courtesy and respect. 65 64 59 85 The judge hearing my case listened to me and was courteous, respectful, and fair. 60 32 44 85 I understood the instructions of the judge/court and what I need to do next. 85 54 74 85 My case was handled promptly and efficiently. 10 18 29 31 I feel I was treated fairly by the court. 45 25 24 54 I think the court performed impartially and effectively in my case. 35 25 21 54 Effectiveness. Approximately half of treatment court users were able to receive the information they needed when they came to the courthouse and felt their experience was efficient. But substantial variation exists between provinces. For example, 42% (Bandundu 35%, Katanga 63%) were able to receive the information they needed, and 24% (Bandundu 35%, South Kivu 24%) felt that their cases were handled efficiently. The number of court user respondents from control courts was low, which may reflect the inefficiency of the control court. No clear trends or patterns were identified in the court user data, suggesting that each court faces a unique set of conditions in delivering an efficient court experience. Case management was a key aspect of the ProJustice Program. While the average time to process a case, as reported in the ProJustice Performance Management Plan (PMP), may have been reduced, the population’s experience indicates that much work remains to be done. The majority of court users— 59% in treatment courts—felt that their case was not handled promptly or efficiently. Only 10% of treatment court users in Bandundu, 18% of treatment court users in Katanga, and 29% of treatment ` Final Evaluation: ProJustice Program in the DRC 18 court users in South Kivu felt their cases were efficiently managed. The household survey results suggest similar findings in that 43.3% of individuals who had their cases processed at a treatment court reported a favorable speed in the resolution of their case, whereas 32.5% of respondents feel that the treatment court handles cases more quickly now than in the past five years. The broader household survey found that 45.22% of treatment communities were satisfied with the manner in which the court handled their cases. In both survey tools, less than half the population viewed the court’s efficiency favorably. The slight improvement in household respondents’ answers is likely due to the change in attitudes attributed to the passage of time. The evaluators expected court users to respond less favorably as their experience was measured on the day of transaction. While all figures are relatively low, it is encouraging that women reported a similar level of satisfaction as the general population. There is household survey evidence that suggests ProJustice was able to affect the efficiency of court management (Table 7). Nearly twice as many individuals in the treatment communities (32.5%) felt an improvement in case-handling times over the reported experience in the control communities (16.6%). In Katanga Province, individuals in treatment communities are now twice as likely to use the court system in the future, with 43% versus 22% in the control communities. TABLE 7. COMMUNITY VIEWS ON COURT EFFECTIVENESS (%) Total Sample Women Respondents Treatment Courts Control Courts Treatment Courts Control Courts What is your impression of the quality of service you received? Good 24.8 23.4 24.8 22.5 Acceptable 19.1 9.0 19.9 9.9 Poor 49.4 62.8 48.2 62.0 How would you rate the professionalism of the court officers in resolving your case? Good 19.1 19.3 17.0 14.1 Acceptable 28.7 20.7 30.5 21.1 Poor 46.5 49.0 44.7 52.1 How do you judge the speed of the court to resolve your case? Good 19.8 20.0 17.7 25.4 Acceptable 23.6 13.8 26.2 11.3 Poor 49.0 62.1 48.2 59.2 Do you think the court handles cases more quickly now than in the past? Yes 32.5 16.6 30.5 16.9 No 57.6 73.1 56.7 77.5 These findings indicate improvement in case management in pilot jurisdictions. Interviews with court personnel found consistent self-reporting among court clerks and secretaries that their capacity to deliver more efficient services to court-users was a direct result of ProJustice support. The effectiveness in court operations appeared to be linked with the presence of legal aid for beneficiaries of CSO support. FGD participants consistently remarked how the assistance of a CSO expedited their case handling and improved the overall effectiveness in bringing a case to trial. One female FGD participant from Bukavu summarized the reality succinctly, “before [legal aid] a case could be 2, 5, to 10 years before a judge would make a solution, but with CSO help there can be a solution [in ` Final Evaluation: ProJustice Program in the DRC 19 the court] in less than 5 months.” In all three provinces, over 75% of FGD participants have reported that their cases had been effectively moved to trial and that their cases had been completely processed. With regard to the ProJustice experience of court personnel, key informant interviews generated favorable results in relation to improving the efficiency of the courts while highlighting areas for continued improvement. The prosecutor of the TGI of Bandundu stated that, “thanks to the support of ProJustice, the TGI managed to rule on 100 cases in a single year for the first time in its history.” The treatment court in Bagata, which received the least ProJustice treatment court resources, highlighted some of the challenges to efficient court operations it still faced. One magistrate interviewed in Bandundu reported that, “The lack of a usable prison in Bagata—the one available being in complete disarray—has made it impossible to have effective prosecution and when detainees were being transferred to Bandundu some of them escaped.” Transparency. Corruption is institutionalized at virtually every level of the Congolese judiciary and manifests itself in a variety of forms in the tribunals. Two ProJustice efforts to reduce the irregular costs associated with bringing a case to court were (1) providing each treatment court with public signs clearly documenting all court fees and (2) building information kiosks staffed by volunteers from local civil society groups. The evaluation team did not observe fee information publicly displayed in any of the treatment courts. In some cases, staff could produce the information at the team’s request. One such example was the appeals court in Bandundu, which possessed two printed court fee signs. Both signs were located in a locked room inside the courthouse. When asked why the signs were not made available outside the courthouse entrance, the staff informed the evaluation team that the signs were stolen each time they were displayed publically. All nine treatment courts visited failed to clearly display court fees in a public space. Similarly, not one information kiosk visited by the evaluation team was staffed. The lack of staffing was a function of the winding down of the ProJustice Program. Congolese partners were not prepared to staff the kiosks without funding from the program. The additional value the kiosks brought to the overall court experience when compared with the civil society and legal aid support mechanism is unclear. No direct beneficiaries in either the FGD or program surveys reported using the kiosks. The lack of cost transparency is also reflected in the survey data. Only 13% of court users in the treatment courts felt comfortable bringing forward cases because the court fees were clearly documented and available (Appendix 1: Court User Table). The treatment courts in South Kivu demonstrated the best record, with one in four court users reporting clearly posted court fees. The control court in South Kivu reported the best outcome in court fees being made clearly available, with 38% of court users reporting favorably. While the evaluation team did not specifically investigate the source of fee documentation, it is likely that during the five years of the program court users would have visited the courts when fees were clearly posted. ProJustice staff reported repeatedly requesting that the court fee signage be displayed publicly and frequently posting the fee signs themselves. With no clear and transparent court pricing, the costs associated with bringing a case before a judge in the DRC varied widely (Table 8). The monetary cost for pursuing a case at a treatment court averaged $300.83, whereas control courts demanded an average of $394.16. When the costs are broken down by province, additional interesting trends appear: treatment courts in Bandundu, Katanga, and South Kivu provinces showed substantial variations in the sample at $148.89, $276.54, and $395.74, respectively. ` Final Evaluation: ProJustice Program in the DRC 20 Note that when only women are accounted for in the sample, the cost of judicial processes dropped to $241.75 in treatment communities and $367.29 in control communities—a reduction of nearly 20% and 6.8%, respectively. Women in the sample did not appear to demand different services from the court. A possible explanation, provided by FGD participants in all three provinces, was that women were seen as having fewer financial resources from which to extract fees, and thus their court “costs” were lowered accordingly. The clear majority of local Congolese deemed these costs very expensive across the entire sample, with less than 10% of the population suggesting costs were not expensive. TABLE 8. ACTUAL COST OF JUSTICE IN THREE PROVINCES (standard deviations shown in parenthesis) Average Price (US$) Treatment Communities Control Communities Observation Total Sample (entire population) 300.83 394.16 249 (493.57) (611.06) Total Sample (female only) 241.75 367.29 169 (297.57) (560.2) By Province Bandundu 148.89 — 46 (164.16) — Katanga 276.54 225.38 127 (398.52) (301.75) South Kivu 395.74 442.38 184 (643.39) (667.48) Ultimately, the monetary costs associated with the courts also carried opportunity costs. The lack of transparent court costs was a significant reason respondents in the community surveys opted to use alternative dispute mechanisms. Nearly 60% of respondents in treatment communities did not use the court system because they were worried about the price they would have to pay. The evaluation team queried survey respondents who opted for alternate dispute methods rather than accessing the court on their perception of court costs. These respondents estimated the cost to use the court would average $248.55, somewhat lower than the actual average cost of justice of $300.83 in the treatment courts. Although the variance in cost per province is considerable, the lowest reported estimated costs in each province suggest the cost of justice remains too high for most Congolese to pursue legal action through the formal court system. Over 80% of both the treatment and control communities reported using alternative dispute resolution methods, with most individual respondents preferring to use alternative arrangements to resolve their problems. (See Appendix 2: Table Notes G5—Those who chose not to use the courts.) Two thirds of those who used alternative methods reported that their problem was resolved. Accessibility. On average, over half of all court users found it difficult to be physically present at the courthouse. Common reasons provided by FGD participants suggest that, particularly for women, finding friends or family to care for children and time away from household responsibilities were major obstacles to court attendance. Overall, FGD participants cited having difficulty in arranging for witnesses to support their cases and transportation costs as significant contributing factors. Only in Lubumbashi did two out of three court users say that they found it easy to access the tribunal. The ease with which court users arrived at the courthouse did not vary statistically between the treatment and control courts. ` Final Evaluation: ProJustice Program in the DRC 21 CSOs were most active at the tribunals in Lubumbashi, Katanga, where 41% of court users received some level of support from a CSO or legal aid organization before arriving at the courthouse. Evidence collected through FGD and CSO members in the east suggests that the increased number of court users in the control court of South Kivu (23%) likely indicates increased international donor community presence, where there are multiple donors contributing funds to a variety of local CSOs. Within the treatment areas of Bandundu and South Kivu, only 15% and 5% of court users had the benefit of CSOs, respectively. While the court user surveys provided a snapshot of a day’s activities at the courthouse, the household survey was able to capture perceptions over a longer time period. Treatment communities were twice as likely as control communities to receive legal support while pursuing their cases. Generally the respondents felt that the quality of legal aid available was acceptable or good. Between those households that used CSOs and the overall sentiment among CSO users, civil society seemingly is filling a portion of the gap in the judiciary system. As a male lawyer in Lubumbashi stated, “People do not know their rights and do not know how to claim their rights. Awareness has made known to the people that they had a free consultation with lawyers to plead their cases.”24 This awareness, and increased access, was observed in treatment jurisdictions. For all survey respondents, access to legal support did not appear to be a strong motivator to use the court system: one in five respondents actively sought legal support and one in three reported awareness that free legal support was available in their communities (Table 9). This reflects a general absence of knowledge about available legal aid services within the communities. However, treatment communities clearly sought out and used legal aid services nearly twice as often as control court communities. In the case of FGD participants, legal aid was consistently mentioned as the fundamental reason for success in bringing a matter before the court. Legal aid and legal clinics were highlighted as the most effective method for bringing about justice for the population in all three provinces. To quote one female FGD participant/ProJustice legal aid beneficiary from Bukavu, “The NGO made us feel like we had someone behind us giving us strength.”25 The findings also show an asymmetry in the flow of information on CSO legal services between specific groups within communities—for example, those who attended a church where the pastor shared information on local CSOs (16.43%) as opposed to a portion of the community population that did not benefit from any public outreach and remained uninformed (11.19%). TABLE 9. ACCESS TO LEGAL SUPPORT (HOUSEHOLD SURVEY) Treatment Communities (%) Control Communities (%) Percentage of respondents who say they had access to legal support while they were pursuing their court case Total Sample (entire population) 47.45 27.59 Total Sample (female only) 40.4 28.2 By Province Bandundu 87.04 — Katanga 51.49 31.3 South Kivu 26.19 26.55 24 6 Sept. 2013; Dynamique des Femmes pour le Développement du Congo FGD. 25 18 Sept. 2013; OCET FGD. ` Final Evaluation: ProJustice Program in the DRC 22 Quality of Service. The overall quality of service experienced at the courthouse is a combination of the effectiveness, transparency, and accessibility of court operations. In general, both treatment and control communities reported similar levels of satisfaction in the quality of service they received at the courthouse 24.8% and 23.4%, respectively (Table 10). One in four individuals reported a good overall impression of the quality of service received at the local tribunal, with one in five reporting a high degree of professionalism from the court officers in resolving their case. Nearly half of all respondents felt the court officers and the overall quality of service at the courthouse were poor. TABLE 10. COURT SERVICE PERCEPTIONS (%) Treatment Courts Control Courts What is your impression of the quality of service you received? Good 24.8 23.4 Acceptable 19.1 9.0 Poor 49.4 62.8 How would you rate the professionalism of the court officers in resolving your case? Good 19.1 19.3 Acceptable 28.7 20.7 Poor 46.5 49.0 3.3.4 Component 4: Increased access to justice for vulnerable populations Under Component 4, ProJustice sought to support access to justice for vulnerable people by establishing grant opportunities for CSOs to provide vulnerable people with legal services. During the lifetime of the program, the term “vulnerable” was defined as all those who were victims of sexual violence, or all children or elderly people in need of legal services. The program forged partnerships with CSOs working to defend human rights in the target areas, as well as with Bar associations in these areas that could provide legal services. Thus, any person who was a victim of sexual violence, or a young or old person in need of legal support, could potentially be supported by the ProJustice designated CSOs in each province. These CSOs designated to provide legal support to citizens were selected annually. The activities of the designated organizations were monitored by the ProJustice coordinator in each province. In all four provinces the approach to working with designated CSOs was the same. ProJustice would identify a group of CSOs in each province, perform a rapid organizational assessment and based on the results of the assessment, select a group of CSOs to be supported. Studies were to take place annually in communities where short pilot programs were carried out to monitor the work done by the CSOs and the improvement in court services. The selection of beneficiaries was undertaken directly by local, community-based CSOs that were already known for their activities in the area of human rights. Information was spread through awareness-raising sessions via radio, plays, or direct meetings with community members. At times, potential beneficiaries approached the CSOs, which then enlisted them in the program, within its funding capacity limits. During data collection, 34% of women interviewed in the communities undertaking the short pilot program stated that they had benefited from legal support within their communities, as opposed to just 6% in the control court communities. Among women who had decided to bring their cases before the ` Final Evaluation: ProJustice Program in the DRC 23 courts, 40% stated that they had access to legal support from a local organization under the pilot scheme, as opposed to only 28% in the control court communities (where CSOs not funded by the ProJustice where still providing a very limited type of legal support to community members). Over its five-year lifetime, the program had supported the provision of legal support to approximately 2,626 vulnerable people in pilot/treatment court communities. These beneficiaries were supported by several organizations in each province. For example, in South Kivu, the representative of the CSO Œuvre Communautaire pour l'Education pour Tous (OCET) stated that the organization had supported 24 people in 2012. The representative of the CSO Fondation Mgr Emmanuel Kataliko (FOMEKA, in the same province) revealed that his organization ultimately supported 80 people with the grant received from ProJustice. However, during the focus groups organized for the beneficiaries of OCET and FOMEKA, two participants stated that they had to pay fees to the court, even though they were defined as vulnerable beneficiaries under the program. According to the representatives of these organizations, this was a normal practice in cases when the proceedings were lengthy. Indeed, according to the latter, they did not receive enough funds from the program to meet the costs of all cases. They added that in addition to the problem of inadequate resources in meeting the needs of those selected for assistance, they were forced to reject applications for support from other vulnerable people due to greater demand than funding and a lack of additional funding. In addition, an average of 131 people (less than 5% of the case load in pilot courts) received legal support in each province for each year of the program. This figure appears very low given the number of vulnerable people in each province. According to representatives of OCET and FOMEKA, their work was limited by budgetary constraints rather than a lack of demand. FOMEKA, for example, stated that ProJustice contributed up to $195 per person supported; OCET received only $80 per person. This sum covered lawyers’ transportation costs, and sometimes those of the victim, from their homes to the courts, and occasionally covered court fees too. These amounts, according to the two legal assistance organizations, were not sufficient to meet all the costs incurred to manage a case. To solve this problem they often appealed to legal defenders, who lacked the same level of qualification as lawyers but could at least offer assistance to these people during their legal proceedings. At the Bar, which comprises solely lawyers, all cases taken on behalf of ProJustice were defended by lawyers. However, both the president of the Bar of Bukavu and the president of the Bar of Katanga convinced some of their members to volunteer their services and thus attained the target numbers set by ProJustice. The president of the Bar of Bukavu stressed that the grant received from the program was grossly inadequate and only enabled them to work with people living in the city, although other communities within the jurisdiction of Bukavu were often much more vulnerable. He gave the example of Walungu, 50 km from Bukavu, as well as Fizi, Kalehe, and Shabunda, where he had no knowledge of any assistance given to communities, despite these communities being the poorest and farthest from the city, and therefore the most likely to have large numbers of unserved cases involving vulnerable groups. 3.3.5 Component 5: Windows of Opportunity The fifth component of the program was to help judges establish associations where they could network and discuss issues of common interest. This component was meant to remain open to adapt to changes in the justice community by providing opportunities to discuss issues and to speak with one voice to convey their message to the hierarchy. ` Final Evaluation: ProJustice Program in the DRC 24 The program aimed to: • Empower judges, through support for a judges association, to stimulate exchange between professionals in the judiciary • Facilitate judicial conferences in which common issues and concerns related to the judiciary would be discussed • Work to introduce a strong reform leadership and a culture of excellence from top to bottom for a more effective justice system. Little was implemented on this front throughout the program’s lifetime. According to ProJustice staff, this component, which was allotted 5% of the program budget, was not implemented due to a number of external factors such as: • Weak demand for financing the activities. The activities and results to be achieved under this component were not clearly defined. As a result, the legal and MoJ staff did not seek funding for activities. • Lack of adequate opportunities for its implementation. The ProJustice staff did not put adequate measures in place to enable the identification of activities to be financed under this component and to gain USAID approval for their funding. In addition, the similarity of the activities of Component 1 with the provisions under the Windows of Opportunity meant that the majority of what was carried out was done under Component 1. This component was therefore not implemented by ProJustice. 3.4 DO COMMUNITY MEMBERS IN SUPPORTED COURTS SEE PROJUSTICE ACTIVITIES AS EFFECTIVE? Community members in supported court jurisdictions reported limited exposure to ProJustice activities. The general level of knowledge of ProJustice in supported courts was evaluated within the general community population as well as specifically through program beneficiary activities of CSOs supported by the program. The overall success of the community outreach activities within ProJustice pilot jurisdictions was assessed. In treatment court communities, nearly two in five households reported having previously heard of the ProJustice Program, whereas one in four households in control court communities reported knowledge of ProJustice activities (Table 11). TABLE 11. SURVEY RESPONDENTS WHO HAD PREVIOUSLY HEARD OF PROJUSTICE (%) Type of Courts/Response Yes No Treatment Communities 38.10 61.90 Control Communities 26.82 72.63 The relatively similar finding between treatment and control communities is likely explained, in part, by the methods used by ProJustice to increase awareness. In both treatment and control communities, the vast majority of individuals—approximately 80% in both instances—learned of ProJustice activities ` Final Evaluation: ProJustice Program in the DRC 25 through one of two means: (1) legal awareness sessions and (2) mass media broadcasts via radio or television (Table 12). TABLE 12. MEANS BY WHICH SURVEY RESPONDENTS HEARD OF PROJUSTICE Treatment Communities (%) Total Observations Control Communities (%) Total Observations At the tribunal 13.10 22 2.04 1 Legal Awareness Sessions 38.69 65 40.82 20 Radio or television 40.48 68 44.90 22 Other 7.74 13 10.20 5 The legal awareness sessions appeared to be a popular method of sharing information, particularly amongst vulnerable populations. As one female FGD participant from Bukavu stated, “[At the legal awareness session] we were taught how to introduce the issue and how to have a free lawyer.”26 FGD in all three provinces produced similar findings. Awareness sessions were instrumental in providing the public with information on the ProJustice Program and the support provided in the judicial process; however, the limited number of sessions did not address the vast lack of judiciary knowledge within the population. Highlighting this point, another female FGD participant in Bukavu shared that “vulnerable people do not have any awareness in relation to the provision in law that may give them the opportunity to go to court. Those who do not know are always afraid to go to the courts. This happens in the villages. Awareness is not well done.”27 The high level of legal awareness sessions reported in the control community surveys was clarified during the FGD held in Bukavu. The small grants component of the ProJustice Program supported a variety of CSOs in their effort to increase legal awareness. One such CSO conducted multiple trainings and provided legal support to communities considered “outside” the treatment jurisdictions. One of the CSO beneficiaries explained that “they [the CSO] came door to door in our village for us to talk.”28 Given the increased funding for targeted CSOs in the east during the first three years of ProJustice, these findings are consistent with the increased outreach efforts in the rural communities in South Kivu. Any gains made in ProJustice target courts in improved efficiency, transparency, and accessibility did not directly translate as ProJustice activities for the queried community members. This being said, the evaluators found frequent evidence in all program provinces of individual community members benefitting from the program: • In Lubumbashi, a male FGD participant stated, “the reduction of the processing time for cases according to the NGO is effective due to the interaction of all new instruments that are available to the community: mobile courts, legal clinics, and pamphlets.”29 26 18 Sept. 2013; OCET FGD. 27 18 Sept. 2013; OCET FGD. 28 19 Sept. 2013; NAPO FGD. 29 7 Sept. 2013; Lubumbashi Bar Association FGD. ` Final Evaluation: ProJustice Program in the DRC 26 • In Bandundu, a female FGD participant highlighted the value in the ProJustice-supported “open door” activity, “at open door, everyone had access to ask any question to prosecutors and judges. We could go there and learn.”30 • In Bukavu, a female FGD participant summarized the groups’ overall benefit to ProJustice as being “Awareness. We passed through the CSO to educate us by telling us about our rights and procedures in the judicial system.”31 The most frequently discussed benefits of the ProJustice Program amongst the FGD participants were (1) the distribution of documents and pamphlets that allowed beneficiaries to learn about the law; (2) availability to lawyers and legal aid at no cost to them; and (3) by providing access to free legal aid, the time and costs to the plaintiff associated with bringing a case before a judge was substantially reduced. 3.5 HOW HAVE MEN AND WOMEN BENEFITED FROM PROJUSTICE, AND HOW HAS THIS PROGRAM IMPACTED THEM DIFFERENTLY? ProJustice sought to weave a gender dimension into several activities and aspects of the program: • Gender and legislation. The program sought to address inequalities in laws in the DRC, so that each sex is truly equal and has equal access to justice. • Gender equality on the bench. The program aimed to increase the representation of women in the judiciary and court personnel, including working with and helping to develop women’s professional associations. • Gender sensitivity within the judiciary. Through training and workshops, the program aimed to make members of the judiciary and court personnel aware of their own hidden biases, and to guide decision-making throughout the judicial system for all sectors of the population. ProJustice facilitated an increased access to promotions and higher levels of the justice system for female judges, within the Congolese judiciary. This has been facilitated and institutionalized through legislation and awareness raising. The program sought to serve survivors of sexual violence and although survivors have been supported, the number of direct beneficiaries was not substantial. In the few cases where there were convictions, most survivors did not receive financial compensation. According to the president of the Bar Association of Bukavu, the reason is that when the court condemns a rapist to pay financial reparations to a survivor, the survivor must pay upfront 10% of the amount of compensation of officials who will assist with the recovery of reparations from the perpetrator. In addition, owing to a lack of an articulated gender strategy, the program did not ensure that a targeted number of women were included in trainings. Hence, there was no net increase in the number of women that were trained, and this trend reflects typical exclusion practice in the Congolese justice system. Data for the program also were not always disaggregated by sex, which allows for monitoring 30 29 Sept. 2013; CEPROSEF FGD. 31 19 Sept. 2013; NAPO FGD. ` Final Evaluation: ProJustice Program in the DRC 27 and tracking progress against gender strategy objectives. Although the program design articulated a clear interest in gender issues related to RoL, there was not a clear strategy for how this would be achieved (i.e., there were no defined benchmarks and methodologies for gender integration in the program approach). Moreover, the overall DRC judicial system is weak and is nonexistent across large swaths of the country. ProJustice aimed to combine efforts to establish the presence and existence of justice and RoL throughout the DRC with efforts to create parity and fairness for vulnerable groups within the existing and reformed justice system. The program sought to address gender issues in terms of access to and quality of justice provided by the judiciary to vulnerable groups. It also sought to provide greater access to the judicial bench by encouraging the creation of systems and policies for recruiting more female members of the judiciary and promoting female jurists to higher level positions in the judiciary. Despite the challenge of trying to treat the nuances of vulnerable communities and the challenges to gender integration, the program executed several critical interventions that focused on providing women with parity under and within the Congolese judicial system. The program interventions in the area of gender inclusion succeeded in building momentum toward creating greater parity for women in access to the Congolese bench and women’s promotion to higher level positions in the judiciary. This may in turn sustainably support a more equitable and gender-sensitive Congolese judiciary. Moreover, the success of this aspect of the program will have some of the most far-reaching impact on improving quality and access of vulnerable groups to justice in the Congo for generations to come. Lastly, the program staff were predominantly male. Few efforts were made to recruit a significant percentage of females to the staff. 3.6 WAS THE PROGRAM DESIGNED AND STAFFED TO WORK COLLABORATIVELY AND EFFECTIVELY ACROSS ALL COMPONENT AREAS? 3.6.1 Program staffing The program had a national office in Kinshasa and a regional office in each of the target provinces. All advisers, technical experts, and program managers worked out of the national office. One or two program representatives were based at the regional office from where they monitored program activities on the ground. The regional offices were visited regularly by Kinshasa-based staff who offered different types of support. At the provincial level, the program signed Memoranda of Understanding (MoUs) with the courts to be supported, which enabled staff to work within the courts with the magistrates and support personnel. The regional coordinators also acted as focal points for all the specialists on the ground. In addition to the planning activities they undertook with the courts, they coordinated and monitored the work of organizations that received ProJustice grants. According to the testimonies received, the Kinshasa office had sufficient staff to perform their tasks but the regional offices did not. For example, there were only two staff members available for the entire province of Katanga, which had a total of four pilot/treatment courts. ` Final Evaluation: ProJustice Program in the DRC 28 3.6.2 PACT methodology The benchmark development approach to the ProJustice Program was the Tetra Tech DPK proprietary Program Action Plan Trademark (PACT methodology) in the DRC. The PACT process objectives were to obtain commitment from court officials to work on agreed results packages and to provide reasonable incentives to participating courts. This approach intended to (1) stimulate progress, (2) develop a strategy for disseminating pilot court practice, (3) encourage civic engagement in court monitoring, and (4) carry out ongoing review and support of the pilot process. In theory, the PACT methodology provided a clear and structured process by which to encourage stakeholder buy-in at all levels of the program. In reality, the PACT methodology in the DRC was uneven in achieving its intended results within the ProJustice Program. The “Court Procedures and Practice Assessment,” which is part of the PACT methodology, was instrumental in shaping the programmatic design of the ProJustice Program. Workshops were held in Kinshasa and in pilot jurisdictions to define the list of desired court improvement activities in a framework document consistent with the JRP and other national judiciary objectives. The outcome of this process was the signing of four MoUs—one for each pilot jurisdiction of Bandundu, Katanga, Maniema, and South Kivu—from November 2009 to January 2010. ProJustice elected to sign MoUs with each pilot jurisdiction for three primary reasons: • Pilot jurisdictions were the core of ProJustice activities; specialized MoUs allowed for a more targeted approach in each jurisdiction and responsibilities could be better explained to the counterparts. • The specialized MoUs clearly outlined procurements and delivery of equipment to each jurisdiction. • A general MoU with the MoJ was regarded as too general for program activities and difficult to monitor counterpart responsibilities at the provincial level. The signatories of each MoU included the first president of the appeals court and the general prosecutor of the appeals court in each pilot jurisdiction in tandem with a Tetra Tech DPK representative before being forwarded to the USAID Mission in Kinshasa. A key element of the PACT process was to hold individual courts accountable for their own performance. This approach encouraged pilot jurisdictions to propose specific reform activities in the form of a plan of action that fit within the context of the PACT framework as well as recognizing and rewarding the courts’ successes in implementing these action plans. The intended result by which pilot courts would act as the agents of change in their communities was not achieved. Pilot jurisdictions were responsible for proposing their own specific reform activities, an approach that appeared flawed from the beginning because pilot courts did not select substantial activities to implement nor did they design indicators to measure their performance. Even when it tried, the program did not systematically measure progress towards objectives over time This evaluation consistently heard the following requests from pilot court personnel regarding any future funded program: • Support for training, although no specifics were mentioned as to what type of training exactly was needed • Support for resources and supplies such as filing folders and paper • Financial assistance for transportation and salaries. ` Final Evaluation: ProJustice Program in the DRC 29 The assumption that the pilot courts would develop action plans aimed at improving the quality of and access to justice over the long term was flawed. In fact, observations of the pilot courts showed that, as with most courts in the DRC, they continue to function on short-term strategies based on needs assessment. The program’s approach had no ability to penalize poor performance within the PACT process. This was largely due to a lack of any disincentive for poor performance. The same courts benefitted from the program throughout the entire ProJustice lifetime, irrespective of performance. No single interviewee during the evaluation spoke of the PACT process or his/her role in it. There appeared to be no threat of replacement, which was likely a function of there being too few courts in the pilot areas to begin with. In short, the evaluation found no evidence that the courts were bound to a specific result with any consequence. The PACT process encouraged civic engagement in monitoring court performance and accountability in all pilot jurisdictions. Although the program was successful in engaging numerous NGOs, CSOs, and law schools, its impact in monitoring court performance (fair treatment, transparency, rapidity in case management) was negligible. Low levels of capacity, coupled with high community demands, resulted in a multiplicity of goals and activities for local NGOs. Few ProJustice grantees had the sole mandate of working in legal support. Most provided legal services as a secondary focus to health and poverty assistance. The NGOs were ill prepared and incapable of providing the level of monitoring and accountability necessary within the court system. The involvement of Bar associations fared no better. Lawyers were perceived by the public, particularly amongst vulnerable groups, as an extension of the judiciary and were not exempt from public distrust, claims of corruption and/or poor performance. ProJustice missed an opportunity to engage more deeply with law schools in the pilot regions. The Catholic University of Bukavu received the greatest level of ProJustice support over the course of the program. Material support and legal clinics were provided for advanced law students; however, there was no clearly defined strategy to this approach. As such, the overall impact on civic oversight was minimal. 3.7 ARE THE MONITORING, EVALUATION, AND REPORTING SYSTEMS ADEQUATE—ESPECIALLY PROTOCOLS FOR COLLECTING AND UTILIZING DATA ON COURT MANAGEMENT PROCESSES? WAS INFORMATION EFFECTIVELY CAPTURED AND USED INTERNALLY AND EXTERNALLY? A robust monitoring system provides for an accurate description of program dynamics, which better informs program management of program design and implementation policy. A well-developed monitoring and evaluation (M&E) protocol improves the overall success of the program by providing the framework with which to address program opportunities and challenges, reflect on lessons learned, and provide valuable information through appropriate knowledge and learning systems. The PMP linked the ProJustice Program activities to program results. But the PMP lacked a clear vision for the ProJustice M&E approach and methodology, which created a disjointed guidance document over ` Final Evaluation: ProJustice Program in the DRC 30 the five-year program. In addition, the indicators selected for performance management were not clearly defined in Performance Indicator Reference Sheets, causing consistency issues in the reporting. U.S. Government F-Indicators were modified to reflect custom outputs. PMP targets were changed without documenting the reasons for the changes. It remains unclear whether the changes were formally approved by USAID. While the idea of data collection appears to have been integrated into the ProJustice management culture early on, the absence of outcome and impact indicators and clear guidance on reporting responsibilities in the PMP resulted in a fractured learning system. Recording for the PMP indicators was thorough, generally well organized, and available for reference at program close out. The ease with which data could have been validated during the program’s lifetime is unclear, considering that the source documents were kept at program field offices. However, ProJustice systems were in place to allow electronic documentation to be shared with the M&E specialist after each program activity in the field. As the evaluation occurred after the closure of the ProJustice Program, the evaluators were unable to validate the information on the documentation processes in the field. In certain instances, data reported to USAID in periodic reports were not consistent with data kept on record at the ProJustice office in Kinshasa. Confusion among program staff about the specific definitions and methodologies for tabulating performance metrics, in combination with confusion over responsibilities for reporting among the component leads and the M&E specialist, likely generated data discrepancies. The ProJustice M&E strategy was in part hindered by an absence of political will on the part of the program counterparts. The ProJustice M&E team frequently experienced difficulty in gaining access to information from the court clerks. This was true for all courts in all program areas. It was particularly challenging at the CSM and MoJ. The unwillingness on the part of court personnel to allow ProJustice staff to verify records complicated the data quality and collection process and reduced the ability of ProJustice staff to cross-check clerical records. 3.7.1 Program implementation of surveys and evaluations ProJustice implemented a key survey at the start of program implementation, which served to develop the program strategy at the initial stages of the project. In June 2009, ProJustice implemented the Justice Path Assessment Survey (J-Path) to be used in mapping the capacity of the DRC judiciary as well as providing an organizational audit of the MoJ. The J-Path evaluation was conducted in collaboration with the DAI Rule of Law Project and the EU-funded Governance project. Finalized on 20 January 2010, the assessment report mapped the DRC justice system structure as a complex and dysfunctional organization with departments that had overlapping responsibilities and missions. The J-Path report served as the basis for ProJustice Program activities in structural reform and change within the DRC justice sector. In addition, a baseline report was conducted in September 2009 to assess accessibility to justice in the pilot sites. This survey documented the populations’ knowledge of judiciary procedures, experiences, and perceptions of the judicial system. The survey would have served as an excellent baseline for the final evaluation; however, no survey results could be found during the final evaluation time period. Lastly, a variety of small surveys were conducted over the course of the program to measure program results and reporting indicators, particularly with regard to training activities. ` Final Evaluation: ProJustice Program in the DRC 31 4. DISCUSSION AND CONCLUSIONS Here we return to the questions posed in our scope of work in order to draw conclusions from the findings in each area. The sections below discuss analysis that links back to the findings and provides the basis for recommendations presented in the final section of this report. 4.1 IS THE PROGRAM APPROACH STILL RELEVANT TO DEVELOPING, REINFORCING, AND EMPOWERING CONGOLESE JUDICIAL INSTITUTIONS? As explained above, acting on the supply and demand sides of the judicial sector was indeed, and still is, important for addressing the development and support of Congolese judicial institutions. As presented through findings, the benefits of this approach are clear, with judicial staff working with up-to-date legislation and having the opportunity to discuss interpretations of legal texts with their peers at meetings facilitated by ProJustice during training sessions. In some courts more cases are processed as a result of the logistical support received from the program. The demand side is also relevant in supporting CSOs and promoting the interests of vulnerable groups and the general public within the justice system. However, it is not sufficient to solely focus on supply and demand factors of the judicial system when it comes to empowering justice system institutions, because of the absence of a commitment from the state to support the system in future. This lack of political will or government passivity shifts expectations for effective, transparent, and inclusive judicial services away from the state and towards the donor community. The program approach did not treat the role of the government as a variable, but rather as a given, as part of the unchanging context. Government support was actually treated as a risk factor.32 The Tetra Tech DPK award document for ProJustice stated that “the greatest risk was that the government does not provide minimum levels of funding and support to the new institutions and the courts to function more effectively.” 33 The program design document had therefore rightly identified the “passivity” of the government as an obstacle to achieving the overall goals of the program. Remarks from stakeholders during site visits (reported above in the Findings section) point to the lack of takeover by the state of courts and tribunals at the end of the program. This situation affects the sustainability of actions undertaken at the supply and demand levels. From a certain perspective, the root causes of the poor working conditions under which members of the control courts operated can be understood as resulting from the state’s lack of involvement and commitment in fulfilling its obligation 32 The meaning of the risk factor is that it has a great influence on the results and impact of the project. A strong support and commitment from the government would mean a high probability of a project with great impact, while less support would mean a very high likelihood of having a project with a lower level of impact. 33 See Tetra Tech DPK award, p. 17: Risk management section. ` Final Evaluation: ProJustice Program in the DRC 32 to support the justice system. In the context of reconstruction of the country, following nearly two decades of instability after Mobutu’s departure, the ProJustice Program performed activities that are typically the responsibility of the government, with the aim of encouraging it to continue them at the end of the program. Unfortunately, as it was foreseen at the program’s onset as the greatest risk, the GoDRC never really played it part. In addition to maintaining efforts aimed at increasing transparency and increasing the support from the state toward the country’s judicial system, there is also a need to continue with the logistical support provided to the judicial system by the ProJustice program. The TGI president in Bukavu stated that efforts should be devoted to forcing the government to abide by Article 149 of the Constitution,34 which stresses the independent nature of the judiciary authority and its prerogative of having its own budget. He said that the donor community could push the government to “disburse” the money that is budgeted for the judiciary. He said, “We have to start somewhere, maybe first by supporting transportation costs” and that “this would be possible if there was a greater collaboration between the executive and the judicial powers.” The above discussion is conducted based on the assumed veracity of the statements made by some magistrates on insufficient funding from the state. Although the evaluation team did not challenge information brought forward by some key informant magistrates on insufficient funding by the government,35 USAID should be careful in taking the above affirmations simply at face value. This is due to corruption prevailing in the country and the rather high level of corruption reported under the question dealing with the transparency of judicial system.36 This point of caution is also drawn based on the fact that “corrupt” judicial staff would be inclined to overstress the lack of funding from the state to justify the corruption practices. The deputy prosecutor of Bandundu TGI stated, for example, that “a magistrate would be inclined to take a bribe if faced with an offer while having left a sick child at home.” He further added that, “ProJustice had overlooked the human resources side of the judicial system,” meaning that salaries were still too low and not always arriving regularly. A study tracing government funding for the judicial system is worth undertaking in order to reconcile key informant interview reports from some magistrates versus the increase in the budget for the judicial sector, as reported by official sources. It is understandable that at a time when the state is still being rebuilt after many years of instability, there is a need to ensure that some of its basic responsibilities are fulfilled, in order to avoid prolonged poor governance practices and the absence of RoL. The results of ProJustice program enhanced the judicial system in the treatment sites and to some extent also helped to consolidate Congolese RoL, despite the fact that the state is still struggling to meet the basic needs of the courts. This could explain why many interviewed magistrates and court clerks wished to see the ProJustice program continue. This wish was also expressed by the prosecutor of the TGI in Bandundu, as well as the court clerk of the 34 This article stipulates that, “The judiciary authority is independent from the legislative and the executive authorities.” The article also stresses the power devoted to various courts and tribunals within the judicial system. 35 This option was taken to avoid influencing the answers of the interviewees for impartiality sake. 36 The country corruption index still ranks it among the worst in the world. ` Final Evaluation: ProJustice Program in the DRC 33 TGI in Bukavu, and all participants in the final ProJustice workshop.37 The Bandundu TGI prosecutor stated that the court “had managed to stay operational thanks to the support of ProJustice.” In spite of the gains made by ProJustice to strengthen Congolese judicial institutions, many of them are still experiencing several issues that reduce their efficiency. Nevertheless, the approach used by the program is still relevant to developing and reinforcing these judicial institutions. 4.2 ARE THE PROGRAM’S RESULTS APPLICABLE IN THE CURRENT CONTEXT OF DG AND JUSTICE SECTOR REFORM NEEDS AND OBJECTIVES? Some of the most important ProJustice Program successes in justice sector reform were (1) its ability to support the development of the CSM during the early stages, (2) establishment of an adequate filing system in pilot courts, and (3) the revitalization of disciplinary chambers for magistrates in target provinces. Apart from ensuring discipline, these chambers have also strengthened the judicial system by shifting the burden of decision-making regarding the disciplining or firing of magistrates away from the executive branch of government. Nevertheless, the Congolese judicial system still needs to be strengthened because of its current condition. The support received from ProJustice allowed the system to perform better, but only for a limited period of time and in a limited number of sites. There is a constant need to support the judicial system on the logistical front. On the other hand, capacity building is an ongoing process. These activities also need to be spread to more sites to have a greater effect. The duration of such activities could also be extended to facilitate greater results. It is very important to seek direct support from the state as far as the logistical and training needs of the system are concerned. One option would be the building of a partnership between donors and the state to ensure a sustainable future for support activities within the judicial sector. The achievements of ProJustice are relevant in the present context, as they close a gap created by the lack of sufficient support by the state. Actions to fill this gap should be implemented in future with some concomitant commitment from the state to resume its responsibilities—even if only by agreeing to directly fund part of the program. If activities such as those undertaken by ProJustice through the CSOs did not exist, the most vulnerable members of society would remain marginalized in terms of equal access to justice in the target sites. ProJustice has the merit of having showed that the justice sector could perform well if it receives better support in terms of capacity building, logistics, and the involvement of CSOs to support court users. ProJustice demonstrated that a better justice system is a realistic objective when the necessary resources are provided. It also demonstrated that the government has a role to play in terms of sustainability. The actions undertaken by the program through the MoJ; the CSM; pilot courts; and grants channeled with the help of NGOs such as Ceprosoc and PROSADEF, Vision Sociale, FOMEKA, and others are therefore still relevant and required in the current context, as the state does not yet meet the majority of the needs of the justice sector. 37 See the recommendation to USAID outlined in the “Rapport de Cloture de ProJustice,” p. 19. ` Final Evaluation: ProJustice Program in the DRC 34 4.3 EFFECTIVENESS OF EACH OF THE PROGRAM COMPONENTS 4.3.1 Component 1 As stated earlier, the support provided to the MoJ, the establishment of the CSM and disciplinary chambers, and the holding of training sessions all contributed to strengthening the supply side of the judicial sector, which suffers from the “passivity” of the host government. Without ProJustice, the practice of the judicial profession in the treatment sites would have been much worse. For example, that 100 cases were processed in a single quarter in the TGI of Bandundu city reveals the important contribution that the program made in terms of strengthening that particular judicial institution. Another legacy of ProJustice is the training of magistrates and the provision of training modules (technical data sheets) that will be very useful in the future. One interviewee stated that the new PARJ program “was reaping the benefits of the work undertaken by ProJustice” because the magistrates recruited by PARJ came equipped with training management knowledge gained from the ProJustice Program. A discussion with Tetra Tech DPK staff also revealed that training modules developed by ProJustice for judges working at peace tribunals were being used by other partners, such as the UNDP and MONUSCO, for reinforcing the capacity of other magistrates. The legacy described here will outlive the existence of ProJustice and is a direct contribution to strengthening the judicial system through members of the CSM. In terms of impact, considering the short duration of the program, there have been no major changes that could be considered as attributable solely to the program. Positive results and effects do exist and have been presented in this evaluation, but the program was not broad enough in space and time to have a major impact. The fact that the constitutional court was not established kept the program from working on its growth, as originally planned at the design stage. As this court is a very important segment of the Congolese judicial system, future RoL programs should contribute to its strengthening in an effort to support the establishment of new judicial institutions. Recommendation. Future RoL programs should provide support to the constitutional court and the other newly established judicial institutions. In addition, for the future, it is important that activities under this component be conducted with the help of the government as a partner to ensure sustainability. This will require greater interaction between programs such as ProJustice and representatives of the government working on the reform of the sector. Actions undertaken by ProJustice, acting on supply- and demand-side factors, ought to be reproduced, in light of the government’s low level of involvement. 4.3.2 Component 2 Under Component 2, ProJustice was able to develop quality modules for the training of magistrates. That the training materials were used by other programs indicates the high quality of the training material. ProJustice successfully introduced a new filing system in pilot courts. The prosecutor of Bandundu TGI stated that the filing practices would continue even after ProJustice ends. ` Final Evaluation: ProJustice Program in the DRC 35 Other positive achievements included efforts to avoid prolonged preventive detentions. According to the deputy prosecutor of the TGI of Bandundu, during program implementation, efforts were made to abide by the legal terms of duration for detention, unlike in the past. This was possible thanks to regular visits to prisons and the organization of mobile courts. The program attempted to make pilot courts display court fees to their users, but the evaluation team noted that most of the visited courts did not put that into practice. It is the evaluation team’s understanding that such a practice would reduce the opportunity to charge extra fees to court users. The program also restarted the School of Training for Judiciary Personnel (EFPRJ), which is responsible for the training and retraining of all legal professionals, including judges. Since the school trains cohorts who will later be deployed throughout the country, it is best to work to establish quality modules on all relevant topics and provide them during training sessions for trainers at the EFPRJ. Therefore, a multiplier effect will be observed during the deployment of training beneficiaries to their postings. Recommendation. Future trainings of judiciary personnel should be channeled via the EFRPJ. 4.3.3 Component 3 The fundamental assumption in the ProJustice Program was that the effective administration of justice ultimately is local. But the general sense from this evaluation is that court systems, processes, and costs remain a mystery to most of the Congolese population. ProJustice gave significant attention to improving service delivery by the courts by working with pilot jurisdictions, incorporating and replicating best practices developed collaboratively amongst project staff co-located in each jurisdiction, central-level judiciary, and MoJ staff and expert consultants and program staff. ProJustice sought to improve case management, better manage judicial archiving, and implement formal practices to improve judiciary service delivery and facilitate access to justice in the four pilot jurisdictions. Under Component 3, ProJustice achieved the following results: • The PACT and delay reduction committees were established to produce combined proposals to improve case management, reduce delay in the short and long term, and improve archive management. The PACT and delay reduction committees met regularly to develop delay reduction and management strategies. • ProJustice provided substantial trainings to court management personnel, including magistrates, clerks, and prosecutors’ secretaries over the entire five-year period. • ProJustice delivered shelves and archive equipment to the pilot courts in the four pilot provinces and began preparing a national judiciary archive plan. The project also delivered computers, IT equipment, and furniture to all pilot courts to improve the work of the magistrates and clerks. • ProJustice successfully completed renovation work in the justice palaces of Bandundu, Lubumbashi, and Kindu. • Bulletin boards were installed to provide better information for citizens entering the courthouses. • ProJustice provided support for the standardization of mobile courts and to increase mobile court sessions. ` Final Evaluation: ProJustice Program in the DRC 36 Effectiveness. The acquisition of court supplies improved court personnel’s work experience; however, efforts by ProJustice to introduce more effective methods into the program courts were uneven and reflected across the whole of the evaluation. Despite a judicial system regarded as inefficient, ProJustice was able to make small inroads into improving the effectiveness of the judiciary. All program courts proudly showcased their improved recordkeeping methods and supplies during the evaluation field visits. The majority of court clerks and secretaries with whom the evaluators spoke described benefiting from training programs provided by ProJustice and frequently requested that this type of professional development continue in the future. These activities, which benefitted lower level court personnel, successfully improved working conditions in the courts and professionalized their efficiency. These improvements into the effectiveness of court operations resulted in marginal improvements reported by community members and FGD participants. The low levels reported in the survey results suggest that ProJustice had limited success in improving the overall quality of service experienced by Congolese citizens. In the context of the DRC, however, even limited improvement should be considered a success. Whether these improvements prove to have any sustainability will ultimately be the real measure of success in altering the deeply frail justice system. The evaluation team believes that the ProJustice Program did improve the physical environment of the tribunals and as such improved the efficiency of the courts. But ProJustice did not do enough to capitalize on the Congolese population’s ability to hold their judicial leaders accountable by providing them with adequate information and skills building on the methods for effective, transparent, and accessible court operations. Transparency. ProJustice efforts to improve transparency in the legal system were significantly hindered in every pilot tribunal. The two key strategies for increasing transparency—court fee signage and information kiosks—were stymied by a complete and total lack of interest by court personnel brought on by a judicial system that intentionally and systematically operated in the dark. The level and scope of corruption within local jurisdictions were such that no incentive existed to force the pilot courts into more transparent court operations, regardless of ProJustice intervention. This posed a significant burden on ProJustice staff, who regularly reposted signage on field visits and coordinated with local CSOs to ensure kiosks were staffed. Accessibility. The evaluators consistently found that accessibility was clearly linked to activities that increased legal awareness by the public. The GoDRC struggles to provide even the most basic of services to its population. As a result, few in the general population understand the formal processes and methods of the government. This lack of knowledge creates an environment where accountability is scarce. The local population, uncertain of government processes but acutely aware that their leaders are not held accountable, is left with little reason to seek access to justice despite a clear desire for it. The fundamental structures of the ProJustice Program did not intend to enable or engage the local population in any meaningful and empowering way. Indeed, the ProJustice strategy was a “top-down” approach to governance. In a country such as the DRC, a mixed approach would have better served the program to reinforce the governmental structures and to enable the local population with legal knowledge and awareness to hold the judicial system accountable. When the Congolese understand their legal rights, the findings suggest they will then consider the legal system accessible to them. Recommendation. Legal awareness and legal aid should be increased to include a broader reach into the population, including legal awareness sessions for elected officials and community members. ` Final Evaluation: ProJustice Program in the DRC 37 4.3.4 Component 4 The program managed to reach several people who are a priori vulnerable: women, children in conflict with the law, victims of sexual violence, and the very elderly. However, the criteria of vulnerability defined by the program do not necessarily encompass all vulnerable members of target communities (e.g., people with physical disabilities, members of marginalized ethnic groups, the extremely poor, people with chronic illnesses, etc.). Thus, even though the program did facilitate access to legal services for some segments of the vulnerable population, many others remain in these communities who, through lack of recognition, were not afforded access to legal support. Recommendation. More inclusive vulnerability mapping should be done to identify all vulnerable members of target communities. Working with local organizations has achieved the maximum possible given the budgetary constraints of the program. Working with grassroots organizations is a good approach to expand the reach of the program to the maximum number of people. Furthermore, the Bar associations of Bukavu and Lubumbashi were called upon to provide legal support to vulnerable people identified by the program. However, given that few of their members can volunteer their time, their ability to fulfill this task was more difficult than for local organizations whose mission and business model are primarily volunteer based. Indeed, the majority of people who received legal support through the program were reached through local organizations. Moreover, 84% of these beneficiaries stated that they had received high-quality legal support. One can conclude that although these local organizations are not composed of legal professionals, they may, if and when trained, respond adequately to the legal needs of members of their communities. The legal defenders used by most of these local organizations offer a valid and locally based legal service. Recommendation. Work with local organizations to strengthen their networks of legal defenders, who can provide legal support at a lower cost. One of the most important factors in defining vulnerability is economics: vulnerable people in general have very limited financial resources. For all cases heard at the courts, when a victim is due to receive financial compensation, he/she is required to pay a tenth of the compensation awarded before the court will attempt to help him/her receive what is owed. The only way to avoid this rule is to obtain a certificate of indigence from administrative authorities at a cost of $50. This requirement is an almost insurmountable obstacle for most vulnerable people in these communities. What is more, most legal documents come with legal fees attached, in addition to the illegal informal fees charged by court staff for their services. These obstacles combine to ensure that Congolese citizens in general, and vulnerable people in particular, are reluctant to seek justice through the courts. They prefer to settle their problems in alternative ways, including via traditional councils or laws, which do not always observe Congolese law. 4.3.5 Component 5 Component 5 of the program facilitated several meetings with MoJ staff. Note, however, that the objectives of this component were not achieved, due to several factors. The most decisive of these factors was the fact that the expected results of this component and procedures to release program funds ` Final Evaluation: ProJustice Program in the DRC 38 were not clearly defined at the beginning of the program. In addition, even potential activities that could be carried out under this component also appeared to be funded under Components 1 and 2. ProJustice did not achieve the objectives assigned to this component. 4.4 HOW HAVE MEN AND WOMEN BENEFITED FROM PROJUSTICE, AND HOW HAS THIS PROGRAM IMPACTED THEM DIFFERENTLY? Female magistrates have benefited from advocacy. Coupled with the efforts of others, their number has increased from 210 in 2010 to 660 in 2013. According to the president of the Congolese Association of Female Magistrates (AFEMAC), it is certain that 19% of female judges were promoted between 2010 and 2013, largely in part due to the efforts ProJustice. In addition, 75 female justices were trained in leadership skills that helped improve their capacity as leaders and influential members of the judiciary. ProJustice supported the creation of a resolution to establish a minimum target of 30% female recruitment to the judiciary annually and approaching parity through a target of at least 30% for promotions as well. Adopted on 30 April 2013, that resolution also facilitated the inclusion of 19% of female justices in the most recent cycle of promotions. Many of those who were promoted had not received promotions since their appointment. The recruitments provided female members of the judiciary with increased responsibility, influence, and career advancement, according to the president of the AFEMAC and information collected at the CSM secretariat. In the immediate and medium term, the targets and policies will help to increase the number of women in the judiciary significantly and eventually lead to more gender-equitable RoL outcomes in the DRC. In all sampled provinces, both regional program coordinators and representatives of local organizations said the legal support for community members in the pilot courts prioritized victims of sexual violence, the majority of whom are women. Several sources interviewed by the evaluation team have concurred. The program still cannot establish the exact number of victims of sexual violence that have been supported because of inconsistency in how data are collected and stored amongst program partners. It should be noted, as indicated by the regional coordinators of the program in South Kivu and Katanga, the support given to victims of sexual violence was limited to accompanying them through the court process and providing legal counsel. Often, program beneficiaries saw their perpetrators sentenced to imprisonment and ordered to pay financial compensation to the victim, which in almost all cases they have not received due to the complex procedure established by the administration. Therefore, the legal support provided by program partners to victims of sexual violence in the pilot courts while successful on convictions did not guarantee total satisfaction for survivors. Outside of the abovementioned interventions, there were no specific activities aimed exclusively at women or men. However, it is worth noting that most activities benefitted men, because they had higher numbers within the judicial system and were more likely to seek justice through the judicial system. The efforts to improve access to justice for vulnerable groups could have benefited from a baseline early on to explore the true obstacles to justice for vulnerable segments of the community, particularly women. This would have informed program design and ensured program interventions that addressed more directly the needs of all genders and vulnerable communities equally. The principal conclusion is that ProJustice’s largest contribution to gender equality was in the increase of recruitment and promotion of women within the Congolese justice system. These changes must be ` Final Evaluation: ProJustice Program in the DRC 39 solidified through cultural, legal, and behavioral shifts. With time and continued efforts, the inclusion policies and practices will be established as norms. Moreover, the support to survivors of GBV was limited because it was not possible to ensure that they could recover the court determined compensation or cover court cost. Recommendation. Future RoL programs should begin with a complete gender situational analysis and exploration of obstacles to justice for both women and men. This will ensure that program design is informed by the true gender-based factors that block access to justice and ensure that appropriate interventions are integrated and implemented. 4.5 HOW EFFECTIVE WAS PROJUSTICE IN ENGAGING, COORDINATING, AND/OR SUPPORTING LEGAL INSTITUTIONS, GOVERNMENT ENTITIES, CIVIL SOCIETY, THE LARGER DONOR COMMUNITY, AND OTHER STAKEHOLDERS? 4.5.1 Collaboration with legal institutions and governmental entities Collaboration with the MoJ and the CSM. The MoJ’s chief of staff has recognized that ProJustice contributed significantly to improving the system of management of the department. For example, in an interview he pointed out various activities piloted by the program, such as the organizational audit of the ministry, the new organization chart, and description of tasks of each agent. By supporting the nascent CSM, the program has also contributed to the improved management of the judicial system in the DRC. Representatives of both the MoJ and the CSM have noted the importance of the work that has been done by ProJustice. However, at the same time, they shared that there are still many things remaining and required to establish RoL in the country. Pilot Courts. In all pilot courts, the staff recognized the importance of collaboration with the ProJustice Program. Several interviewees acknowledged that with the support of ProJustice, they had learned new skills through trainings and through the materials and documents that they received. Nonetheless, almost all the interviewees in those same courts supported by the program have acknowledged the need for continued support to consolidate ProJustice achievements. 4.5.2 Collaboration with the civil society Local Organizations. The training sessions and the support provided by ProJustice were appreciated by representatives of local organizations. They all emphasized the importance of the work done by the program through sub-agreements that they helped to execute. However, they noted that the process used was not the best. Indeed, the program signed contracts with them on an annual basis, which according to their representatives was too short a time to be able to do all the work and trigger sustainable behavior change in their practice and at the court level. The ProJustice CoP stressed that the program could not commit to longer periods because of the funding mechanism that USAID used—that is, first a three-year contract, then two 1-year extensions. Recommendation. Facilitate the work of local organizations by giving them more time to achieve their objectives. Work with them should be based on two-year periods for any future RoL program. ` Final Evaluation: ProJustice Program in the DRC 40 The Bar Associations in Bukavu and Katanga. The presidents of the Bar associations in Katanga and Bukavu have both indicated that the collaboration between ProJustice and their organizations was good but still did not meet their expectations. Each organization worked with ProJustice as the program has done with local organizations by providing small grants to support a limited number of vulnerable people according to the criteria of the program. Both presidents considered the grants as too small to cover the legal needs of vulnerable people in their jurisdictions. The president of the Bar association of Bukavu also noted that the program should develop a training program for lawyers and give them access to the documentation distributed in the court. This, he felt, would have increased their capacities that in turn could make a significant contribution to the establishment of a transparent and fair justice system. 4.5.3 Collaboration with the larger donor community and other stakeholders Collaboration with UNDP and MONUSCO. The training modules on legal procedures developed by ProJustice were shared with program partners. According to ProJustice management, both UNDP and the United Nations Joint Human Rights Office (UNJHRO) have followed the example of ProJustice by supporting the disciplinary chambers in several other provinces. ProJustice has been invited to most RoL events organized by the UNDP or UNJHRO, and the program did the same in return. Collaboration between the program and the UN agencies seems to have been optimal. Collaboration with PARJ. Collaboration with PARJ has not been always successful, even though it and ProJustice were both working to increase the capacity of the justice sector. According to ProJustice management, the problem was that PARJ seemed to claim ownership over program results. The staff of ProJustice cited as an example the MoJ’s internal organization to which they have contributed a lot. According to their accounts, the level and quality of collaboration with PARJ decreased significantly when they realized that PARJ exclusively attributed to itself the progress made on this front. Mixed Justice Committees. The mixed justice committees were chaired by the MoJ and a representative of the EU (a grantor). The committees were put in place to facilitate the information￾sharing and activity coordination for RoL actors. ProJustice’s CoP said the program has worked with the committees by sharing information on its activities and plans and finding agreed solutions to problems faced by the judiciary. But the frequency of the meetings was often irregular and unpredictable due to lack of sufficient coordination from the MoJ, which was to initiate and schedule them. In any event, the coordination of the different activities from RoL actors has not always been optimal. Collaboration between ProJustice, UNDP, and MONUSCO appears to be a success because of the constant communication between the institutions. Each entity has been able to implement its activities, and show its results with no possibility of overlapping. In the case of PARJ and that of the mixed justice committees, lack of coordination between the stakeholders has greatly reduced the effectiveness of the collaboration. Recommendation. The mixed justice committees should be revitalized for better collaboration between RoL actors in the DRC. Importantly, almost all entities that have collaborated with the program have highlighted the importance of the work that has been accomplished under ProJustice. At the same time, they have all expressed concern over what remains to be done to establish a fair and transparent judiciary in the DRC. As a pilot program, it is understandable that ProJustice could not meet all the needs of all those entities. ` Final Evaluation: ProJustice Program in the DRC 41 Thus, in spite of the program successes, many of the issues it was trying to address are still relevant and program gains are fragile without future support and new programming. 4.6 WHAT FACTORS HAVE CONTRIBUTED TO OR MILITATED AGAINST PROJUSTICE PERFORMANCE? As discussed in the findings (section 3), there are many internal and external factors that influenced ProJustice performance. The most important among them are: • The relevance of the program • The program approach of work through local organizations • The size of the provinces and the distances between the different courts • The delay in starting the activities of the constitutional court • The lack of operating budget reported by court personnel • The duration of the sub-agreements between Tetra Tech DPK and its local partners. The relevance of the program in relation to the fundamental needs of the justice sector were key in the achievement of program results. Indeed, the justice sector in the DRC is not well supported by development partners. The support provided by the program in terms of training, distribution of materials, and implementation of procedures in the judiciary addressed fundamental needs at all levels across the DRC’s justice sector. To reach vulnerable communities and improve access to legal assistance, ProJustice worked with and through local organizations and provided legal support to nearly 2,626 individuals. Without local partners, it would have been difficult for the program to achieve this level of performance. At the organizational level, the size of provinces and the consequent scale of the distances between the different courts was a real problem that significantly reduced the level of activity. In contrast, in larger cities such as Lubumbashi, the TGI, the Tripaix, and the court of appeals can be located in the same area or even in the same building. However, a glance at a map of any given province showing courts supported by the program underscores how travel between two localities within the same province may easily take up to a day. For example, travel from the two program areas of Bandundu to Kikwit, which are supervised by the same person, took over eight hours during the period of the evaluation. On another note, although the law creating the constitutional court was filed at the office of the President of the Republic in May 2011, it was not enacted during the lifetime of the program. Therefore, aside from some advocacy activities, the program could not carry out other activities related to the constitutional court. This remained the situation until the end of the ProJustice Program. In addition, while the program advocacy activities helped to increase the operating budget of the judicial system, none of the courts visited during the evaluation acknowledged receiving support for operating cost from the government on a regular basis. They were thus in a permanent deficit situation, unable to afford paper for the issuance of acts of judgment, or cartridges for printers provided by the program. Lack of funds also limited the ability of the courts to financially participate in organizing mobile courts or replicate their use in areas previously identified as target zones. ` Final Evaluation: ProJustice Program in the DRC 42 The contracts signed between USAID and the implementing agency seemed to have impacted the capacity of the implementing partner to have long-term planning with its local partners. Indeed, ProJustice was a three-year program with two 1-year extensions. Therefore all planning made by the implementing agency and its partners was based on that cycle. After each new contract with USAID, the Agency would restart the planning process with both the local partners and the courts and sign new MoUs. The cyclical pauses and stoppages incurred during this process was noted by Tetra Tech DPK and its partners as a constant challenge to program effectiveness. Both parties would have preferred to have longer contracts of at least five years to enable them to have a longer period to plan and fund their activities. 4.7 WHAT ARE THE SUSTAINABLE ELEMENTS OF THE PROJUSTICE PROGRAM THAT WILL REMAIN AFTER PROGRAM COMPLETION? The data demonstrate that on the policy front, ProJustice contributed significantly to several highly meaningful and important accomplishments—specifically efforts to raise awareness and advocate for the increased presence of women in the judiciary. This clearly resulted in legislation that serves as a benchmark for ensuring that DRC judiciaries will have significantly increased levels of representation of women and gender parity, transforming the face of the Congolese judiciary with each cohort of new magistrates. Clear targets of 30% have been integrated into national policy to enforce and institutionalize the practice and reinforce the ownership of the CSM. These factors provide the foundation for ProJustice to have a lasting impact on gender parity in the country’s justice system. This is a positive contribution of the program that will over time transform the legislation and practice of justice and RoL in the DRC. The program’s success in relation to access to justice, particularly for women, is unclear. It is certain that simply by increasing the general public’s access to judicial services, the program improved the ability of women and vulnerable groups to access justice; what is uncertain is the quality of this access. It is not clear whether women and vulnerable groups receive equal treatment as men before the law. Tangential and anecdotal data gathered during the FGD suggest that there is some improvement in this area. But unfortunately evidence on this aspect is limited and neither clear nor systematic. The program has worked with several local organizations to provide institutional support that would enable them to in turn provide legal assistance to community members. Each year, ProJustice renewed its list of partner organizations. Each partner agency recognized the benefit of the partnership on their institutional capacities. Some, like the League for the Rights of the Congolese women Katanga, improved their tools and work procedures and will train other organizations. But to continue their work in the field, many of the partner agencies will be confronted with a lack of financial and material support. CSOs and other organizations trained by the program are now able to continue their activities on a smaller scale in the absence of the program. The program had significant success at advancing its agenda to lobby for an increase in the judiciary’s budget. The 300% increase reported is a significant achievement, although it is currently purely theoretical, according to key informants working in the justice sector. The next step for RoL programs in the DRC will be to effectively advocate for the disbursement of the operational budget funds allocated to the judiciary. Finally, the mobile courts strategy contributed to facilitating access to justice for people in more remote rural areas. While none of the courts visited during this evaluation was receiving funding to support ` Final Evaluation: ProJustice Program in the DRC 43 their operational costs, several judges and clerks who were interviewed recognized the need to continue to maintain and hold mobile courts beyond ProJustice. In Lubumbashi for example, the president of the court of appeals stated that he has prioritized maintaining mobile courts, although he will not receive program funding. 4.8 WHAT COULD PROJUSTICE HAVE DONE BETTER TO ENSURE THAT RESULTS WERE ACHIEVED AND SUSTAINED? In light of the findings, it appears that certain measures could have helped ProJustice to ensure that its results were achieved. Seeking formalized partnerships with the state and synergies with other projects such as those from the EU or the UN from the onset could multiply the effects of the program. For example, ProJustice could have provided trainings through the EFRPJ. This would have increased synergies with other existing activities and perhaps trigger support from other donors. Running it through a permanent institution (such as the EFRPJ) could also be more cost effective. The selection of the pilot and control courts should have been made based on the knowledge of sites having wide-range impact potential. The proximity or distance of sites from provincial capitals should impact funding plans, likewise sites with more or fewer international NGOs (donors) working in the justice sector should also have different funding plans. The initiatives to display fees payable at tribunals and courts should have been enforced, multiplied, and shared with other sites. Courts that did not respect protocols regarding transparency set forth in the MoUs with the program should have been forced to do so even if that required a suspension of their funding. Recommendation. Improve program design to ensure that performance-based milestones are clearly identified and reinforced. The activities under ProJustice’s Windows of Opportunity (Component 5) have not been executed. In reality, this component lacked clearly defined objectives. It was supposed to cover the resources mobilized for activities deemed useful for justice and that were not taken into account during the design of the program. It would have been more beneficial to consult with stakeholders in the field of justice to identify problems that had not been addressed by the program design. Several key informants indicated their fear and concerns surrounding discontinuation of effective program activities due to program closure (e.g., the logistical support provided for the mobile courts, the documents and trainings given by the program, the organization of the disciplinary chambers). Involving the government and/or representatives of the MoJ in the preparation of the program’s exit strategy would have been beneficial. For example, seeking funds to sustain key program activity in the future or identifying synergies with other donors. There is a need for advocacy and coordination in this area. ` Final Evaluation: ProJustice Program in the DRC 44 4.9 WHAT ARE THE PROGRAM SUCCESSES THAT DESERVE REPLICATION? Several approaches used by the program have proved to be effective and deserve to be replicated in future RoL programs. These include: • Partnerships with local organizations to provide legal support to vulnerable segments of the target communities • The signing of performance contracts with targets courts • Revitalization of the disciplinary chambers • Promoting improved filing and document management within participating courts. Program results showed that partnering with local organizations is a success that deserves to be replicated. Local organizations can help a program reach a greater number of people in target communities in the most cost-effective way. In addition, most of the time, local organizations are able to accurately detect the vulnerability status of the community members who live in their zones of intervention. The signing of performance contracts is in itself a good practice. These contracts should be results￾based. Courts and judicial institutions that do not comply with the terms of their performance contracts should be sanctioned accordingly. Recommendation. Future RoL programs should improve program design to ensure that performance￾based milestones are clearly identified and reinforced. The revitalization of the disciplinary chambers in the four target provinces is significant to both the GoDRC and its other development partners in building discipline and combating corruption on the bench. It is critical that any subsequent RoL program have a component to strengthen disciplinary chambers in the fight against corruption and thus increase the populations’ confidence in RoL. Recommendation. Revitalize the disciplinary chambers to fight against corruption in the justice sector. Lastly, the filing system promoted by the program in the pilot courts should be replicated in as many courts as possible. This system increases the transparency and efficiency of the courts and their services to citizens. Recommendation. Replicate the filing system in all future pilot courts to increase their performance. 4.10 WHAT MEASURES CAN FUTURE PROGRAMS TAKE TO INCREASE ACCESS TO JUSTICE FOR VULNERABLE POPULATIONS? On the basis of the results of the ProJustice Program and the findings of the evaluation, several factors can be identified as having facilitated or limited access to justice on the part of vulnerable populations. These factors are: • Lack of proper identification of all vulnerable groups in communities ` Final Evaluation: ProJustice Program in the DRC 45 • Ensuring the involvement of local organizations and Bar associations in the provision of legal support, provided they are already established within the communities • The cumbersome and drawn-out proceedings faced by court users • For all vulnerable people, the severe difficulty in receiving financial compensation when their cases are handled in the courts. Measure 1: Establish clear criteria of vulnerability to be adopted uniformly across all program areas. The certificate of indigence sometimes required in provinces such as South Kivu—although not required in other provinces—serves no real purpose and simply increases the amount of bureaucracy faced by citizens. In addition, to receive it, the user must pay $50, a very large sum for vulnerable people in these communities. Furthermore, in the DRC, the extremely poor and the vulnerable are largely composed of victims of sexual violence, young people, members of marginalized ethnic groups such as the Batwa, and those with physical or mental handicaps. All of these groups may be identified within communities by grassroots associations, without having to rely on a certificate of indigence. Future programs should focus on providing them with legal support when requested in the pilot courts, in an effort to increase vulnerable groups’ access to judicial services. Recommendation. A more inclusive vulnerability mapping should be done to identify all vulnerable in target communities. Measure 2: Continue to work with civil society and Bar associations in target areas to provide legal support to those who need it most. There are not enough program staff in all the target provinces to provide the necessary support to beneficiaries in the courts. Collaboration with CSOs working in the area of human rights and Bar associations at the provincial level has enabled the program to have a multiplier effect, and at a lower cost. Indeed, it is much more efficient to use community grassroots organizations already established within the target areas than to directly recruit lawyers to provide this support. In addition, working with community organizations builds their capacities and contributes to program sustainability, as these organizations remain on hand to provide legal services to vulnerable groups. Recommendation. Increase the work with CSO and their network of legal defenders. Measure 3: Establish criteria of performance for the pilot courts to monitor the collaboration between the courts and all future programs. This will enable the achievement of specific results such as the observance of time limits in the processing of cases of those systematically identified as vulnerable, such as victims of sexual violence, and children, for example. Over the medium to long term, the program should focus on the timeliness of the processing of cases in the pilot courts. Measure 4: Continue to lobby state authorities to ensure that people directly identified as vulnerable by the program through grassroots organizations (e.g., women victims of sexual violence, children, members of marginalized ethnic groups, handicapped people, etc.) are not required to provide 10% of the amount of damages accorded to them in advance in order to get assistance from the court in implementing judgments in their favor. This discourages many citizens from seeking justice, especially those who are most economically disadvantaged. Ultimately, even if they are in the right and the court recognizes this, they must first pay this money with no guarantee that they will recover it in the event that the condemned cannot pay, as is often the case. As a result, many vulnerable people, who are ` Final Evaluation: ProJustice Program in the DRC 46 economically or socially marginalized, are more likely to seek alternative ways to resolves their disputes, which may not always guarantee that their rights are observed. 5. RECOMMENDATIONS On the basis of analysis of the data, the following recommendations are proffered to inform future RoL programs in the DRC: Recommendation 1. Future RoL programs should provide support to the constitutional court. The future of the judicial sector in the DRC indeed depends on strong institutions. The constitutional court will be in charge of ensuring, among other things, that the democratic process in the country is fair and that the various reforms being introduced do not undermine the constitutional rights of the population. The next presidential elections38 in the country will probably involve new candidates only. It would therefore be imperative to have a strong constitutional court in order to safeguard the democratic process. This is critical to avoid losing the advances made on the democratic front since 2006, when the first presidential elections deemed free and fair were held. The support should mirror the assistance provided to the CSM. This is all the more necessary because this component of the program, while planned, was not implemented. Recommendation 2. Future trainings of judiciary staff should be undertaken through the EFPRJ to further strengthen it. This initiative will not only be a continuation of efforts started through ProJustice, but will also provide support without duplicating training vehicles. This approach will be more sustainable than the one used when the EFRPJ was still inoperative. This endeavor could also be an example of how the donor community could work with a host government body. Roles and responsibilities would be shared and the approach of using matching responsibilities could be followed in order to generate buy-in from the government. Trainings organized via the EFRPJ would make them more sustainable and ensure that new initiatives are in place to reinforce pre-existing ones. The constant development of new initiatives means that the risk of duplication exists. Savings can also be made by working with existing structures. Recommendation 3. The filing system promoted by the program in the pilot courts should be replicated in as many courts as possible. In fact, this system increases the transparency and efficiency of the courts and services to citizens. Recommendation 4. Improve program design to ensure that performance-based milestones are clearly identified and reinforced. The lack of effective disincentives or any significant consequence for poor performance created an environment of relative impunity within pilot jurisdictions of the ProJustice Program. The PACT process failed to produce “painful enough” disincentives to encourage counterparts to implement all aspects of the program, not simply those deemed most beneficial to the court. Structuring an effective method of incentivizing the courts is undoubtedly a challenge. Future programming should hold the counterparts accountable for their 38 According to the constitution, the current president may not run for reelection in 2016. ` Final Evaluation: ProJustice Program in the DRC 47 responsibility in implementing the program. The opportunity to replace poorly performing courts should be considered in any follow-on programming. Recommendation 5. Continue to make publically and readily available all court costs and fees at the tribunals. With over a third of respondents “undecided” in their opinion on using the courts again, an opportunity exists to build confidence in the judiciary and improve the standing of the legal institutions in the eyes of the average Congolese. ProJustice was able to improve the low levels of transparency within the court experience through material support and training. The Congolese public strongly desire a functioning justice system. Despite rampant corruption and lack of confidence in the system, citizens continue to use the court system largely because few viable alternative methods exist. Future programming should build upon the ProJustice efforts that increased transparency and continue to improve the court experience for the population at large that have not written off the option of using the courts in the future. Recommendation 6. Work with local organizations to strengthen their networks of legal defenders, who can provide legal support at a lower cost. Working with legal defenders was considered as good as the assistance from lawyers. Since the legal defenders reside in the communities and are more inclined to work for low (or no) wages with CSOs, it is important for future programs to try to strengthen the network of CSOs with legal defenders in order to give more sustainable legal support at a lower cost to participating communities. Recommendation 7. Facilitate the work of local organizations by giving them more time to achieve their objectives. Work with local CSOs should be on the basis of two-year sub-contract periods for any future RoL program. Working with local organizations is important. However, they do not always have the same level of organizational capacity and rigorous procedures as international organizations. To address this they should be assigned grants to plan, execute, and evaluate annually. A better performing organization should be granted two-year contracts with midterm reviews, when necessary. Recommendation 8. Future RoL programs should begin with a complete gender situation analysis and exploration of obstacles to justice for both women and men. This will ensure that program design is informed by the true factors that block access to justice for females and males and ensure that appropriate interventions are integrated and implemented. With regard to access to justice for women, it appears that the program had automatically placed all females seeking justice under the category of vulnerable person without systematically determining what prevented them from accessing the same recourse to justice as their male counterparts. Similarly, no specific information exist to guide the program to understand why some men did not have access to justice. It is important to conduct a study to identify barriers to access to justice for both genders and from there determine appropriate intervention activities and support for each beneficiary group. Recommendation 9. Establish criteria for vulnerability for the program to use to identify all vulnerable members of target communities. Vulnerability criteria defined by ProJustice did not allow for the identification of all vulnerable groups in the target communities. It is important to conduct a comprehensive analysis of vulnerability and the obstacles to accessing justice for vulnerable community members. This analysis must include a broader conceptualization and criteria for vulnerability. Recommendation 10. Legal awareness and legal aid should be increased to include a broader reach into the population, including legal awareness sessions for elected officials ` Final Evaluation: ProJustice Program in the DRC 48 and community members. This evaluation found that the Congolese population consistently correlated accessibility with awareness. The presence of imperfect information at the community level on the legal process specifically, and RoL in general, suggests benefits would be gained from increased legal awareness outreach efforts. The recommendation is twofold: FGD participants repeatedly suggested providing increased awareness trainings to their elected officials and the police as a means of increasing their view of transparency, and that increased awareness of their rights would empower the population to seek out justice. By propagating information publically and with greater scope, this would lessen the fear vulnerable populations reported as a key hindrance to seeking access to justice and would encourage greater accountability from judicial personnel. Religious organizations could provide a significant opportunity to increase public legal outreach efforts into communities as many Congolese are members of religious communities where outreach activities could be organized before or after church services. Recommendation 11. Lastly, the mixed justice committees should be revitalized for a better collaboration between RoL actors in the DRC. The mixed justice committees allow better coordination of activities in the country. These committees assembled all the actors of the justice sector to improve their coordination and limit inefficiency. It is important to maintain coordination with other actors to avoid duplication of effort and mismanagement of resources for future RoL programs. ` Final Evaluation: ProJustice Program in the DRC APPENDICES The following pages contain the eight appendices used to conduct the final assessment of the ProJustice program: • Appendix 1: Court user survey by province • Appendix 2: Household survey—demographic data • Appendix 3: Updated ProJustice M&E plan at the time of this final evaluation • Appendix 4: Evaluation of the scope of work • Appendix 5: Final evaluation plan • Appendix 6: Evaluation tools • Appendix 7: Household questionnaire • Appendix 8: Information sources and contacts. ` Final Evaluation: ProJustice Program in the DRC A-1 APPENDIX 1: COURT USER SURVEY BY PROVINCE Note: All numbers shown as percentages. Treatment Courts Control Courts Bandundu Katanga South Kivu Totals South Kivu yes no yes no yes no yes no yes no Getting to the courthouse was easy. 40 60 66 43 40 54 50 47 46 54 It was easy to get the information I needed when I came to the courthouse. 35 65 63 32 46 43 51 42 85 15 I received support from a civil society group before I came to the courthouse. 5 95 41 57 15 81 24 74 23 77 I was encouraged to come to the courthouse because of a pamphlet or billboard on I saw. 5 95 5 93 24 63 14 80 31 69 I felt comfortable coming to the courthouse because the court fees were clearly posted. 10 90 2 91 24 56 13 74 38 38 Court Personnel had all the records I needed to be seen by the judge. 80 10 66 16 44 38 58 26 54 23 Court personnel treated me with courtesy and respect. 65 20 64 36 59 26 62 29 85 8 The judge hearing my case listened to me and was courteous, respectful, and fair. 60 5 32 63 44 34 42 41 85 8 I understood the instructions of the judge/court and what I need to do next. 85 0 54 7 74 16 63 10 85 8 My case was handled promptly and in an efficient manner. 10 75 18 63 29 51 22 59 31 54 I feel I was treated fairly by the court. 45 25 25 61 24 35 27 44 54 30 I think the court performed impartially and effectively in my case. 35 45 25 55 21 38 24 46 54 30 Overall, I have confidence that the court is capable of accepting all Congolese for access to justice. 65 25 38 44 31 37 38 38 46 38 Observations: 20 56 68 144 13 ` Final Evaluation: ProJustice Program in the DRC A-2 APPENDIX 2: HOUSEHOLD SURVEY—DEMOGRAPHIC DATA Total Treatment Communities Control Communities Household Standing (%) Head of household 47.1 44.7 53.1 Spouse 36.0 35.2 38.0 Other 16.9 20.2 8.9 Gender (%) Male 53.2 53.7 52.0 Female 46.8 46.3 48.0 Average Age* Total Sample 38.27 37.94 39.11 (12.67) (12.49) (13.10) Male 38.73 38.11 40.33 (13.04) (12.55) (14.17) Female 37.74 37.75 37.73 (12.22) (12.46) (11.68) Literacy Rates (%) Total Sample 85.5 90.7 72.6 Male 91.5 94.1 85.0 Female 78.3 86.3 59.3 Level of Education (%) Basic Literacy 6.0 5.4 7.3 Primary 11.0 6.6 21.8 Secondary 47.3 51.3 37.4 University 23.2 27.9 11.7 No formal education 12.6 8.8 21.8 Principle Employment (%) Farmer 7.9 5.7 13.4 Shopkeeper 47.4 48.8 44.1 Civil Servant 18.4 19.5 15.6 Private Sector 13.6 13.8 12.9 Other 10.7 10.4 11.2 Unemployed 2.1 1.8 2.8 Observations by Province Bandundu 118 118 0 Katanga 221 171 50 South Kivu 281 152 129 * Standard deviations in parenthesis. ` Final Evaluation: ProJustice Program in the DRC A-3 Notes: G3—Those who sought justice through the courts* Total Sample Women's Sample Treatment Courts Control Courts Treatment Courts Control Courts What is your impression of the quality of service you received? Good 24.8 23.4 24.8 22.5 Acceptable 19.1 9.0 19.9 9.9 Poor 49,4 62.8 48.2 62.0 How would you rate the professionalism of the court officers in resolving your case? Good 19.1 19.3 17.0 14.1 Acceptable 28.7 20.7 30.5 21.1 Poor 46.5 49.0 44.7 52.1 How do you judge the speed of the court to resolve your case? Good 19.8 20.0 17.7 25.4 Acceptable 23.6 13.8 26.2 11.3 Poor 49.0 62.1 48.2 59.2 Do you think the court handles cases more quickly now than in the past? Yes 32.5 16.6 30.5 16.9 No 57.6 73.1 56.7 77.5 * All numbers shown as percentages. Notes: G3—Those who sought justice through the courts* Average Price ($) of Justice in DRC Treatment Communities Control Communities Observations Total Sample (entire population) 300.83 394.16 249 (493.57) (611.06) Total Sample (female only) 241.75 367.29 169 (297.57) (560.2) By province Bandundu 148.89 - 46 (164.16) - Katanga 276.54 225.38 127 (398.52) (301.75) South Kivu 395.74 442.38 184 (643.39) (667.48) * Standard deviations in parenthesis How respondents considered the price of justice Treatment Communities Control Communities Very expensive 62.13% 67.36% Acceptable 27.94% 8.33% Not expensive/cheap 8.46% 4.17% No opinion 16.91% 19.44% Observations 272 144 ` Final Evaluation: ProJustice Program in the DRC A-4 Treatment Courts Control Courts yes no undecided yes no undecided For individuals who used the court system at least once before, would they use the court system again in the future? Total Sample (entire population) 39 33 29 30 51 19 Total Sample (female only) 34 35 31 26 57 17 By province Bandundu 31 14 54 - - - Katanga 43 36 22 22 42 36 South Kivu 39 43 17 33 54 12 *All numbers shown as percentages. Notes: G4—Those who sought justice through the courts Treatment Communities Control Communities Percentage of respondents who say they had access to legal support while they were pursing their court case Total Sample (entire population) 47.45 27.59 Total Sample (female only) 40.4 28.2 By province Bandundu 87.04 - Katanga 51.49 31.3 South Kivu 26.19 26.55 Percentage of respondents who felt the quality of the legal support they received was acceptable or good Total Sample (entire population) 84.6 82.5 Total Sample (female only) 82.46 90.0 By province Bandundu 87.23 - Katanga 78.3 70.0 South Kivu 93.94 86.67 Percentage of respondents who are satisfied with the way their case has been handled by the court Total Sample (entire population) 45.22 34.48 Total Sample (female only) 43.26 32.39 By province Bandundu 74.07 - Katanga 39.55 28.13 South Kivu 38.89 36.28 Percentage of respondents who felt the court proceedings respected their rights Total Sample (entire population) 50 36.55 Total Sample (female only) 52.48 33.8 ` Final Evaluation: ProJustice Program in the DRC A-5 Notes: G4—Those who sought justice through the courts Treatment Communities Control Communities By province Bandundu 75.93 - Katanga 46.27 28.1 South Kivu 42.86 38.94 Percentage of respondents who at one time or another felt their rights were in jeopardy Total Sample (entire population) 59.87 61.38 Total Sample (female only) 60.28 54.93 By province Bandundu 42.59 - Katanga 61.94 62.5 South Kivu 65.08 61.06 Total possible observations for this section 314 145 Notes: G5—Those who chose not to use the courts Treatment Communities Control Communities If you had a problem, which could have been resolved, by the courts, why did you choose not to go to the court? The court is located too far away 5.51 8.82 The costs associated with going to court are too high 26.77 35.29 I do not have confidence in the justice system 40.94 44.12 Other 26.78 11.77 Note: All statements answered as a percentage in the affirmative. I was advised not to pursue justice through the court system. Total Sample (entire population) 45.67 47.06 Total Sample (female only) 47.62 40.0 By province Bandundu 43.75 - Katanga 45.95 55.56 South Kivu 45.24 37.50 I did not go to the court because I did not think I would have received a fair trial. Total Sample (entire population) 62.99 79.41 Total Sample (female only) 60.32 73.33 By province Bandundu 48.44 - Katanga 78.38 83.33 South Kivu 76.19 75.0 I did not use the court system because I was worried about the price I would have to pay for justice. Total Sample (entire population) 57.48 47.06 Total Sample (female only) 55.56 53.33 By province Bandundu 64.06 - ` Final Evaluation: ProJustice Program in the DRC A-6 Notes: G5—Those who chose not to use the courts Treatment Communities Control Communities Katanga 56.76 44.44 South Kivu 45.24 50.0 I estimate that it would cost me on average ____ to seek justice in my area. Total Sample (entire population) $248.55 $507.27 (184.30) (300.97) Total Sample (female only) $201.88 $570.00 (186.58) (305.37) I did not use the court system because there was an alternative method to resolving my problem. Total Sample (entire population) 82.68 82.35 Total Sample (female only) 84.13 73.33 By province Bandundu 79.69 - Katanga 78.38 88.89 South Kivu 88.1 75.0 I chose to use _____ to resolve my problem. Mediation 7.09 11.76 Arrangement 56.6 50.0 Traditional authorities 10.24 11.76 Other 26.07 26.48 When I had my problem, I tried to find someone to provide me with legal assistance. Total Sample (entire population) 19.69 11.76 Total Sample (female only) 22.22 13.33 By province Bandundu 18.75 - Katanga 32.43 11.11 South Kivu 3.85 12.5 I was aware I could get free legal support in my community. Total Sample (entire population) 28.35 5.9 Total Sample (female only) 34.92 6.7 By province Bandundu 40.63 - Katanga 18.92 0.0 South Kivu 11.54 12.5 My problem is now resolved. Total Sample (entire population) 66.14 64.71 Total Sample (female only) 68.25 46.67 By province Bandundu 75.0 - Katanga 45.95 77.78 South Kivu 64.29 50.0 While I did not use the court in my last problem, I would go to the court if my problem occurred again in the future ` Final Evaluation: ProJustice Program in the DRC A-7 Notes: G5—Those who chose not to use the courts Treatment Communities Control Communities Total Sample (entire population) 50.39 26.47 Total Sample (female only) 55.56 33.33 By province Bandundu 65.63 - Katanga 35.14 27.78 South Kivu 30.95 25.0 Total possible observations for this section 127 34 Standard deviations in parenthesis. Notes: G6—All household survey respondents Treatment Communities Control Communities Average distance in kilometers to the nearest tribunal to your home? 5.67 15.24 (10.56) (51.91) Note: All statements answered as a percentage in the affirmative. I have heard of mobile courts being held in my community. Total Sample (entire population) 29.25 13.41 Total Sample (female only) 29.41 10.47 By province Bandundu 46.61 - Katanga 24.56 2.0 South Kivu 21.05 17.83 I heard about the mobile courts through _____. The courthouse 41.04 48.15 An awareness session 11.19 3.70 A radio or television spot 31.34 18.5 Through some other means 16.43 29.65 Observations 134 27 Myself, or a family member, has used the services of a mobile court in my community. 53 individuals 18 individuals I think there are blockages in the justice system in my community. Total Sample (entire population) 72.56 75.42 Total Sample (female only) 73.53 74.42 By province Bandundu 48.31 - Katanga 77.19 70.0 South Kivu 86.18 77.52 Total possible observations for this section 441 179 *Standard deviations in parenthesis. ` Final Evaluation: ProJustice Program in the DRC A-8 APPENDIX 3: UPDATED PROJUSTICE M&E PLAN AT THE TIME OF THIS FINAL EVALUATION Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Objective 1. Support the Establishment of New Judicial Institutions Result 1.1 Ratification and implementation of new legislation for the CSM and Constitutional Court supported 1.1.1 Percentage of ProJustice project’s initiatives (as assessed through point system), which directly impact judicial independence efforts or support establishment of emerging judicial institutions. Determination of the percentage of ProJustice project’s initiatives that support the new judicial framework created by the Constitution of the Democratic Republic of Congo (DRC) and contribute to the establishment of new judicial institutions i.e. the CMS39 and Constitutional Court Type of initiatives Quarterly MoJ and CSM reports and achievements 0 % 60% 67% 70% 57% 75 % 160% 85% 40% 100% 50%  Data missing for FY09-11  Indicator 1.1.1. in this PMP is not the same as the indicator in the “final” PMP wording. Lack of a clear definition resulted in confused target and result figures  Unclear if program results meet program final target 1.1.2 Number of laws, regulations, and procedures related to judicial independence supported with USG assistance.40 Number of regulations and sub-regulations related to the CSM and Constitutional Court that are passed. This indicator refers to laws, regulations and procedures that are official and have been passed in accordance with the country’s legal requirements. Provide technical assistance to the CSM Disciplinary Board to help it implement Laws, regulations, and procedures Annually The Official Gazette of the DRC or CSM registry 0 2 1 3 3 4 1 6 2 8 7  Lack of a clear definition resulted in confused target and result figures; issue likely the result of cumulative vs. annual counting  Program results did not meet program final target 39 Conseil Supérieur de la Magistrature (High Judicial Council) 40 Indicators in bold are operational plan indicators. ` Final Evaluation: ProJustice Program in the DRC A-9 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 disciplinary proceedings for judges in a transparent manner, based on provisions in the Magistrates Statute of 2006 and the CSM Organic Law of 2008. Develop a nationwide case law handbook on disciplinary matters that includes case studies from all provinces. Develop a legal reference handbook containing all the laws and decisions related to the establishment of the Judicial Code [Legal Handbook, Volume 2]. Assist with the development of needed CSM regulations and secondary legislation. Assist with the development of needed Constitutional Court regulations and secondary legislation (such as internal regulations, regulations of the prosecutor’s office attached to this Court). ` Final Evaluation: ProJustice Program in the DRC A-10 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Result 1.2 Transparent, merit-based criteria for recruitment, selection, discipline, and promotion of magistrates adopted 1.2.1 Number of merit–based criteria or procedures for justice sector personnel selection adopted with USG assistance. Number of procedures or sets of criteria. Merit￾based selection criteria or procedures are based on educational background, experience, and related factors. Even if political criteria also come into play, the merit standards should set a minimum below which no contacts will help. Criteria or procedures could be assessed for Judges, Prosecutors, Police or Defense Attorneys. Organize a workshop on women’s involvement in the magistracy and develop an action plan to increase women’s participation in the justice sector. Provide technical assistance to the CSM and the Constitutional Court to design transparent and merit-based selection of 3 magistrates to be appointed to the Constitutional Court by the CSM. None Annually The Official Gazette of the DRC 041 4 3 5 1 6 1 7 1 8 6 Program results did not meet program final target 41 Neither the CSM nor the MoJ has so far developed merit-based criteria or procedures for the selection of new magistrates or justice sector personnel. ProJustice will continue to provide assistance to both institutions in that regard during year 12 and 13. ` Final Evaluation: ProJustice Program in the DRC A-11 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Result 1.4 Public and justice sector awareness of CSM’s process increased and new judicial selection and promotion process established 1.4.1 Number of USG-assisted campaigns and programs to enhance public understanding, NGO support, and media coverage of judicial independence and accountability. Number of campaigns funded and/or supported by ProJustice. Campaigns include all organized outreach activities intended to build support for the particular cause, relying, for example, on electronic media, print media, public meetings, plays, etc. Institution conducting the campaigns, programs and/or public awareness activities Quarterly ProJustice’s reports; NGO reports 0 1 4 3 24 5 499 350 392 355 1,095  Lack of a clear definition resulted in confused target and result figures  Unclear if program results meet program final target Result 1.5 CSM Established and functional with internal operating procedures and organizational structures 1.5.1 Number of regulations or procedures that improve judicial transparency adopted with USG assistance. Number of procedures, regulations, administrative reforms, management changes, or civil society initiatives enacted. Regulations or procedures that improve transparency include legal, regulatory, administrative, or management changes. Also included are civil society initiatives that enhance public access to legal and judicial processes or to information about those processes and their outcomes. a) Assist with the development of needed CSM regulations and secondary legislation. b) Assist with the development of needed N/A Annually CSM records 0 1 2 2 2 3 1 4 1 5 6 Program results exceeded program final target ` Final Evaluation: ProJustice Program in the DRC A-12 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Constitutional Court regulations and secondary legislation (such as internal regulations, regulations of the prosecutor’s office attached to this Court). Objective 2. More Effective and Transparent Management of the Judiciary and MoJ Result 2.1 Management skills among magistrates, CSM staff, and MoJ staff enhanced 2.1.1 Mechanisms established by the MoJ and CSM support a more effective and transparent management of the judicial system. Number of administrative and/or budgetary systems developed and implemented by the MoJ and CSM to improve the management of the justice system. Organize a workshop on women’s involvement in the magistracy and develop an action plan to increase women’s participation in the justice sector. Provide technical assistance to the CSM Disciplinary Board to help it implement disciplinary proceedings for judges in a transparent manner, based on provisions in the Magistrates Statute of 2006 and the CSM Organic Law of 2008. Develop a nationwide case law handbook on disciplinary matters that Type of systems, e.g., administrative, budgetary, ethical, recruitment career, promotion, and training systems. Quarterly MoJ and CSM reports 0 3 0 4 4 5 2 7 3 9 10  Data records do not substantiate the reported achieved targets  Program results exceeded program targets.  dTS was only able to verify 8 mechanisms and thus program results did not meet program targets ` Final Evaluation: ProJustice Program in the DRC A-13 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 includes case studies from all provinces. Provide financial and technical support to help the CSM organize its 2012 General Assembly. Organize a workshop on capacity-building for the members of the Constitutional Court. Result 2.2 Budget and resource management by the CSM and MoJ improved 2.2.1 Judicial budget adopted and implemented through transparent procedures and reflecting real expenditures of the justice system. The budget refers to the document drafted by judicial authorities with ProJustice’s technical assistance. A numeric scale for measuring progress will include the following scores: 1. Development of an inventory of pilot courts’ resources and needs (1 point); 2. Elaboration of a forecasted budget for the judiciary (1 point); 3. Budget is presented to the Government and Parliament (2 points); 4. Budget is passed (1 point); 5. Systems are in place in the CSM and the Ministry of Finance for the N/A Annually Government of the DRC’s budget and ProJustice’s files 0 1 point 4 4 points 1 9 points 6 12 points 2 15 points 13  Poorly worded custom indicator combined with a lack of a clear definition resulted in confused reporting, target and result figures  Issue with cumulative vs. annual counting  Program results did not meet program final target ` Final Evaluation: ProJustice Program in the DRC A-14 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 implementation of the budget; and actions are taken by the CSM and Ministry of Finance to implement the budget (4 points). a) Establish and support a working group responsible for managing the judiciary budgeting process. b) Organize a training workshop for the CSM and MoJ on financial management and strategic planning. c) Train the members of the Constitutional Court on financial management and strategic planning. d) Assist the Constitutional Court in preparing budget forecasts. e) Help set up mechanisms to develop transparent budget ` Final Evaluation: ProJustice Program in the DRC A-15 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 execution between CSM and MoJ. 2.2.2 Number of justice sector personnel that received USG training. Number of judges, CSM, MoJ staff, and justice personnel trained with ProJustice support. Personnel include judges, magistrates, prosecutors, advocates, inspectors, and court staff. Training refers to all training or education events whether short-term or long-term, in country or abroad. Gender - Male/Female Quarterly Training records from training providers (ProJustice project, law schools, training schools) 0 200 194 Male 160 177 Female 40 17 300 208 Male 240 170 Female 60 38 500 123 Male 400 104 Female 100 19 650 306 Male 515 223 Female 135 83 800 1,097 Male 640 875 Female 160 290  Data records do not substantiate the reported achieved targets  According to ProJustice the program results exceeded program targets.  dTS was only unable to verify 2013 figures, which see a disproportionate jump from the prior four years’ activities and determine the results to be suspect Objective 3. More Effective, Transparent, and Accessible Court Operations in Pilot Jurisdictions Result 3.4 Improvements to court operations realized through PACT planning process 3.4.1 Number of USG-assisted courts with improved case management. Number of courts with improved case management systems implemented by ProJustice.42 Pilot courts - Court of Appeals, First Instance Court, Peace Court Annually Case management and registry’s records in pilot courts 0 12 0 12 9 12 10 12 11 12 11  Program results did not meet program final target 42 Examples of improvement include: case documents are available to parties upon request; statistical data on cases is routinely compiled; system manages flow of cases through scheduling set by procedural law; data elements are coherent and compatible with related information systems such as those of the police, prosecution, and corrections agencies; cases are uniquely identified, registered, indexed, and filed. ` Final Evaluation: ProJustice Program in the DRC A-16 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Result 3.5 Accessibility increased by new policies and through improved court performance 3.5.1 Ratio of new case filings to case dispositions in USG￾assisted courts in the area of case management. Calculation of the annual ratio of new case filings to case dispositions in courts assisted by USG in the area of case management. Courts as defined by specific jurisdiction. The ratio of cases filed to cases dealt with during a judicial year will be calculated in pilot courts assisted by ProJustice. Pilot courts Judicial year Pilot courts’ registries TBD43 1%44 5% 10% 15% 20%  Program indicators 3.5.1, 3.5.2, and 3.5.3 were reported on as a separate document from the core PMP.  All three indicators are standardized USG F-Indicators although exact program definition of each indicator is unclear.  All three indicators’ program results exceeded program targets 3.5.2 Ratio of new case filings to case dispositions in USG￾assisted courts and in courts not assisted by the USG (Control group). Comparison of the ratio of cases files to cases disposed in a Court of Appeals, First Instance Court, and Peace Court where no court management system has been implemented. N/A Judicial year Pilot courts’ registries TBD TBD _ _ _ _  Program indicators 3.5.1, 3.5.2, and 3.5.3 were reported on as a separate document from the core PMP.  All three indicators are standardized USG F-Indicators although exact program definition of each indicator is unclear.  All three indicators’ program results exceeded program targets 3.5.3 Mean case disposition time in courts assisted by USG in the area of case management. Determination of the number of months from fillings to dispositions. Median case disposition time is measured from Pilot courts Judicial year Pilot courts’ registries TBD45 -5%46 -10% -20% -25% -30%  Program indicators 3.5.1, 3.5.2, and 3.5.3 were reported on as a separate document from the core PMP. 43 Ratio to be established in an assessment that will precede the implementation of the case management system in ProJustice’s pilot courts. 44 After the determination of the baseline, ProJustice will reduce by 1, 5, and 10 percent the ratio of cases filed and dealt with in the project’s pilot courts. 45 Baseline will be determined during an assessment on the average time from filing to disposition of cases in courts assisted by ProJustice. 46 Once baseline is established, ProJustice will reduced by 5, 10, and 20% the average time necessary to deal with a case in courts assisted by the project. ` Final Evaluation: ProJustice Program in the DRC A-17 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 filing to the date parties are notified of case resolution by the court. Courts as defined by specific jurisdiction.  All three indicators are standardized USG F-Indicators although exact program definition of each indicator is unclear.  All three indicators’ program results exceeded program targets Result 3.6 Mobile court system expanded and systematized 3.6.1 Number of courts operating in areas of low income populations with USG assistance. Number of mobile courts operating in low income areas with ProJustice assistance. Areas of low income should be those where 60% of the population has an income in the lowest quintile of the country as a whole. Pilot jurisdictions Quarterly ProJustice project’s records 0 0 11 8 12 12 11 12 11 12 11 Program results did not meet program final target Objective 4. Increase Access to Justice for Vulnerable Populations Result 4.1 Capacity of CSOs to provide access to justice services increased 4.1.1 Percentage of vulnerable populations in the project’s targeted sites/ jurisdictions with improved knowledge to access the judicial system. Determination of the percentage via a survey carried out on a representative sample of vulnerable population in ProJustice’s target sites or populations. Gender,47 ethnic group, and province Annually Survey to be carried out by the ProJustice project TBD48 36.5% +20%49 36.5% + 30% 53% + 50% 59.5% +55% 65.0% +60% 76.5%  Annual survey data not readily available for verification  Program results do not meet program targets 47 Gender target in this case is 50% female. 48 Baseline will be determined through a survey of populations in the project’s targeted sites/jurisdictions. 49 For Years 1, 2, and 3, ProJustice will increase by 20, 30, and 50 percent the percentage of vulnerable populations, as revealed by a baseline survey, who know how to access the legal system. ` Final Evaluation: ProJustice Program in the DRC A-18 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Result 4.2 Grants provided to CSOs to help increase activities and impact in justice sector 4.2.1 Percentage of ProJustice’s grantees that develop activities to increase information, access, or availability of justice services for vulnerable populations. Determination of the percentage of beneficiaries of the grants programs that successfully increase access to justice services via the implementation of legal assistance, awareness campaigns, training and other legal aid activities. Regional locations Quarterly ProJustice grants programs’ records 0 60% 75% 65% 100% 70% 100% 100% 100% N/A Program results equal program targets 4.2.2: Number of domestic human rights NGOs receiving USG support. Number of Congolese domestic human rights NGOs focusing on human rights that receive USG assistance via ProJustice.50 Location and areas of interventions of grantees Quarterly ProJustice project’s reports 0 8 11 10 76 12 71 57 33 N/A51 214  USG F-Indicator, which has been reworded to reflect custom indicator. Lack of a clear definition resulted in confused reporting, target and result figures  Program results exceeded program final target 4.2.3: Number of individuals/groups who receive legal aid or victims assistance with USG support. Number of individuals or groups assisted. Legal aid or victim’s assistance includes legal consultation, advice, representation, or mediation services for poor, marginalized, or other vulnerable groups Gender – Male/Female Quarterly ProJustice’s records 0 60 263 Male 30 Female 30 150 1,201 Male 75 Female 75 300 879 Male 150 Female 150 900 264 Male 450 Female 450 N/A52 2,626 50 To be considered human rights NGO, the organization should be focused on strengthening human rights, e.g., in providing services, reporting, advocacy, outreach, education, or protection of citizens. USG assistance includes training, grants, or other support designed to improve the human rights services, reporting, and advocacy for citizens. Training measured is the assistance provided with USG assistance, whether short-term or long-term, in country or abroad. 51 There will be no Grant program during the final fiscal year of the project. 52 There will be no Grant program during the final fiscal year of the project. ` Final Evaluation: ProJustice Program in the DRC A-19 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 who have difficulty paying for or accessing legal services. Legal aid is often combined with referrals or assistance accessing other types of support, including health and psycho-social counseling. Assistance can be provided through state￾sponsored public defenders, bar associations, NGOs, semi￾private, or private agencies, or law students through legal clinics or internship programs.  USG F-Indicator, which has been reworded to reflect custom indicator. Lack of a clear definition resulted in confused reporting, target and result figures  Tetra Tech DPK rewording created opportunity for higher reporting in results  Program results likely exceeded program final target 4.2.4: Number of USG-assisted legal aid groups and law clinics. Number of legal aid groups and law clinics assisted by ProJustice. Legal aid groups means NGOs or other private or private/state mixed entities existing to provide legal advice to citizens. Law clinics means programs established by the bar and law schools where students, under close supervision, give legal advice to citizens. N/A Quarterly ProJustice’s grant records 0 2 0 4 12 5 8 11 8 N/A 28  USG F-Indicator, which has been reworded to reflect custom indicator. Lack of a clear definition raised issues of double counting and confusion in reporting, target and result figures  Program results likely exceeded program final target ` Final Evaluation: ProJustice Program in the DRC A-20 Performance Indicators Unit of Measurement and Definition Disaggregated by Frequency of Collection Data Source Baseline (2009) Target Year 1 2 3 4 5 Objective 5. Take Advantage of Windows of Opportunity to Advance Judicial Reform Result 5.1 Windows of Opportunity are identified and transformed into activities promoting justice sector reforms 5.1.1 Number of activities undertaken to take advantage of opportunities to promote the reform/institutional change agenda. Number of previously unplanned activities implemented to promote judicial reform.53 Theme Annually ProJustice project’s reports. 0 2 1 3 3 4 1 5 2 6 13  Evaluation team questions the relevance and value of an indicator which measures the hypothetical realm of program implementation  Program results likely exceeded program final target, although the impact of this indicator remains suspect 53 Depending on USAID’s request or approval. ` Final Evaluation: ProJustice Program in the DRC A-21 APPENDIX 4: EVALUATION OF THE SCOPE OF WORK INTRODUCTION USAID/DRC wishes to use the Washington‐based Evaluation Indefinite Quantity Contract (IQC) to undertake an external final performance evaluation of its RoL program: ProJustice, implemented by Tetra Tech DPK. USAID/DRC is imposing a tight deadline of August 12, 2013 for completion of the evaluation final report to enable consideration of the findings, conclusions and recommendations prior to the start of fiscal year 2014. I. BACKGROUND A. Country Context Following a protracted civil war and economic decline, the DRC held national elections in 2006 that resulted in its first democratically elected government since independence in 1960. While the 2006 Constitution provides a foundation for improved democracy and governance, many political practices have yet to change substantially. A 2008 DG Assessment of the DRC described the social, economic, and political situation as “extremely challenging”, a sentiment echoed in the 2012 DG Assessment. Corruption remains endemic, RoL is not sufficiently respected, and a pervasive culture of impunity undermines the legitimacy and credibility of the government. Transparency International ranked the DRC 160th out of 176 countries surveyed in its 2012 Corruption Perception Index. The DRC was ranked 181 (fifth‐to‐last) in the World Bank’s “Doing Business” 2012 report, describing it as one of the most difficult places in the world to do business. Freedom House noted a continued lackluster country performance in its 2012 “Freedom in the World” report, where the DRC retained a score of ‘6’ in the survey’s political rights and civil liberties categories (the comparative assessment uses a seven‐point rating scale, with ‘7’ representing the least free status). The judicial system is both underfunded and poorly staffed, with only a limited presence outside Kinshasa. The physical infrastructure of the judicial system is inadequate, with crumbling buildings (or no buildings at all) and an information system that fails even to make copies of existing laws available to judges or court staff. In rural areas, where courts are often absent, justice is frequently administered on an ad hoc basis, by whichever authority is available, creating extraordinary opportunities for corruption and abuse of power. The absence of functioning courts contributes to a culture of impunity that allows not only ‘official’ corruption but also widespread human rights abuses, including sexual violence, arbitrary violence and extortion by security forces, and abuse of office by public officials. Prison conditions are deplorable, and the inefficiency of the justice system keeps the accused incarcerated for long periods of time. The judicial branch remains weak, and does not represent an independent branch of government capable of providing checks on the powers of the legislative and executive branches. The leadership of the judicial sector, however, is committed to reforming and strengthening the country’s court system. Building up the judiciary not only supports the delivery of judicial services to citizens, but also could provide important government oversight and foster a more effective balance of power at all levels. Throughout the last five years, ProJustice has operated within the dynamic environment of evolving ` Final Evaluation: ProJustice Program in the DRC A-22 Donor priorities, system constraints, and a lack coherent direction from the GoDRC, which is supposed to lead coordination amongst donors. These represent only some of the challenges that the program faced during implementation. B. Program Identification The ProJustice program was funded in response to USAID’s 2009 Country Assistance Strategy, the overall goal is to support the security conditions and governance structures necessary for the improvement of Congolese social and economic sectors, and to permit extension of state authority across the country. Under its governing justly and democratically objective, USAID determined the need to promote and strengthen the justice sector in order to achieve stability and support a functional system of checks and balances between different branches of the DRC Government. The program was implemented through a task order issued under the Regional Democracy Initiatives (REDI)‐Justice IQC managed by USAID/Egypt. The 2006 justice sector assessment played a large role in informing the design of this program, and highlighted the need to support the establishment of an independent judiciary and expansion of access to justice in targeted provinces. Given the importance that the justice sector can play in the restoration of state authority and legitimacy, the program received supplemental funding to operate in post‐conflict area such as South Kivu and Maniema. This program was intended to be a comprehensive long‐term follow‐on to emergency RoL programs that were implemented between 2006 and 2008 by Global Right and Avocat Sans Frontière (these focused mostly on addressing human right issues and increasing access to justice). Unlike these two emergency programs, ProJustice was designed to address the systemic, rather than symptomatic, issues that the judiciary system faces, while accompanying reform initiated by the GDRC as delineated in the 2006 constitution. In light of this strategic purpose, ProJustice supported the new established CSM, and targeted courts to improve the managerial and administrative aspects of justice provision, while also working with civil society organizations to support increased access to justice. 2. EVALUATION RATIONALE AND KEY QUESTIONS A. Purpose and Use of This Evaluation This evaluation will be reviewed by the Mission’s DG team as well as by Mission Management. The timing of this evaluation is propitious for providing a more informed base on which the Mission can begin to strategize, prioritize and design future assistance programs for the DRC. The objectives of this evaluation include assessing: (a) the assumptions and analysis upon which the program was designed and (b) the performance of the program. The evaluation will assist USAID/DRC/DG to: • Learn lessons from current engagement in the justice sector • Consider options for expanding RoL programs under USAID/DRC’s upcoming new Mission‐wide strategy • Determine whether USAID should consider an extension beyond the current projected end dates and the objectives and activities that any cost‐extension might include. This evaluation will determine whether the assistance met the stated objectives and provide a detailed picture of the major accomplishments/weakness of the program, indicating what results are most likely to achieve sustainability. ` Final Evaluation: ProJustice Program in the DRC A-23 The evaluation will also provide important feedback to the implementing partner that should assist them understand both the strengths and areas where technical, administrative and management efforts could be improved. In addition the evaluation will serve: • To identify any implementation problems, unmet needs or unintended consequences or impacts, especially taking into account any unanticipated changes in the host country environment. • To provide a better understanding of the progress made by the program on issues including impact, relevance, sustainability, and cost‐effectiveness (e.g. cost per beneficiary). • To confirm the validity of the overall development hypotheses underlying the design of this program. The nature of this evaluation is in line with the USAID evaluation policy as laid out in the Automated Directive System (ADS) 203.3.1.3 which requires that an evaluation be conducted for each large project implemented. In addition, the Mission is currently engaged in a strategic planning process, this performance evaluation may help inform this process, and contribute to USAID’s understanding of the democracy and governance sector more broadly (complementing recent DRG assessment findings). The evaluation report will be shared with GDRC counterparts and other stakeholders (including GDRC, CSM, MoJ, local and provincial judicial staff, and other donor organizations) in order to disseminate lessons learned from the approach and implementation of this program. B. Evaluation Questions This evaluation will assist the Mission by providing information that responds to the following questions: Development hypothesis employed: • Have stronger civil society and more informed citizens generated greater demands on political leaders and justice system actors and ensured that the judicial system is both accessible and responsive? • Has the judiciary become better managed and resourced, and has accessibility increased at the local level? Has the standardization and reduction of fees occurred? Have these developments made the judicial process more transparent and available to external oversight, and expanded the reach of the justice system? Achievement of intended results: • Have efforts to support new judicial institutions created more effective and more independent judicial institutions? • To what extent is the management of the Ministry of Justice and the Judiciary more effective and transparent? • Are court operations in pilot areas more effective, transparent and/or accessible? • Has access to justice for vulnerable populations changed? Methodology/development approach: • How did the development approach employed by the program (PACT methodology) impact the achievement of intended results in each program component? ` Final Evaluation: ProJustice Program in the DRC A-24 Lessons learned/unintended consequences: • Where did implementation fall short of achieving intended results and what factors constrained implementation from achieving greater progress towards targeted results? • Over the course of implementing the program, what ‘major lessons’ emerged? • Based on the results the program was able to achieve, in which areas should USAID assistance be targeted in the future? • Should USAID consider a cost extension of the program? If so, what specific aspects of the program warrant continued focus? • Did implementation result in unintended consequences? • What additional analysis and/or changes to the program would have improved the focus, targeting and quality of activities? 3. TEAM COMPOSITION For planning purposes the team for this evaluation will consist of up to three senior‐level consultants and a mix of consultants who will provide varying technical (subject matter) expertise to the team, and support staff. However, USAID encourages prospective contractors to propose alternative staffing scenarios. The required areas of technical (subject matter) expertise that should be represented on the team are detailed below (following are experiences and knowledge expected from experts): • Team Leader (key personnel): This senior level expert should demonstrate an excellent theoretical and practical background in conducting evaluations. A strong background in the evaluation, design or management of development programs supporting justice systems similar to the Congolese justice system would be suggested. Specific expertise in court management and policy making and capacity-building would be desired. • Mid‐level consultant: Should demonstrate an excellent theoretical and practical background in the evaluation, design or management of development programs supporting justice systems. Specific expertise in court management and policy making and capacity-building would be desired • Local expert consultant: Significant experience in working with or in support of the Congolese justice system. He/she should have significant experience or knowledge of capacity-building approaches and other development methodologies employed in the DRC. • Data Collectors: Should have in depth knowledge of the field. It is recognized that having a Team Leader with experience with USAID is often an important factor, particularly for evaluations designed to inform future programming, but proposals will be reviewed taking into account the overall strengths of the assembled team’s senior‐level technical expertise, evaluation expertise, and the expertise and ability of the team’s leadership to manage budget and staff resources. The Team Leader will assume the responsibility as the Point of Contact between the team and the USAID Mission. ` Final Evaluation: ProJustice Program in the DRC A-25 APPENDIX 5: FINAL EVALUATION PLAN PHASE ACTIVITY RESPONSIBLE PARTY(S) TIME SCHEDULE Award Finalize team arrangements and task order kick-off call with USAID/DRC PMT, Evaluation TL Within 3 days of award Phase 1: Preparation Request/gather background documents from USAID/DRC and ProJustice PMT, Evaluation TL Week 1 Travel to Kinshasa, team meetings, team building PMT, Evaluation team Week 1 Initial meetings with USAID/DRC, ProJustice, and Stakeholders Evaluation team, Week 1 Review background documents Evaluation team Week 1 Prepare/pilot interview guides, survey instruments, data collection protocols Evaluation team Week 1 Prepare & submit final work plan Evaluation team Week 1 Arrange logistics and schedule for field work Evaluation team Week 1 Phase 2: Field Data Collection and Synthesis Additional Desk review Evaluation team Week 2 Stakeholder interviews (including field missions) Evaluation team Weeks 2, 3, 4 Interviews and data collection with ProJustice Staff, beneficiaries and other stakeholders. Evaluation team Weeks 2, 3, 4 Data Collation and Analysis Evaluation team Weeks 2, 3, 4 Validation workshop(s) with USAID/DRC, GoDRC, ProJustice, Evaluation team Week 4 Out-brief with USAID/DRC and presentation of preliminary findings and recommendations Evaluation team Week 4 Team departs DRC Evaluation team End of Week 4 Phase 3: Draft and Final Reporting Develop and submit draft final report Evaluation team, PMT Week 8 USAID provides comments on draft final report USAID/DRC Week 9 Respond to USAID comments on draft report; submit final report Evaluation team, PMT Week 11 ` Final Evaluation: ProJustice Program in the DRC A-26 APPENDIX 6: EVALUATION TOOLS Members of CSM Interview Questions 1. Did you establish the Needs Assessment Committee to run a diagnosis of the judicial system and changes needed? 2. What were the strengths and weaknesses of the process? What original input if any did the committee bring to your work practice? 3. What are the changes you recently introduced in the judicial system and do you think that these will have lasting presence or that they will not outlive the project? Explain why. 4. Do you think that the work done by the committee will have long lasting effect or will it not? What should be done to improve on the sustainability of these changes? 5. Do you have any tool to track the effect of your work on the performance of magistrate or on the public opinion? If so, how are you using it? 6. Did the CSM Committee do any baseline assessment and make recommendations on the management and running of HR, budget and finances? 7. If the Committee did provide some recommendations, were they followed or considered in the implementation? 8. Do you think there is going to be a long lasting effect if any and why? 9. Do you know if the level of self-generated resources inside the judicial system is greater or not as important when compared to the past? If there is a change, do you know the reason why? Could you explain? 10. How do you associate/not associate the DFID (UK Department For International Development) and USAID programs that both targeted the strengthening of the judicial system? Explain with actual examples or facts. 11. Could you list any important practices or procedures that were identified as necessary to improve the management practice in provincial courts as a result of the USAID project? 12. Have you noted an enhancement in the court management practice and transparency? 13. If yes, could you give examples or comparisons to the past? Also do you think that such changes could have long lasting effects or that they will not. What could be done to improve the situation? 14. Could you list any important practices or procedures that were identified as necessary to improve court accessibility for vulnerable people, as a result of the USAID project? 15. What would you say about the progress or lack of progress within the CSM of supporting vulnerable citizens? 16. What elements helped you proceed with your analysis? ` Final Evaluation: ProJustice Program in the DRC A-27 Kinshasa-based Magistrates’ Interview Questions 1. What procedures have you personally been introduced to as a result of your ProJustice trainings? 2. Do you feel that ProJustice activities increased the capacity of the Congolese judiciary to improve (1) performance, (2) accountability and (3) public confidence in the rule of law? 3. Did you receive any new or continuing legal education training from ProJustice staff in the past 5 years? Was this training effective in providing you with new information about how to implement the laws of the RDC? 4. Has your ability to implement the law in a fair and impartial manner improved since you received your training? How? 5. Did you visit another country’s justice system with ProJustice staff? Was the trip useful to your continuing legal education? Why or why not? 6. What additional information would you have liked to learn or achieve while visiting another country’s judicial system? What would you recommend to ProJustice to improve their legal exchange program? 7. Is there anything you now do differently at the CSM because of your experience with the ProJustice program? 8. Do you think that becoming a magistrate is a fair process and what changes have you noticed in the appointment of magistrates in the past 5 years? 9. If there have been changes, do you think these changes will remain 10 years from now? If not, what should be done in order to ensure that changes have lasting effects? 10. Have your organizational and finance/budgetary capacities been improved or reinforced through the ProJustice project? Has this been useful to you and/or your staff? 11. Before your participation in the ProJustice project, was the management of the court’s budget better or worse? How and why? 12. What changes have you noticed in the way magistrates run the administration and caseloads of the courthouse? Do you believe this will continue into the future? 13. Does the judicial system have more or less resources available since the start of the ProJustice project? Do you think it is because of advocacy by the ProJustice project and do you believe that these resources will remain available 5 years from now? 14. What suggestions would you make to increase resource availability for the Congolese judiciary? 15. Were there any new practices or procedures that were introduced to improve the management practices in the provincial courts as a result of the ProJustice project? 16. Could you describe the ones you feel are most important to improving the judicial system in the provinces? 17. Have you noted an improvement in how the court manages cases? If yes, could you give examples or comparisons of this improvement? ` Final Evaluation: ProJustice Program in the DRC A-28 18. Do you feel that the court management changes will have long lasting effects? If not, what do you feel would be necessary to permanently improve the management of the courts? 19. Are there particular measures, actions or activities, which you personally are aware of that, were implemented by ProJustice to support the cause of vulnerable people? 20. In your opinion, did these implemented measures manage to bring changes in the performance and accountability of the judicial system for vulnerable people? 21. What activities or policies do you think would improve the overall court experience for women and children attending the courts in Congo? Provincial Magistrates’ Interview Questions 1. What procedures have you personally been introduced to as a result of your ProJustice trainings? 2. Do you feel that ProJustice activities increased the capacity of the Congolese judiciary to improve (1) performance, (2) accountability and (3) public confidence in the rule of law? 3. Did you receive any new or continuing legal education training from ProJustice staff in the past 5 years? Was this training effective in providing you with new information about how to implement the laws of the RDC? 4. Has your ability to implement the law in a fair and impartial manner improved since you received your training? How? 5. Did you visit another country’s justice system with ProJustice staff? Was the trip useful to your continuing legal education? Why or why not? 6. What additional information would you have liked to learn or achieve while visit another country’s judicial system? What would you recommend to ProJustice to improve their legal exchange program? 7. Is there anything you do differently in the courthouse because of your experience with the ProJustice program? 8. Do you think that becoming a magistrate is a fair process and what changes have you noticed in the appointment of magistrates in the past 5 years? 9. If there have been changes, do you think these changes will remain 10 years from now? If not, what should be done in order to ensure that changes have lasting effects? 10. Have your organizational and finance/budgetary capacities been improved or reinforced through the ProJustice project? Has this been useful to you and/or your staff? 11. Before your participation in the ProJustice project, was the management of the court’s budget better or worse? Why? 12. What changes have you noticed in the way your court management staff runs the administration and caseloads of the courthouse? Do you believe this will continue into the future? ` Final Evaluation: ProJustice Program in the DRC A-29 13. Does the judicial system have more or less resources available since the start of the ProJustice project? Do you think it is because of advocacy by the ProJustice project and do you believe that these resources will remain available 5 years from now? 14. What suggestions would you make to increase resource availability for the Congolese judiciary? 15. Were there any new practices or procedures that were introduced to improve the management practices in the provincial courts as a result of the ProJustice project? 16. Could you describe the ones you feel are most important to improving the judicial system in the provinces? 17. Have you noted an improvement in how the court manages cases? If yes, could you give examples or comparisons of this improvement? 18. Do you feel that the court management changes will have long lasting effects? If not, what do you feel would be necessary to permanently improve the management of the courts? 19. Are there particular measures, actions or activities, which you personally are aware of that were implemented by ProJustice to support the cause of vulnerable people? 20. In your opinion did these implemented measures manage to bring changes in the performance and accountability of the judicial system for vulnerable people? 21. What activities or policies do you think would improve the overall court experience for women and children attending the Congo? Civil Society Representative Interview Questions 1. What kind of support did you received from the project? If yes, did you participate in the drafting of the project? Did this plan reinforce your capacities? 2. Has your ability to support the judicial needs of vulnerable people been reinforced by the support you received from ProJustice? 3. Do you think that you will be able to continue to provide your services at your current level in 5 years? 4. What actions have you noted that were undertaken by either the courts or yourself, which has simplified and shortened the court proceedings in the past 5 years? 5. If any, please explain. 6. Did the time between filing and disposition change as compared to the past improved, gotten worse of stayed the same? Do you know why this happened? If the answer is “gotten worse” or “stayed the same”, what are the bottlenecks you experienced and could you provide suggestions to improve the situation? 7. Do you think the court system is more responsive to cases brought by the population now as compared to the past? Does this have anything to do with the ProJustice project? Are there other factors (projects), which you feel could explain this? ` Final Evaluation: ProJustice Program in the DRC A-30 8. Do you feel there are other ways the judicial system could be made more responsive today and in the future? Please explain. 9. Have you witnessed any new efforts by the court system to encourage the public to use the judicial system? Are there any kiosks, public outreach materials, or mobile courts outside provincial capitals that have been undertaken? 10. Have you seen the court system implement provisions of law that gives the possibility to vulnerable population to bring sensitive cases before courts? Could you provide personal experience? 11. Do you think that court accessibility has changed for vulnerable population or it has remained the same? Please provide explanations and examples. 12. Is there more awareness among vulnerable people that there is a provision in the law that gives them the possibility to bring sensitive cases before courts? 13. Do you think that the ProJustice project had any impact in increasing awareness and access to justice for women and children? Could you provide an example? Community Focus Group Discussions 1. Have there been any actions, which you have seen that were undertaken to simplify and shorten court proceedings in the past 5 years in your area? If any, please explain. 2. Did the time between filing and disposition change as compared to the past 5 years or is it the same? If the answer is "YES", do you know why this happened? If the answer is "No", what are the bottlenecks and could you give suggestions to improve the situation? 3. Do you think that the court system is responsive enough to cases brought by the population now as compared to the past 5 years? 4. What are the reasons behind this? 5. If the system is more responsive today, is there any other way it could be made even better? Please explain. 6. Have you see any new types of efforts by the court institutions to encourage the public to use the judicial system? 7. Were there or are there any information points, public outreach initiatives, or courts hearing, which you were aware of in the past 5 years? Please provide examples. 8. Have you seen the court institutions implement provisions of law that gives the possibility to vulnerable population to bring sensitive cases before courts? Could you provide some examples? 9. Do you think that court accessibility has changed for vulnerable population or has it remained the same over the past five years? Please provide explanations and examples. 10. Is there more awareness among vulnerable people that there are provisions in the law, which gives them the possibility to bring sensitive cases before courts? ` Final Evaluation: ProJustice Program in the DRC A-31 Court User Satisfaction Survey The following brief questions will be asked of individuals leaving the courthouse. They may choose one of the following responses: (1) I agree with the statement, (2) I disagree with the statement, or (3) this statement is not applicable to my visit to the courthouse today. 1. Getting to the courthouse was easy. 2. Finding where I had to go in the courthouse was easy and convenient. 3. It was easy to get the information I needed when I came to the courthouse. 4. I received support from a civil society group before I came to the courthouse. 5. I was encouraged to come to the courthouse because of a pamphlet or billboard on I saw. 6. Court personnel had all the records I needed to be seen by the judge. 7. Court personnel treated me with courtesy and respect. 8. The judge hearing my case listened to me and was courteous, respectful, and fair. 9. I understood the instructions of the judge/court and what I need to do next. 10. My case was handled promptly and in an efficient manner. 11. I feel I was treated fairly by the court. 12. I think the court performed impartially and effectively in my case. 13. Overall, I have confidence that the court is capable of accepting all Congolese for access to justice. ` Final Evaluation: ProJustice Program in the DRC A-32 APPENDIX 7: HOUSEHOLD QUESTIONNAIRE Bonjour. Je m’appelle ..........Je travaille pour …………………………. Nous avons été choisis pour aider dans l’evaluation du programme ProJustice, financé par le gouvernement américain en RDC et s’occupant de l’amélioration des services rendus par le système judiciaire aux citoyens en RDC. Nous cherchons à comprendre comment est-ce que ce financement a amélioré les services que donnent les tribunaux dans la zone d’intervention. Nous voulons vous poser quelques questions, à vous comme à beaucoup d’autres ménages dans cette localité de ………………. pour jauger le niveau d’atteinte des résultats de ce projet et voir si nécessaire comment améliorer son fonctionnement dans le futur. Les réponses sont confidentielles et ne seront pas divulguées individuellement. L’entretien va durer entre 20 et 30 minutes. Etes-vous d’accord ? Numero Questionnaire:|__|__|__|__| G2 G2.1 G2.2 G2.3 G2.4 G2.5 G2.6 G2.7 G2.8 G2.9 G2.10 G2.11 Relation avec le chef de ménage Sexe Age Sait-elle lire et écrire Quel est son niveau d’études? Activitéprincip ale Avez-vous déjà reçu directement un service de ProJustice? Avez-vous déjà entendu parler du projet ProJustice? Si Oui, où en avez-vous entendu parler? Avez-vous (ou un membre de la famille) eu un problème qui nécessitait recourir aux services d’un tribunal au cours des cinq dernières années? Si Oui, vous-etes vous rendu au tribunal ? |___| |___| |__|__| |___| |__|__| |___| |___| |___| |__|__| |___| |___| [1] Chef de ménage [2] Epouse [3] Autre adulte dans le ménage [1] M [2] F [98] Ne sait pas [1] Oui [0] Non [1] = Alphabétisée [2] Niveau primaire [3] Niveau secondaire [4] Niveau universitaire [95] Aucun [98] Ne sait pas [1] Agriculteur (trice) [2] Commerçant [3] Fonctionnaire [3] salarié(e) privé [4] Autre [5] Pas employé [1] Oui [0] Non [1] Oui [0] Non [1] = au niveau du tribunal [2] session de sensibilisation [3] Radio￾Télévision [4] Autre, Préciser [98]Ne sait plus [1] Oui [0] Non [1] Oui [0] Non Si Oui allez à G3 Si non, allez à G5 ` Final Evaluation: ProJustice Program in the DRC A-33 G3 G3.1 G3.2 G3.3 G3.4 G3.5 G3.6 G3.7 G3.8 G3.9 G3.10 G3.11 Quelle est votre impression sur la qualité du service qui a été rendu ? Avez￾vous payé des frais pour le service ? Si Oui comb ien avez￾vous payé? Comment jugez-vous ces frais ? Pensez-vous que ces frais ont baissé maintenant? Comment jugez-vous la rapidité du tribunal à solutionner votre cas? Pensez-vous que le tribunal traite les cas de ce genre plus rapidement maintenant ? Comment jugez-vous le profession nalisme des agents du tribunal dans la solution de votre cas ? Pensez-vous que le système est devenu plus professionnel maintenant ? Feriez-vous recours au tribunal dans le futur pour de pareilles situations ? Pour quoi? |___| |___| |___| |___| |___| |___| |___| |___| |___| |___| [1] Bonne [2] Acceptable [3] Pas bonne [4] Sans opinion [1] Oui [0] Non [1] Trop chers [2] Acceptable [3] Pas chers [4] Sans opinion [1] Oui [0] Non [1] Bonne [2] Acceptable [3] Pas bonne [4] Sans opinion [1] Oui [0] Non [1] Bonne [2] Acceptabl e [3] Pas bonne [4] Sans opinion [1] Oui [0] Non [1] Oui [0] Non G4 G4.1 G4.2 G4.3 G4.4 G4.5 G4.6 G4.7 G4.8 G4.9 G4.10 G4.11 Avez￾vous eu accès à un appui quelconq ue pour ce cas? Par qui a été apporté cet appui? De quelle nature était cet appui ? Comment jugez-vous la qualité de cet appui ? Pourquoi ? Votre cas a-t-il été vidé ? Avez-vous l’impression que la procédure a respecté vos droits ? Avez-vous eu l’impression à un moment ou un autre que vos droits étaient entrain d’être spoliés ? Pourquoi ? Finalement êtes vous satisfait de la manière dont votre cas a été (ou est entrain d’être) traité ? Pourquoi? |___| |___| |___| |___| |___| |___| |___| |___| [1] Oui [0] Non [1] Avocat [2] OSC [3] Projet [4] Autre [1] Pécuniaire [2] Conseil [3] intermediatio n [4] Autre [1] Bonne [2] Acceptable [3] Pas bonne [4] Sans opinion [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non ` Final Evaluation: ProJustice Program in the DRC A-34 G5 G5.1 G5.2 G5.3 G5.4 G5.5 G5.6 G5.7 G5.8 G5.9 G5.10 G5.11 Si non pourquoi ne vous ne vous y êtes pas rendus ? Quelqu’un vous aurez t-il conseillé de ne pas aller au tribunal ? Pensez￾vous que vous y auriez eu un traitemen t juste ? Vous êtes vous soucié du prix que vous auriez à payer pour ce cas ? Combien cela vous aurait il couté si vous vous étiez rendus au tribunal ? Y-avait-il une autre alternative pour solutionner votre cas ? En quoi cela consistait-elle ? Avez-vous tenté de chercher un appui juridique durant cette période ? Saviez-vous que vous auriez pu obtenir un appui juridique gratuit au niveau de votre communauté ? Votre cas est il maintenant résolu ? Iriez-vous au tribunal maintena nt si le cas se reposait ? |___| |___| |___| |___| |_______| |___| |___| |___| |___| |___| |___| [1] = le tribunal est trop loin [2] Les frais sont élevés [3] Manque de confiance en la justice [4] Autre, Préciser [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non [1] Mediation [2]Arrangement [3] Autorités traditionnelles [4] Autre, preciser [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non [1] Oui [0] Non ` Final Evaluation: ProJustice Program in the DRC A-35 APPENDIX 8: INFORMATION SOURCES AND CONTACTS NAME INSTITUTION POSITION/TITLE CONTACT: EMAIL OR TELEPHONE Daniel Dobrovoleljec Tetra Tech DPK Former CoP ddobrovoljec@ProJustice-ci.com Justin Ntakobajira Tetra Tech DPK CoP Jmushaga@hotmail.com Daniel Dobrovoleljec Tetra Tech DPK Former CoP ddobrovoljec@ProJustice-ci.com Jessica Vapnek Tetra Tech DPK Director major projects jvapnek@dpkconsulting.com Fabien Buetusiwa Tetra Tech DPK Senior PCPC Esai Kabwit Tetra Tech DPK PCPC Kabnjimb@gmail.com Lyliane Ankwarpen Okarintsung Tetra Tech DPK Grants Manager Telesphore Kavundja Consultant ProJustice kavundja@yahoo.fr Evariste Kaniama Tripaix Bagata Greffier Titulaire 0810959058/0821438015 Kayembe Ngoie Smith Tripaix Bagata Juge 0811985134/0998329730 Freddy Mabangaba Mbate Tripaix Bagata President Jean Pierre Ndunba TGI/Bandundu Procurer 0991297823 Esperance Ikavu Ceprosoc Administration and Finance 0817141373/0970694906 Maitre Franck Kilombp PROSADEF Lawyer 0814501215/0898418786 MaitreHippolyte PROSADEF Lawyer 0812855866/0897267355/08547 58145 Famille Mungenga (Iswa et Merveille) PROSADEF GBV victim Case number RP 7623 TGI Mpangi Vilma PROSADEF GBV victim Case number RP 7622 TGI Junior Bosukula PROSADEF Lawyer 0898952848/0825792356 Abbe Kibangu Societe Civile Bagata President 0810594746 Mathieu Su Arye Administration Publique Chef de Cite Bagata 0816705617 Makila Fambemb Societe Civile Bagata Membre 0822747454 Lievin Hokoly Unjour Societe Civile Bagata Membre 0813804037 Ngoy Silambwa Societe Civile Bagata Membre 0897522336 Ebondo Moke Societe Civile Bagata Membre 0821307540 Paulin Mampia ONG Bunkote Bagata Membre 0819884933 Mweni Luza, Ceprosoc Beneficiaire ProJustice Mbenga Sylvie Ceprosoc Beneficiaire ProJustice 0896448581 Nzazi Patience Ceprosoc Beneficiaire ProJustice 0891013096 Nkufi Papilenge Ceprosoc Beneficiaire ProJustice 0817476363 LupiFlor Ceprosoc Beneficiaire ProJustice 0813538681 Amerlin Ngenzi Ceprosoc Beneficiaire ProJustice 0824022315 Munetukumeso Vital Ceprosoc Beneficiaire ProJustice 0993875737 ` Final Evaluation: ProJustice Program in the DRC A-36 NAME INSTITUTION POSITION/TITLE CONTACT: EMAIL OR TELEPHONE Mayele Lukie Ceprosoc Beneficiaire ProJustice 0819728443 Angelique Kipulu LDFC/Kinshasa Sec. Gen 0898955440 Anne Matondo LDFC/Kinshasa Chargée des Operations 0997889729 Huguette Songabau LDFC/Kinshasa Chargée des Finances Muguetsonga yahoo.fr Jean-Pierre Cakwangasha Cour d’Appel de Lubumbashi Premier president 0818150106 Caïphe Useni Parquet Général de Lubumbashi Procureur Général 0999942689 Mavungu Parquet de Grande de Lubumbashi Procureur de la République 0819993700 Daniel Lokoya Tribunal de Grande Instance de Lubumbashi President 0814708298 Mujinga Parquet de Grande de Lubumbashi Secrétaire Divisionnaire 0997037526 Jean-Toussaint MBUMB Tribunal de Paix de Lubumbashi￾Kamalondo Président 0810007777 Odette Mwamba Dynamique des Femmes pour le developpement du congo L’shi Coordonatrice 0997016861 Olivier Kabeya Barreau Lumumbashi Avocat 0998763977 Odette Kazadi Dynamique des Femmes pour le développement du Congo LFDC Coordonatrice 0997016861 Flory Okandju ProJustice/avocat Coordonateur 0814079119 Jean Claude Tripaix de Kasumbalesa Procureur Martin Guizebo Association Estudiantine des Auditeurs de radio Okapi Coordonateur Hyppolyte Ngampimpa Barreau de Bandundu Avocat 0812855866 Jules Bosukula Barreau de Bandundu Avocat 0898952848 Frank Kilombo Barreau de Bandundu Avocat 0814501215 Ngoy Tangizya Cour d’appel de Lumumbashi Greffier principal 0815475159 Cakwangasha Jean Pierre Cour d’appel de Lumumbashi Premier president 099 702 1921/ 081 815 0106 Daniel Lokota TGI Lumumbashi President 099 99 081 94/ 081 470 8298 ` Final Evaluation: ProJustice Program in the DRC A-37 NAME INSTITUTION POSITION/TITLE CONTACT: EMAIL OR TELEPHONE Tripaix de Kasumbalesa Greffier titulaire Mme KITETE AFEMAC Vice-Présidente: 099 70 25 874 Me Patrick Kabajika Vision Sociale Administrateur Muhindo Kamasita Alexandre Cour d’appel de Bukavu President Caiphe Parquet Bukavu Procureur General de la Republique Donatien Lukulunga Greffe Bukavu Greffier principal TGI Bukavu Jean Claude Dunia Ordre des avocats Bukavu Batonnier Venance Sivirwan Remy FOMEKA Uvira Coordinateur 099 0223304/081 546 8844 Me Innocent Bahati OCET Bukavu Coordonateur 0971301353/ 081 355 1835 Parquet Kavumu Procureur Jean Pierre Buledi CEDAC Bukavu Charge de recherche et documentation ` U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov