DRG LEARNING, EVALUATION, AND RESEARCH (DRG-LER) ACTIVITY Impact Evaluation of USAID Haiti PROJUSTICE Program Pretrial Detention Component TASKING 031 April 2017 Contract No. GS-10F-0033M/AID-OAA-M-13-00013 April 21, 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. DRG LEARNING, EVALUATION, AND RESEARCH (DRG-LER) ACTIVITY TASKING N031 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT (APRIL 2017) Prepared under Contract No.: GS-10F-0033M/AID-OAA-M-13-00013 Prepared by: Tara Slough, Columbia University Christopher Fariss, University of Michigan Submitted to: Morgan Holmes Submitted by: NORC at the University of Chicago Attention: Renee Hendley, Program Manager Bethesda, MD 20814 Tel: 301- 634-9489; E-mail: Hendley-Renee@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | i CONTENTS CONTENTS ..................................................................................................................... I ACRONYMS ................................................................................................................... II TABLES......................................................................................................................... III FIGURES ....................................................................................................................... III EXECUTIVE SUMMARY ................................................................................................ 1 1. INTRODUCTION...................................................................................................... 3 Background................................................................................................................. 4 Theory of Change ....................................................................................................... 6 Evaluation Objectives................................................................................................. 7 Survey Questions ....................................................................................................... 8 Evaluation Questions ................................................................................................. 9 Methodology ............................................................................................................... 9 Outcome Data ...........................................................................................................17 Data Collection and Quality Control........................................................................19 2. KEY FINDINGS...................................................................................................... 20 Characteristics of PTD in Haiti.................................................................................20 The Causal Effects of Legal AssistANCE................................................................26 Costs and Benefits of the Provision of Legal Assistance......................................33 Conclusion and Recommendations ........................................................................37 ANNEX A, DESIGN REPORT ...................................................................................... 39 ANNEX B, SURVEY INSTRUMENT(S) ........................................................................ 40 DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | ii ACRONYMS 2SLS Two-Stage Least Squares CACE Complier Average Causal Effect CdB Prison of Croix-des-Bouquets GoH Government of Haiti IPW Inverse Probability Weighting ITT Intent-to-Treat Effect IV Instrumental Variables LATE Local Average Treatment Effect NORC National Opinion Research Center (NORC at the University of Chicago) OLS Ordinary Least Squares PI(s) Principal Investigator(s) PN Penitencier Nacionale d’Haiti PTD Pretrial Detention PV Women’s prison at Petionville RCT Randomized controlled trial USAID United States Agency for International Development DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | iii TABLES Table 1. Selection into experimental sample............................................................ 10 Table 2. Actors responsible for data collection and cleaning.................................... 19 Table 3. Reasons for non-compliance...................................................................... 24 Table 4. Formulations of the treatment assignment indicator................................... 27 Table 5. ITT estimates.............................................................................................. 29 Table 6. Formulations of the treatment indicator for IV regressions . ....................... 31 Table 7. CACE and LATE estimates. ....................................................................... 32 FIGURES Figure 1. Total prison and PTD detainee populations, 2006-2015. ............................ 5 Figure 2. Age of detainees in experimental sample. ................................................ 11 Figure 3. Year of arrest of detainees in experimental sample. ................................. 12 Figure 4. Quintet blocking and random assignment visualization............................. 13 Figure 5. Assignment to treatment and compliance. ................................................ 15 Figure 6 Variation in treatment dosage. ................................................................... 15 Figure 7. Sources of data and sequencing of data collection................................... 17 Figure 8. Household assets compared to DHS survey data..................................... 22 Figure 9. Baseline levels of trust in Haitian government institutions......................... 22 Figure 10. Self-reported diagnosis rates of disease/illness during detention............ 23 Figure 11. Visualization of ITT estimates. ................................................................ 30 Figure 12. IV estimator implied causal process........................................................ 30 Figure 13. Simulated reductions in detention and associated cost savings.............. 36 DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 1 EXECUTIVE SUMMARY From 2009 to 2016, USAID Haiti sponsored a justice-sector program known as PROJUSTICE. PROJUSTICE’s primary goal was to enhance justice delivery in order to promote stability and security in Haiti. One component of the program addressed this challenge by working to reduce the incidence and prevalence of prolonged pretrial detention (PTD) in Haiti. Prolonged PTD refers to detention prior to trial or judgment that illegally exceeds the Constitutional time limits set forth to govern this type of detention. PTD thus represents a form of arbitrary imprisonment and a grave human rights concern. This concern is particularly grave: 70% of the Haitian prison population is composed of individuals in PTD and prolonged PTD. PROJUSTICE sought to reduce the prevalence of prolonged PTD through the provision of free legal assistance to individual detainees held in PTD. While the Haitian Constitution guarantees the accused access to free legal representation, in practice, the government of Haiti does not employ public defenders. As such, individuals held in prolonged PTD typically have no access to legal representation or advocacy. In this void, through the PROJUSTICE program, the legal assistance provided by the project to selected prolonged pretrial detainees resulted in the procedural advancement of 1,364 cases, the acquittal of 630 individuals, and the release of 578 individuals over the life of the PROJUSTICE project. This report documents the results of an impact evaluation of the component of PROJUSTICE that provided legal aid for the illegally detained. In this context, illegal detention refers to detention that has overrun constitutionally-defined time limits for criminal procedure. The impact evaluation was conducted in two jurisdictions, Port au Prince and Croix-des-Bouquets in 2016 at the end of the PROJUSTICE program. The impact evaluation is a randomized control trial (RCT), allowing for the credible estimation of the causal effects of PROJUSTICE’s legal assistance on the case trajectories of individual cases. The impact evaluation utilizes a novel randomized rollout design. In this design, individual detainees were randomly assigned to an order in which they were offered legal assistance. The design maximized the number of detainees that ultimately received this scarce treatment. It provides variation in the presence of treatment (legal assistance vs. no legal assistance) and in the dosage of treatment (weeks of legal assistance). In the course of the impact evaluation, a wealth of original data was collected in order to characterize the PTD situation at the individual/case level and measure the impacts of the legal assistance intervention. This data consists of: (1) prison registers; (2) baseline and endline surveys and “censuses” of the experimental subjects; (3) the case files collected by PROJUSTICE legal assistants; and (4) administrative data on cases collected from the Courts of First Instance during the endline data collection period. This data allows for two forms of analysis. First, we conduct a descriptive analysis to characterize the PTD situation in Haiti. We find that detainees are disproportionately poor, young men. Unsurprisingly, these detainees do not trust Haitian governmental DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 2 institutions. Further analyses of the barriers to providing the treatment (legal assistance) to some of those assigned to receive it provides descriptive evidence of some characteristics of the Haitian criminal justices system as well as the barriers to reducing the incidence and prevalence of PTD. One specific barrier to the timely processing of cases remains the state of prison and court records. We estimate that approximately 6% of those recorded as pretrial detainees in prison files are not presently detained or have already been judged. The Court of First Instance (judicial sector) files are substantially less accurate than the court records. We then measure the impact of the legal assistance treatment. We find that legal assistance increased the proportion of detainees who were freed nine months after the start of the impact evaluation period. Given the randomized rollout design, we compare the predicted probabilities that the first and last detainees in the randomly assigned order (in each prison) were released during this timeframe. The first detainee in the order was released with a predicted probability of 0.206 versus 0.133 for the last detainee in the order. This 7.3 percentage point Intent-to-Treat (ITT) effect size corresponds to a large 54.9 percent effect upon the predicted probability for the last detainee in order and is statistically significant at the (one￾tailed) level. Examining heterogeneity in this ITT, the effects are positive across jurisdictions, however the effects are substantively much larger in Croix-des￾Bouquets (15.2 percentage points) than in Port au Prince (4.9 percentage points). As such, legal assistance does assist in reducing the duration that cases remain in illegal pretrial detention. However, legal assistance can only reduce the prevalence of PTD under certain conditions. Given the relatively modest effect sizes, legal assistance can only reduce the prevalence of prolonged PTD if reductions in the duration of PTD counterbalance the number of individuals being arrested and detained. To this extent, it is essential to provide a sufficient-sized unit of legal assistants such that all detainees can be represented. Despite the modest effect sizes, we estimate that this USAID-funded legal assistance provided substantial financial benefits to the Government of Haiti (GoH). Imprisonment is quite costly—approximately $40 per detainee per month—and reductions in the length of PTD correspond to substantial savings. Our estimates suggest that the savings associated with the reduction of time spent in PTD outweigh the staffing costs paid to hire lawyers during the period of the intervention. As such, we argue GoH-sponsored legal assistance could provide substantial cost savings. The evidence gathered here suggests that legal assistance is a necessary but not sufficient policy to reduce the prevalence of prolonged PTD in Haiti. Detainees disproportionately view access to a lawyer as the best route to release from illegal PTD. Moreover, we provide systematic evidence that outside of PROJUSTICE, there are scant options of legal representation for the illegally detained. While these demand-side features show a great need for expanded access to legal assistance, there is also suggestive (albeit non-causal) evidence that institutional features within the Courts of First Instance appear to influence the efficacy of legal assistance. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 3 These institutional impediments speak to the challenges to reducing PTD in Haiti. The limited legibility of records suggests a substantial need for systems that can actualize, update, and share case information across the courts and prisons. PROJUSTICE investments in electronic case management information systems (CMIS) provide some remedies to this problem, but there remain jurisdictions that are underserved in this regard. Further, efforts to limit corruption and absenteeism in the courts may render legal assistance more efficacious and cost effective. We provide four recommendations for ways to reduce prolonged pretrial detention through future justice-sector programming in Haiti: 1. Widespread free legal assistance is critical to reducing PTD in Haiti. 2. Legal assistance is necessary but not sufficient to eliminate illegal PTD in Haiti. 3. Investment in recordkeeping may complement legal assistance in addressing PTD. 4. Reform of criminal law may help to alleviate excessive rates of illegal PTD in Haiti. This report describes the motivation, methodology, and results of the USAID Haiti PROJUSTICE Program PTD Components impact evaluation. It provides a systematic effort to describe the conditions of pretrial detainees in two Haitian jurisdictions. It subsequently provides causal evidence that legal assistance is indeed effective in combatting the duration of pretrial detention. The evidence suggests that these effects are conditional on several institutional conditions within the court system. These findings provide implications for future justice sector programs in Haiti and beyond. To the extent that prolonged PTD is common in many countries, the implied policy recommendations generalize to multiple national contexts in which USAID provides justice sector assistance. 1.INTRODUCTION Between 2009 and 2016, USAID Haiti sponsored and administered a justice sector program named PROJUSTICE. PROJUSTICE aimed to enhance justice delivery in order to promote stability and security in Haiti. The program included four foci: (1) promoting access to justice through provision of legal aid, legal education, and mediation; (2) reducing the prevalence of pretrial detention (PTD) through provision of assistance to detainees in PTD; (3) promoting an independent judiciary; and (4) advocating for legal (Constitutional) reform. During the lifetime of the program, activities were implemented in five of Haiti’s 18 jurisdictions: Cap-Haïtien, Croix-des￾Bouquets, Fort Liberté, Port au Prince, and Saint Marc. During the final months of PROJUSTICE in 2016, the second component of the program, legal aid to combat illegal PTD, was subjected to an impact evaluation. This impact evaluation was implemented in three prisons across the Croix-des- DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 4 Bouquets and Port au Prince jurisdictions. This report discusses the motivation, design, and findings of this impact evaluation. This impact evaluation report provides two types of findings derived from the original data collected in the process of the impact evaluation. First, this report provides descriptive information about the conditions of illegal PTD in Haiti. While Haitian prison conditions and the rate of PTD have featured prominently in the US Department of State’s annual human rights reports from Haiti, there exists little information on the characteristics of detainees or their experience within the criminal justice system. This report aims to inform our understanding of pretrial detention in Haiti. Second, this report documents the causal effects of the intervention, the provision of legal assistance, on case advancement and final case dispositions. The impact evaluation was a randomized control trial (RCT). The RCT research design provides a credible counterfactual and thus allows for the estimation of causal quantities of interest (treatment effects). BACKGROUND Prolonged PTD represents a major source of human rights abuse in Haiti. Pretrial detention, also known as remand or preventative detention, is not illegal in its own right. However, it is closely regulated by strict time limits for different components of criminal procedure. For example, the Haitian Constitution specifies that, upon arrest, a detainee must see a judge within 48 hours. PTD becomes illegal or prolonged (used interchangeably in this report) if this 48-hour timeframe is surpassed and the detainee has not yet seen a judge. Subsequent steps in criminal procedure are also routinely overrun. In general, prolonged PTD often stretches for months or years in the Haitian penal system. Illegal PTD represents one form of arbitrary imprisonment. This human rights abuse represents both a violation of Haitian Constitutional law as well as various international human rights declarations and treaties. The right to be free from arbitrary arrest, detention, or exile is enshrined both Article 9 of the United Nations Declaration of Human Rights and Article 9 of the International Covenant for Civil and Political Rights. As such, the prevalence of PTD in Haiti represents a grave concern to human rights advocates. Outside observers have regularly expressed concern at the state of PTD in Haiti. The US State Department’s human rights reports on Haiti have routinely described the gravity of the situation. Figure 1 plots data from the last ten published country human rights reports on Haiti. The left panel depicts the size of the Haitian prison population based on reporting from the Haitian National Police (PNH) which has administrative control over the prisons. In general, with the exception of 2010, the prison population has grown monotonically as has the population of pretrial detainees. In 2010, almost the entire population of detainees in Port au Prince’s Penitencier Nationale d’Haiti (PN) escaped during and after the January earthquake. PN houses nearly half of Haiti’s inmate population. As such, limited ability to DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 5 recapture escaped detainees led to a substantial reduction in the nationwide population of detainees. Figure 1. Haiti’s total prison and PTD detainee populations, 2006-2015 (left) and the proportion of prisoners held in PTD, 2006-2015 (right). Data from US Department of State Haiti Human Rights Reports, 2006-2015. The right panel of Figure 1 depicts the proportion of all Haitian prisoners in PTD as per PNH records. While the quantity declined slightly prior to 2010, since the earthquake, it has remained relatively constant around 70% of the prison population. This nationwide figure obscures the variation between Haiti’s 18 jurisdictions. In Port au Prince, for example, this figure hovers around 90%.1 These aggregate measures also obscure the characteristics of the population affected by Haiti’s excessive rates of prolonged PTD. In general, as we document with original data, these detainees are poor. Detainees may secure faster passage from arrest to release or conviction by hiring a lawyer and/or paying a bribe to court or prison officials. Those detainees who cannot afford to hire a lawyer or pay a bribe are most likely to stagnate in prolonged PTD. While the Haitian constitution assures the right to a free lawyer, public defenders have never been hired by the State. The component of PROJUSTICE under the current evaluation, legal assistance to combat prolonged PTD, parallels closely what a Haitian public defender’s office might look like. Legal assistants were contracted in collaboration with local bar associations and provided legal assistance to individuals held in prolonged PTD. As such, our findings provide evidence that the employment of public defenders could reduce the rate of PTD by limiting the amount of time detainees spend imprisoned prior to trial. In the conclusion, we quantify the savings that may be gained through the creation of a public defender’s office. 1 Interviews with prison and court officials, June 2015 and January 2016. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 6 THEORY OF CHANGE PROJUSTICE facilitated the provision of legal assistance. The inputs included the contracting of 18 lawyers (three also served as supervisors). Twelve of the lawyers were based in the jurisdiction of Port au Prince and the other six were based in Croix-des-Bouquets. PROJUSTICE staff supervised the work of the lawyers largely by reading and advising on the lawyers’ case files and by auditing their work in both the courts and prisons. The activities performed by the lawyers varied to some extent by case. In general, the lawyers would first locate and interview a detainee in the prisons. This initial visit served three purposes. First, they would ensure the detainee was still detained in PTD. As the prison records from which the sample was drawn had some inaccuracies (discussed below), finding a detainee in the prison was critical to understanding whether legal assistance could be rendered. Second, they would assess the eligibility of the individual to receive legal assistance under USAID criteria.2 Finally, they would seek the detainee’s permission to be represented. While all eligible detainees agreed to the free legal representation, it is important from an ethical standpoint that the lawyers sought detainees’ consent. The second main activity of the lawyers was a visit to the relevant Court of First Instance (CFI) to locate a detainee’s case files. This allowed lawyers to determine the procedural status of a case as well as the individual judge or prosecutor responsible for completing the subsequent step. When files were located and read, this provided a second check to make sure the individual was eligible for legal assistance. When case files were not located in the court, lawyers lodged requests to have the case files reconstituted so that the case could move on. Legal assistance was rendered in an effort to advance cases procedurally through the justice system. In a context with high levels of corruption, it is important to clarify that lawyers did not provide bribes to justice system officials at any step in the process. Moreover, legal representation proceeded according to the formal processes specified in Haitian criminal law. The outputs during the impact evaluation implementation period (11 weeks) include the review of 505 cases. Of these 505 cases, 348 were eligible to receive legal assistance and were thus represented by PROJUSTICE lawyers. These activities are documented in detailed case files maintained by PROJUSTICE. This impact evaluation seeks primarily to measure two core outcomes. First, we measure the procedural advancement of cases during the evaluation period. Second, we measure whether detainees had been released from pretrial detention approximately six months after the three-month intervention concluded. As these are the primary goals of legal assistance, they serve as an appropriate test of the efficacy of the program. 2 USAID-funded legal assistance did not defend individuals charged of rape, human trafficking, or drug trafficking. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 7 The long-term goals of this program are conditioned upon the efficacy of the program in achieving its immediate objectives. First, the program seeks to lower the rate of PTD in Haiti. As per Figure 1, there is substantial room for improvement. Second, if legal assistance is effective, this study seeks to demonstrate the cost savings for the government of Haiti via provision of a public defender relative to the long-term pretrial imprisonment of so many individuals. If such cost savings can be shown, then it will help justify the expense of the program for the government. At the detainee level, there are two principal long-term outcomes of legal assistance. First, shorter stays in PTD should improve the economic fortunes of detainees and their families. Detainees are largely young males. Given social and economic tendencies that favor male breadwinners in Haitian society, PTD imposes substantial externalities on the families of the detained. Lower rates of PTD should reduce the economic stresses on families of the illegally detained. Additionally, a reduction in the prevalence of illegal detention may also change attitudes toward the Haitian government and Haitian political institutions in the long term. As we document, individuals in illegal detention demonstrate low levels of trust and confidence in Haitian governmental institutions. While we cannot estimate the causal effect of illegal PTD on these attitudes, we postulate that a reduction in PTD may increase confidence in the performance of some institutions. Promoting trust in the Haitian government is important in the promotion of democratic stability in Haiti. EVALUATION OBJECTIVES This impact evaluation report has two principal objectives. First, we develop descriptive evidence on characteristics of the pretrial detention in Haiti. Here, we focus on previously unstudied individual-level characteristics of detainees and their cases. In order to develop this evidence, we rely on three forms of original data collection. First, in collaboration with the implementer we collected prison records from the three prisons that are included in the impact evaluation in late 2015, prior to the start of baseline data collection. Second we conducted baseline and endline surveys on a random sample of the prolonged pretrial detainees within early 2016 and then nine months later. Finally we collect a set of case records from the courts on this random sample of detainees at the same time as the endline survey. This data provides leverage to assess the accuracy and characteristics of recordkeeping in the Courts of First Instance. Second, the primary evaluation objective is to measure the effect of the provision of the USAID-sponsored legal assistance provided by PROJUSTICE on case trajectories for this sample of detainees. In particular, we focus on final case disposition, or whether an individual is released from pretrial detention by the endline data collection. We also examine the case advancement as an intermediate level outcome. In these specifications, we measure whether a case progresses from one step in the prosecution or criminal prosecution process to the next step. The ultimate objective of this evaluation is to assist USAID by providing the evidence base for two types of activities to remedy illegal PTD. First, this evidence should DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 8 provide a resource for the advocacy efforts performed by USAID and other NGOs in order to persuade the Haitian government to adopt best practices in order to reduce the prevalence of pretrial detention in Haiti. Second, given the worldwide prevalence of illegal pretrial detention, particularly in developing settings, we believe that this evidence may be used to inform aid programs and policies on PTD in multiple settings in which USAID is active. SURVEY QUESTIONS The collection of two-wave panel survey data served several purposes in the present impact evaluation. First and foremost, the baseline and endline surveys serve as a form of a “census” of the experimental sample. We specifically refer to the tracking of individual detainees’ status in the prison at two points in time (baseline and endline) as a “census” because it represents an official count of the prison population. Unlike the rest of the survey for which we only have responses from those that were located and consented to participate in the survey, the census provides information, which is necessary for the entire experimental sample. For this reason, we distinguish the “census” portions of this data collection from the rest of the surveys. Given that we randomly sampled from the prison registers several months before the implementation of the baseline surveys, the baseline version of the “census” allows us to assess changes to detainees pretrial status as well as the legibility of the original records from which we sampled illegal pretrial detainees. We focus intently on this “census”-like measure of individuals imprisoned during the endline wave. This allows us to assess the effect of legal assistance on the probability that a detainee remains in pretrial detention nine months after legal assistance commenced. The survey questions focus on gathering several characteristics of detainees so that we can better quantify the characteristics of Haiti’s population of pretrial detainees. First we ask a battery of demographic questions including age, residence, and family (household) characteristics. Many of these questions are similar to those asked by the DHS surveys, allowing us to compare the characteristics of these inmates to the general population of Haiti. Second, we seek to quantify both detainees knowledge of their imprisonment and the legal status of their case. Further, we seek to probe detainees’ knowledge of their Constitutional rights as pretrial detainees. We accomplish these tasks by asking a series of questions about their understanding of the charges pursued against them as well as their knowledge of the rights of the accused under Haitian criminal law. Additionally, we asked the survey enumerators to evaluate each detainee’s understanding of his/her situation at the conclusion of both the baseline and endline surveys, which provides a third measure of legal understanding or awareness. Third, we seek to measure the detainees’ trust in Haitian government institutions. Given that all baseline and some endline surveys were administered in prisons administered by the PNH, we were limited in the questions that we could ask about the PNH specifically. However, we asked a battery of questions about trust in other Haitian governmental institutions. Our survey questions were developed to parallel DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 9 the Latin American Public Opinion Project (LAPOP) “AmericasBarometer” survey administered in Haiti in 2014. This permits us to compare levels of trust among illegal pretrial detainees to trust levels in the general population. Additionally, in the endline survey, we included modules on the economic characteristics and health conditions of the detainees. We focus on the socioeconomic status of the detained to measure to what extent we might believe that illegal detention imposes externalities on the family members of the illegally detained. Given the male breadwinner model dominates the Haitian workforce, we may believe that the disproportionate imprisonment of men creates significant economic strain on their families’ economic conditions. Finally, we ask questions about the health of detainees during their imprisonment. Previous work documents the state of health within the Haitian prison system. We seek to descriptively document the prevalence of different diseases and conditions within these prisons. In the endline survey, we administered two versions of the survey: one for individuals who were still detained at the endline and another for individuals who had been released from prison by this date. However, different response rates related to the assignment of legal assistance (as we document) limit our ability to make valid inferences about the effect of legal assistance on the basis of survey answers. Among those still detained at the endline, the re-interview (response) rate was 85.4%. Among, those who were surveyed (in prison) in the baseline who had subsequently been released from prison, we had a very low response rate of 9%. The differential response rate stems from the challenges in contacting these detainees and in encouraging them to talk about their time in illegal detention. EVALUATION QUESTIONS The central focus of the evaluation is to determine the causal effect of legal assistance on the cases and thus detention status of the illegally detained. As such, we ask how legal assistances alters the probability that an individual is illegally detained nine months after the beginning of the impact evaluation. This allows us to assess whether legal assistance reduces the duration of prolonged PTD. We also examine the extent to which institutional and case features condition the effectiveness of legal assistance. METHODOLOGY Population and sampling For the purposes of the impact evaluation, the population of interest is individuals who were illegally detained in pretrial detention and three prisons. In the Port-au-Prince jurisdiction, these prisons include the National Penitentiary (PN) and the Women’s prison at Petionville (PV). In the Croix-des-Bouquets (CdB) jurisdiction, the impact evaluation was implemented at the Croix-des-Bouquets prison (a men’s prison). DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 10 Our sample of those who are illegally detained in pretrial detention comes originally from prison register records collected by the implementer approximately two months prior to baseline data collection. From these records, we define the population that is eligible to receive legal assistance. Those who were deemed eligible by USAID and PROJUSTICE include those held for at least six months in illegal pretrial detention, calculated based on their date of arrest. The sample thus comprises inmates who were originally detained between 2010 and mid-2015. As per USAID requirements, the PROJUSTICE legal assistance program did not provide legal assistance to individuals accused of three crimes: (1) rape; (2) human trafficking; or (3) drug trafficking. In order to limit the share of detainees in the sample who were accused of these crimes and would therefore not be able to receive legal representation, we excluded detainees accused of these three crimes from the population eligible to be sampled. In practice, after conditioning on detainees that had spent six months in PTD, this exclusion criteria resulted in very few exclusions, as depicted in Table 1. Note that the number of detainees recorded in prison registers is lower than overall counts maintained in prison offices, particularly in PN. Table 1. Population of detainees with >6 months in PTD according to prison registers as of November 2015 and inclusion in experimental sample. PRISON DETAINEES IN PRISON REGISTER WITH >6 MONTHS IN PTD DETAINEES EXCLUDED BASED ON CHARGES IN PRISON REGISTER EXPERIMENTAL SAMPLE/ELIGIBLE POPULATION CDB 329 0 230/329 (69.9%) PN 1840 10 800/1840 (43.5%) PV 54 0 50/54 (92.6%) Once we defined this population—pretrial detainees held illegally for more than six months and not accused of one of these three crimes—we employed stratified random sampling by prison. Specifically, we randomly selected 800 of the eligible detainees in PN, 230 of the eligible detainees in CdB, and 50 of the eligible detainees from PV. The sample included all the pretrial detainees in PV, given the smaller detainee population. Detainees in PN were under-sampled relative to detainees in CdB given the large population of PN. Thus, the total size of the sample was 1080 individuals. Figures 2 and 3 depict some of the characteristics of our sample. Figure 2 indicates that the sample is disproportionately young adults between the ages of 18 and 35. Though not depicted in this figure, these detainees are disproportionately male as all detainees in PN and CdB are male. (All detainees in PV are female.) As per Table 1, we oversampled from PV relative to PN and CdB, so the data understates this gender-based disparity. Figure 3 presents the year of arrest for the detainees in the sample. Those detainees in PN have, on average, been illegally detained for a longer duration than those DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 11 detainees in PV and CdB. The relatively shorter detentions in CdB reflect the fact that this prison opened in 2014 and experienced a major prison break in 2015. It also may reflect heterogeneity between the Port au Prince and Croix-des-Bouquets jurisdictions in terms of the efficacy of processing cases in the Courts of First Instance, a theme upon which we expand when presenting subsequent results. Figure 2. Self-reported age of detainees in the baseline survey. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 12 Figure 3. The year of arrest corresponding to the present detention for those in the experimental sample. Random Assignment of Legal Assistance The impact evaluation relies on our ability to construct a credible counterfactual to the treatment, the provision of legal assistance. In order to construct this counterfactual, we employ a randomized controlled trial. However, given the prison setting and the characteristics of the population of illegal pretrial detainees, two central ethical considerations inform our design. First we seek to change the way in which PROJUSTICE provided legal assistance to the smallest degree possible. By minimizing the changes to the program, we reduce the probability of increasing the risk to subjects as a result of the impact evaluation. Moreover, it allows us to estimate the effects of legal assistance as it was offered by PROJUSTICE over the program’s lifespan. Our second and most important ethical consideration was to maximize the beneficiary population of the program during the impact evaluation period. The beneficiary population consists of the detainees who are assigned to legal assistance during the 11-week period in which the impact evaluation was conducted in the prisons. Conventional two-arm experimental designs with a single treatment and a single control group set the number of treated units ex ante. In contrast our design allowed PROJUSTICE’s legal representatives to treat as many individuals as possible given time, staffing, and resource constraints. As such, the impact evaluation’s random assignment differs quite substantially from traditional two arm RCTs. The impact evaluation utilizes a randomized rollout design. In this design, within each prison, each of the individuals in the experimental sample was randomly assigned an integer between 1 and the total number of individuals in the experimental sample. This number denotes an ordering. For example in PV, there DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 13 were 50 detainees in the experimental sample. These 50 detainees were assigned an order number ranging from 1 to 50. This assignment represents the order in which a detainee in the sample would receive treatment within the prison. For example, a lawyer would start by attempting to treat detainee assigned to the order number 1 before proceeding to attempt to treat detainee assigned to the order number 2 and so on. Within each of the prisons, we utilize a form of blocking to increase efficiency and estimation and ensure heterogeneity through the entire distribution of randomly assigned order. In order to do this we gathered several covariates from the prison registers based on the availability of each covariate. These covariates include: (1) a binary indicator of violent offense type; (2) a count of the number of charges (offenses) recorded in the prison registers; (3) detainee age; (4) the length (in months) of pretrial detention; and (5) an education indicator (where present). We create quintets of five individuals within each prison through an algorithm that minimizes the multivariate distance on the basis of these covariates. Note that all quintets are contained within the same prison block (i.e. 160 quintets in PN, 46 quintets in CdB, and 10 quintets in PV.) The order was randomized within the quintets and then the order of the quintets was randomized. Figure 4 provides a visualization of the random assignment and role of the quintet blocks in the women’s prison, PV. Note that one individual from each quintet is included in each quintile of the sample. We refer to this order indicator with the variable where indexes the individual and indexes the prison. Figure 4. The quintet blocking scheme within PV. Each color corresponds to a quintet block, calculated as described in the text. First, the individuals within a block were randomly ordered, then the blocks (the colored bands) were randomly ordered. This ensures that each quintile of the sample contains one member of each quintet. Implementation and Compliance with Treatment Assignment PROJUSTICE assumed full responsibility for the implementation of the legal assistance treatment during the impact evaluation period. During this time, lawyers DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 14 working in each prison would seek new cases from the list of detainees included in the experimental sample and following the order prescribed by the randomization. The legal assistants and PROJUSTICE staff kept careful records of the treatment assigned and provided to the detainees within the experimental sample. The randomly assigned order determines whether a subject was assigned to treatment as well as the dosage of treatment she received. We measure dosage as the length of exposure to the legal assistance treatment. However, in some cases detainees assigned to treatment could not be treated. This represents a case of one￾sided noncompliance, also known as “failure to treat.” There are several reasons for this noncompliance as enumerated in Table 3. In particular, some of the cases were not eligible for assistance by USAID. For example, 3.6% of subjects in the post￾filtered sample that were assigned to treatment were indeed accused of infractions that made them ineligible for representation by USAID. This was generally a result of outdated, inaccurate, or incomplete prison records. In the plurality of cases of one-sided noncompliance, detainees were not provided legal assistance because their case had multiple defendants of which one or more were still at large. Cases with multiple defendants proceed more slowly than those with single defendants and limits what legal assistants can accomplish in a short timeframe. As such, at the beginning of the implementation, the investigators and PROJUSTICE agreed that given the limited length of the impact evaluation, treatment was better allocated to those who were not accused with other at-large codefendants. Figures 5 and 6 depict the rollout of treatment assignment. First, Figure 5 indicates the share of subjects assigned to treatment among those present in the baseline census.3 Notably, only about 70% of those assigned to treatment that were in the prison at baseline were treated. There were various reasons for which legal assistants could not treat some detainees, as enumerated in Table 3 later in report. A detainee’s status as a “complier” or “non-complier” is simply a measure of whether treatment was delivered when a detainee was assigned to treatment. Figure 6 disaggregates the left bar from Figure 5 to depict the variation in dosage of treatment that was assigned and delivered. (The low count of the first bar in Figure 6 is due to a partial week of treatment during the first week.) These graphs correspond directly to our empirical strategy for estimating the causal effects of PROJUSTICE. Rates of compliance and assignment to treatment vary within the prison blocks. We account for the heterogeneity in probability of assignment to treatment in the relevant statistical analyses. 3 We discuss this subset of the full experimental sample in the Attrition section. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 15 Figure 5. This graph shows the count of subjects assigned to treatment and control (of those present for the baseline census). It depicts the compliers (assigned to treatment and treated) and non-compliers (assigned to treatment but not treated) among those assigned to treatment. Figure 6 This graph shows the variation in dosage (assigned and delivered) among those subjects assigned to treatment. The blue and green correspond to the blue and green areas in Figure 5. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 16 Our analyses account for this form of one-sided noncompliance in two ways. First we estimate a set of estimands known as intent-to-treat (ITT) effects which we analyze on the basis of a randomly assigned order of assignment to treatment rather than whether or not treatment was delivered or not. In the second set of analyses, we look at the causal effect among certain subpopulations (principal strata) to assess the effect of receiving (as opposed to simply being assigned to receive) the legal assistance. The meticulous records kept by the PROJUSTICE legal assistants allow us to understand the exact interventions that were taken on behalf of all of the subjects assigned to treatment. We do not have comparable records for those subjects who were not ultimately assigned to any non-zero dose of treatment. However, we do have a counterfactual for those who were assigned to treatment given the random ordering and the randomized rollout design. The ordering allows for us to compare individuals with different dosages of the treatment. Sources of Data The impact evaluation relies on five sources of original data collected before during and after the impact evaluation. The sources of data and the measures utilized in the analysis of the impact evaluation are as follows: 1. Prison records collected by the PROJUSTICE in late 2015. The implementer collected the electronic lists of detainees from each of the three prisons in which the evaluation was implemented in late 2015. From these lists, we utilize several variables. First, the collection of prisoner names and identification numbers serve to identify the population of possible subjects. The covariates provided in these registers allow us to define the population of eligible detainees and provide the covariates for the quintet blocking exercise. 2. Survey “census” data. As part of the baseline and endline surveys, enumerators combed through the prison and prison registers to locate each individual in the experimental sample. This allowed them to record the disposition (detained/freed/transferred etc.) of a prisoner at a given point in time. These records are critical for two reasons. They allow us to assess the validity and legibility of the prison records upon which the sample is defined at the baseline. More critically, the endline record of individual dispositions provides our main outcome variable of interest, namely whether or not a prison detainee was released from pretrial detention in the nine months between the beginning of the legal assistance treatment and the endline data collection. 3. Survey data. The baseline survey provides a rich set of baseline covariates about the detainees in the experimental sample. The endline survey provides a number of descriptive measures of the prison and detainee conditions as well as some outcome measurements. 4. Case files by PROJUSTICE. We obtained the case files compiled by the legal assistants employed by PROJUSTICE during the impact evaluation period. These DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 17 provide rich records of the actions of the lawyers as well as case advancement during this 11-week impact evaluation period. Note that these records are only available for 505 detainees who were assigned to treatment during this period (out of a total of 1080 names in the experimental sample). However the randomized rollout design provides a valid counterfactual for these 505 individuals. 5. Administrative case files collected in the Courts of First Instance (CFI). During the endline data collection, the survey firm hired several lawyers to go into the courts of first instance in Port au Prince and Croix-des-Bouquets to collect as much administrative data on the cases as possible. This activity proved particularly challenging because a number of case files were not located. This is consistent with the experience of the PROJUSTICE lawyers who, in many cases, had to ask court officials to reconstitute the case files. This data, however, provides a measure of the legibility of court documents. Figure 7 documents the sequencing of data collection as well as the temporal coverage of each source of data. Figure 7. Sources of data and sequencing of data collection during the impact evaluation. OUTCOME DATA Outcome Data Collected The outcome data comes principally from three sources. Our main outcome measure comes from the measure of an individual’s disposition in the endline “census” at nine months after the start of the treatment. This measure was collected by survey enumerators within the prisons and allows us to assess who was physically in prison at this later date and who had been released. This variable was coded on the basis of enumerators’ ability to actually see and talk to a person in prison as well as the records maintained by prison administrators. This variable represents release from DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 18 pretrial detention. There are three procedural avenues through which this could occur. First, charges could be dropped before the case was sent to a judge or trial. Second, a case could be sent to a judge or to trial and the defendant could be deemed not guilty and thus released from prison. Or, third, a detainee could be convicted; however, via Loi Lespinasse (Lespinasse Law) she could be given credit for time served and released from prison. Our final source of outcome data comes from the PROJUSTICE case files. We look at advancement during the 11- week period in which legal assistance was administered to measure whether legal assistance accelerated the proportion of cases achieving procedural advancement during this period. This serves as a check on the logic underlying the main results on release from PTD. Finally, we utilize some data from the endline survey for descriptive analysis only. Given differential response rates that are correlated with treatment assignment, there are significant challenges in utilizing survey outcome data aside from endline measurement. Response rates are largely determined by whether a detainee was imprisoned in the three prisons (PN, PV, and CdB) at endline, which we show was affected by assignment to treatment. As such, we use this data sparingly. For one component of this project we do utilize descriptive data from this endline survey about household assets, arguing that the endline data provides a lower bound on several sample characteristics under plausible assumptions. Attrition Attrition refers to the missingness of outcome data. In general, attrition represents a major challenge to the analysis of both experimental and nonexperimental data. If missingness is correlated with the potential outcomes—those outcomes we would have observed for each unit under each treatment condition—attrition is apt to bias estimates of treatment effects. In this study, there were two forms of attrition. One serves as a nuisance and one is more pernicious. The first form of attrition is simply due to inaccuracies in the prison registers from which the population and sample were defined. For example, in some cases, the prison register was inaccurate and depicted a detainee as still in pretrial detention when they had actually been released. We cannot collect baseline and endline data for that detainee. However, because this sort of missingness occurred before and independently of a random assignment process, this form of attrition will not bias the estimates of treatment effects in the impact evaluation. This missingness implies some loss of statistical efficiency, as the sample sizes is effectively 15% lower than the total experimental sample. The second form of attrition is much more pernicious. This type of attrition deals with missingness of the endline data that is a function of the status of the case at the time in which endline data was collected. Our estimates show that this status is a function of treatment assignment given evidence that legal assistance reduces the probability that an individual is detained illegally nine months after the beginning of the DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 19 treatment period. As such, conditioning the sample on the availability of endline data is likely to bias our estimates of treatment effects. Most obviously, an examination of re-interview rates of detainees who were imprisoned versus freed at the endline survey reveals drastic differences. Among those who were still imprisoned, the re-interview rate was 85.4%. This figure is quite high relative to the re-interview rates in other panel surveys conducted in Haiti. However, among those released from prison at the endline, enumerators were only able to re-interview about 9% of those individuals. Because an individual’s detention status at the endline is a function of legal assistance, we cannot claim to estimate unbiased estimates of treatment effects on the survey outcomes. Given these two forms of attrition we use language that carefully addresses the threats to inference induced by attrition and clearly identifies the presence or absence of attrition in the outcome data analyzed. DATA COLLECTION AND QUALITY CONTROL Table 2 identifies the actors tasked with data collection and quality control by data source. In general, the only data collected by PROJUSTICE was the baseline prison records prior to the start of the impact evaluation. The sampling strategy and baseline census allow us to measure the accuracy of these records. Moreover, as baseline covariates, any noise or inaccuracy in this data is, by construction, independent of treatment assignment. Table 2. Actors responsible for data collection and cleaning. Source of Data Type of Variables Data Collected by: Quality control performed by: Prison registers Pretreatment covariates PROJUSTICE Academic PIs using baseline census data Baseline survey, census Pretreatment covariates Papyrus (Survey Firm) enumerators NORC Case records Compliance measures, outcome measures Academic PI’s Academic PIs Endline survey, census Outcome measures Papyrus (Survey Firm) enumerators NORC Administrative case files Pretreatment covariates, outcome measures Papyrus-contracted lawyers Academic PIs DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 20 This data collection was conducted in several unconventional environments. In particular, the surveys were conduct (in part) in the prisons in which the intervention was administered. The surveys were conducted with the permission of the Ministry of Justice and the Haitian National Police. Papyrus, the survey firm, sought this authorization prior to data collection. These surveys were conducted using paper survey instruments to avoid bringing tablets into the prison environment. The case records consist of scanned copies of more than 2,000 pages of handwritten files constructed by the PROJUSTICE lawyers. The academic PIs scanned these records after the intervention concluded. They coded core information on the interventions and legal assistance provided. From these records, case advancement data has been collected and recorded as a set of compliance measures. This coding was conducted without reference the treatment indicator so any measurement error should be orthogonal to (independent of) treatment assignment. This reduces the threat of systematic measurement error. The administrative case files from the Courts of First Instance were collected by three lawyers hired by Papyrus. Data was recorded on a survey form on tablets. The data was then cleaned by NORC. 2. KEY FINDINGS CHARACTERISTICS OF PTD IN HAITI Detainee Characteristics We begin with a descriptive analysis of the conditions of prolonged PTD in Haiti. We use a random sample of individuals detained at least six month and not accused of rape, human trafficking, or drug trafficking. As such, the results from this survey provide a descriptive picture of the detainees and their experiences in prison. Given the different probabilities in the prisons, we utilize sampling weights to characterize the situation of eligible detainees in these analyses. Consistent with expectations about the economic costs of illegal imprisonment, 67.5% of the eligible detainees report having at least one child dependent (mean of 1.72 children, with a maximum of 21 children).4 Given that the eligible sample was disproportionately male (97.6%) and male breadwinners are dominant in the Haitian economy, there is suggestive evidence of substantial negative externalities to the families of detainees. In order to gauge socioeconomic status, we compare self-reports of pre-detention household assets from the endline survey to self-reports of household assets from the 2012 DHS survey. The DHS sample is a subset of the Port au Prince metropolitan area for the purposes of comparison. This comparison suggests that, if anything, the pretrial detainees located at the endline (mostly still imprisoned) report somewhat higher levels of household assets than the 2012 DHS respondents. Figure 4 This question asked how many children a detainee was “responsible for.” It correlates highly with the number of reported biological children 𝜌𝜌 = 0.846. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 21 8 provides a comparison of proportions of households with seven services/assets: electricity, radio, television, refrigerator, bicycle, motorcycle, and car/truck. Across all categories, the reported ownership levels among detainees are slightly higher than among the DHS respondents. This comparison merits several caveats. First, overall levels of household assets may have increased between 2012 and 2016. Second, the post-treatment endline sample is no longer a representative sample of the prison, though if anything, we may expect this sample to over-represent poor respondents. If our qualified assessment that prolonged PTDs are as rich as or richer than the average Haitian household, it presents several implications for consideration of PTD in Haiti. To the extent that resources determine access to legal assistance and/or extralegal bribes as a way out of prolonged PTD, our findings suggest that a very large segment of the Haitian population would be vulnerable to PTD if arrested. This suggests that access to free legal assistance is a critical need for much of the population. In the baseline survey, respondents described their trust in Haitian government institutions. Unsurprisingly, most respondents voiced high levels of distrust for these institutions. There is a weakly negative association between expressed trust and length of detention, though the association is not significant at standard thresholds. The high rates of “strongly disagree” responses may inhibit our ability to detect changes in trust over time. Given the police administration of the prisons, detainees were not asked about their trust of PNH. The institutions for which average levels of distrust were highest included parliament, the courts, and the justice system. While this finding is not particularly surprising, it posits one mechanism through which illegal detention may erode subsequent trust in Haiti’s political institutions. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 22 Figure 8. Comparison of reported ownership of household goods/assets between the DHS (2012) survey and the endline survey of (ex-)detainees. Proportions are calculated utilizing sampling weights. Figure 9. Detainee’s self-reported trust in several government institutions. “Haitian Democracy” was phrased as “Haiti’s government of the last 30 years.” Proportions are calculated utilizing sampling weights. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 23 Consistent with earlier work on Haitian prisons, detainees describe contracting and/or being diagnosed with various health conditions while in PTD. In the endline survey of detainees, 35.8% reported some diagnosis while in prison. At this point, all remaining detainees had been imprisoned for at least 15 months. Figure 10 graphs the proportion of individuals reporting six diagnoses during their time in prison. This analysis requires two caveats. First, the detainees imprisoned at endline form a non￾random sample of the experimental population, so estimates may not be “representative” of the population of detainees. Second, we lack an appropriate counterfactual for rates of diagnosis had a detainee not been imprisoned. While diagnosis rates of HIV/AIDS exceeding 2 percent are quite high in a short period (relative to Haiti’s HIV prevalence), other estimates are harder to benchmark. Finally, detainees seem to view legal assistance as the most desired route out of prolonged PTD. In the baseline survey, detainees were asked how a family member could have helped their situation (prolonged PTD). While the modal respondent did not respond to the question, the most common response among those that responded (n = 215) was to “hire a lawyer” (154/215 respondents).5 The second most frequent response was to “pay a judge” (34/215 respondents). There may be some social desirability bias limiting respondents’ willingness to engage in corruption. However, the widespread desire for legal representation nevertheless suggests substantial demand for legal assistance. Figure 10. Rates of diagnosis during PTD, as self-reported reported in the endline survey. Proportions are calculated utilizing sampling weights. 5 We interpret the high rate of non-response as a perception that all available options had been exhausted by family members. This interpretation suggests a perception of helplessness among detainees. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 24 Analysis of Compliance In experimental research, non-compliance refers to instances when the treatment administered does not match treatment assignment. In this context, non-compliance refers to individuals who were assigned to treatment but could not be treated. In conventional analyses, non-compliance serves as a nuisance that can be accounted for in analysis.6 In the present study, however, the case records compiled by PROJUSTICE lawyers provide quite detailed records on the characteristics of cases assigned to a non-zero dosage of treatment. We encounter one-sided noncompliance in the impact evaluation when detainees assigned to treatment and cannot be treated, for any number of reasons. These reasons provide a descriptive picture of the barriers to the administration of legal assistance. To the extent that we seek to estimate the effect of legal assistance on case outcomes, this accounting is informative in quantifying the reasons for which legal assistance may not be feasible in the current setting. Table 3 provides a list of the reasons for which legal assistance was not offered to detainees and the frequency for each reason in the experimental sample. Note that in the sampling procedure, detainees who were charged with rape, human trafficking, or drug trafficking were removed from the sample to the extent possible, so this table understates the proportion of detainees charged with this offense. Given the random sampling employed to select the sample prior to the baseline, these figures provide estimates of the prevalence of each characteristic in the PTD population within these three prisons. Table 3. Reasons for non-compliance among the 505 individuals assigned to be treated. Data comes from PROJUSTICE case records. Reason for non-compliance Frequency Accused of ineligible infractions (rape, drug trafficking, human trafficking) 18 (3.6%) Case has multiple defendants 52 (10.3%) Case proceedings in a different jurisdiction 24 (4.8%) Detainee represented by private lawyer 15 (3.0%) Detainee represented by different aid organization/NGO 4 (0.8%) Detainee already judged or liberated 40 (7.9%) Detainee not found in prison or in court records 2 (0.4%) Detainee died in prison 2 (0.4%) Total 157/505 (31.1%) 6 If the association between a treatment assignment and take-up is sufficiently weak, such problems become more than a nuisance. Additional issues may be imposed by two-sided noncompliance, which did not occur in the present study. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 25 In general, our analysis of noncompliance suggests that there are very few other organizations providing substantial legal assistance within these jurisdictions. While 15 of the 505 subjects assigned to treatment across three jurisdictions were represented by a private lawyer (2.9%) only four detainees were represented by different aid organizations or NGOs (0.8%). These low rates of representation suggest that either (i.) that detainees in prolonged PTD do not have the funds to procure a private lawyer or (ii.) that detainees choose not to hire a lawyer. Qualitative accounts combined with the expressed desire for representation in the survey suggest the former is a more plausible explanation for the low rates of legal representation. This indicates a great need for the provision of free legal assistance, whether by the government of Haiti or by organizations like USAID. Aside from those that could not be represented under USAID guidelines, there are two principal rationales for which legal assistance could not be rendered. First, case characteristics make some cases much more difficult for a small legal assistance program like PROJUSTICE to represent. First, detainees are transferred from one prison to another with some regularity. When a detainee is not in the same jurisdiction as the court processing her case, legal representation is much more difficult. It likely would take a coordinated effort for lawyers to interface across jurisdictions. From a purely practical point of view, it is more difficult to transport detainees across jurisdictional boundaries for trials or investigation. In a low￾resource setting, this poses a particular concern. One programmatic decision made by PROJUSTICE in consultation with the academic PIs at the beginning of the evaluation was not to take on cases with multiple defendants if (a) there were more than five defendants or (b) if not all co￾defendants were in custody. In the experience of PROJUSTICE administrators, these cases are notoriously slow and may fail to progress through the criminal justice system. Protecting the rights of those accused in such offenses should be a priority for reform to the criminal justice system. A second reason for non-compliance relates to inaccuracies in the baseline prison records. Such inaccuracies encompass three reasons for non-compliance: detainees that were: judged/liberated prior to the baseline survey, those that died in prison, and those that were never found. While some individuals were released from prison in the two months between the collection of prison records, we do not have reason to believe that the 9% of cases (44/505) were processed in these two months. State of Prison and Court Records Our data provides evidence as to the state of recordkeeping in Haitian criminal justice institutions. To the extent that the processes that we study require paper records, our ability to quantify and describe the state of these records provides some evidence of the extent to which the illegibility of such records poses barriers to the resolution of the PTD situation in Haiti. In its other activities, PROJUSTICE implemented case-management information systems (CMIS) in several other DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 26 jurisdictions in Haiti, one possible solution to the problem of inaccurate or outdated prison or court records. The baseline census allows us to evaluate the state of the prison records collected at baseline. In general, about 18.9% of the detainees from November 2015 were not in the prisons at baseline in January 2016. Our records indicate that control group detainees (those not assigned to legal assistance) that absent legal assistance, approximately 1.6% pretrial detainees are released from prison per month (95% CI: [0.0074, 0.0258]). Accounting for detainee releases in the two months between collection of prison records and the baseline, we estimate that approximately 16% of the individuals in prison registers are not presently held in prisons. We cannot estimate the share of detainees in prison that do not appear in prison registers, though inaccuracy in this regard is likely rarer, based on qualitative observation. A digitized CMIS in the prisons particularly at booking (as opposed to in a separate office) could help to reduce the inaccuracies in these records. Records from the intervention as well as administrative data collected during endline data collection allow us to assess case findability within the CFIs. First, one of the most common interventions made by PROJUSTICE lawyers in the CFIs was to request that case files be “reconstituted.” Moreover, case record “findability” during the endline data collection was limited. The lawyers collecting data were able to locate only 557 of the case files of the 876 individuals (63.6%) in the prisons at baseline. There was no effect of treatment on findability given two countervailing effects. It was more difficult to find the cases of liberated defendants, which we show was more likely in treatment. This measure is constructed by comparing the found￾case files to the endline “census” in the prisons. However, given PROJUSTICE’s efforts to reconstitute missing case files, the proportion of missing case files was higher in control. The association between baseline characteristics (upon which the blocks are constructed) are not prognostic of case missingness. Regardless, given the state of records in the CFIs, investments in CMIS could greatly improve the legibility of records. THE CAUSAL EFFECTS OF LEGAL ASSISTANCE Here we present evidence that legal assistance reduced the proportion of detainees imprisoned nine months after the implementation of the impact evaluation. We estimate two types of estimands (statistical quantities of interest): ITTs and LATEs. We estimate ITTs because not all subjects assigned to legal assistance received it. For the reasons enumerated in Table 3, some subjects assigned to receive the legal assistance treatment did not receive it. The ITT includes a single indicator measuring assignment to treatment. We operationalize this assignment to the treatment variable in two ways, as described in Table 4. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 27 Table 4. Two formulations of the indicator measuring assignment to treatment. Note that represents the prison-specific quantile function and is the order assigned to a detainee in prison . Assignment Indicator Range Sample Interpretation of Relevant Estimates Continuous [0, 1] Full sample The difference in probability of being released between the detainee that was assigned to receive legal assistance first versus the detainee that was assigned to receive legal assistance last in a prison (block). Binary {0, 1} Two prisons: PN, CdB The average difference in probability of being released between a detainee assigned to receive any dose of legal assistance versus a detainee not assigned to receive legal assistance. We adopt two different formulations of treatment assignment to capture both dimensions across which our treatment varies. First, the randomized rollout design offers variation in the dosage of treatment administered, which serves as a continuous treatment variable (CTV). The continuous measure is constructed as where is the quantile function that returns the percentile of a detainee’s order within a prison. This transformation is utilized to put the assignment variable onto the same scale in three prisons that vary in size. Using this function, this variable takes on values between 0 and 1. We exploit this variation across the whole sample (all three prisons). The binary measure takes advantage of the exogenous end date of PROJUSTICE legal assistance after which legal assistance was not offered to detainees. Note that because all detainees in PV were assigned to (some non-zero dosage) of treatment, this indicator cannot be used in the analysis of data from PV. Given the differing rates at which the legal assistants processed cases in PN and CdB, we utilize inverse probability weighting (IPW) in all specifications using the binary treatment indicator. The weights are simply calculated to be where rp was the probability of assignment to (any non-zero dosage of) treatment in prison p. These weights account for the differential probabilities of assignment to treatment across these two prisons. Intent to Treat Effects All ITT estimates are estimated by Ordinary Least Squares (OLS) with Huber-White heteroskedasticity robust standard errors. We present four specifications below for each assignment indicator, as follows: 1. Full sample of detainees, no covariate adjustment. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 28 2. Full sample of detainees, adjusting for the quintet blocks utilized in the random assignment process. This adjustment is implemented by including fixed effects for each block in the regression model. 3. Port au Prince jurisdiction prison detainees, adjusting for the quintet blocks utilized in the random assignment process. 4. Croix-des-Bouquets jurisdiction prison detainees, adjusting for the quintet blocks utilized in the random assignment process. All estimates are estimated using the following formula, where the is the estimator of the ITT. Note that is a quintet indicator utilized in specifications 2-4 in the preceding list. It appears in parentheses in the equation because it is used only in specifications 2-4. represents the dependent variable, a binary measure of whether a detainee was released 9 months after the start of treatment. Table 5 presents the ITT estimates. The top panel utilizes the continuous treatment assignment indicator. Here, the treatment should be interpreted as the difference in probability of liberation from going from first in a prison’s assignment to last. In other words, as per column 2, our preferred specification, an individual is 7.3 percentage points more likely to be released if he/she was assigned to receive legal assistance first than if he/she was assigned to receive legal assistance last. While this effect is relatively small, it represents a sizeable change relative to the estimated liberation rate of 13.6% for the last person assigned in each prison. This effect is statistically significant at the level in a one-tailed test. We use a pre-specified one-tailed test. Columns 3 and 4 disaggregate the treatment effects by jurisdiction. Because PN and PV are located in the Port au Prince jurisdiction, they are pooled in Column 3. Here, the ITT is positive, indicating that individuals earlier in the treatment assignment order are more likely to be released from prison than those assigned later in the order. However, these effects are substantively much smaller and are not statistically significant at standard thresholds of significance. In contrast, the effects in the Croix￾des-Bouquets jurisdiction are much larger than the pooled ITT in Column 2. These effects are statistically significant at the level. In Croix-des-Bouquets, moving from first to last in the order of treatment assignment increases the probability of liberation by 15.2 percentage points. The bottom panel of Table 5 provides the estimates of the ITT using the binary variable of assignment to treatment. Note that this analysis only utilizes the individuals from PN and CdB as there was no variation in assignment to treatment in PV using the binary treatment indicator. The results are substantively similar with a larger effect (conditional ITT) in CdB than in PN. This formulation obscures variation in dosage among those assigned to treatment, leading to some loss of information and thus efficiency. 𝛽𝛽1 DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 29 Table 5. ITTs using both formulations of the treatment indicator. Heteroskedasticity-robust standard errors in parentheses. Dependent variable: Liberated, Nine Months Later (1) (2) (3) (4) Order Quantile within Prison 0.061* 0.073** 0.049 0.152** (0.043) (0.039) (0.045) (0.078) Mean DV, Treatment = 0 0.136 Treatment Range [0, 1] [0, 1] [0, 1] [0, 1] Observations 876 876 678 198 Subsample All All PN, PV CdB (1) (2) (3) (4) Assigned to Treatment (binary) 0.021 (0.026) 0.034* (0.023) 0.017 (0.027) 0.107*** (0.042) Mean DV, Treatment = 0 Treatment Range Observations Subsample IPW 0.144 {0, 1} 830 PN, CdB yes {0, 1} 830 PN, CdB yes {0, 1} 632 PN {0, 1} 198 CdB Quintet FE Hypothesis Test DV Scale no Upper {0, 1} yes Upper {0, 1} yes Upper {0, 1} yes Upper {0, 1} Notes: DV=Dependent variable. *p<0.1; **p<0.05; ***p<0.01. 1= Full sample, no covariate adjustment; 2= Full sample, quintet block adjustment; 3= Port au Prince sample, quintet block adjustment; 4= Croix-des-Bouquets sample, quintet block adjustment. Figure 11 provides an additional visualization of the ITT estimates from the top panel of Table 5, with emphasis on hypothesis testing. The four plots correspond to the ITT estimates in each of the four columns of the regression table. The points in each plot represent the point estimate of the ITT from each specification. The full lengths of the lines (any width) provide 95% confidence intervals around the respective point estimate. Given the one-tailed hypothesis, the medium black line represents a rejection region at the level for the one-sided (upper) hypothesis. Where this falls above the horizontal red line (the null hypothesis), an estimate is statistically significant at that level. Similarly, the thick black lines represent the rejection region at the level for the one-sided (upper) hypothesis. Where this region falls above the horizontal red line, the estimate is statistically significant at that level. We reject the null hypothesis of an ITT of zero for the full sample (with quintet FE), which is driven by the large ITT observed in Croix-des-Bouquets. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 30 Figure 11. ITT estimates using the continuous formulation of the treatment assignment indicator. These plots correspond to the top panel in Table 5. Local Average Treatment Effects Further analyses allow for the estimation of the effect of receiving treatment, as opposed to simply being assigned to receive treatment. However, since those detainees that were assigned to and received treatment may be systematically different from those that were assigned to and did not receive treatment, it is inappropriate to compare the observed outcomes of compliers to the control group. Instead, we utilize two-stage least squares (2SLS) as an instrumental variables (IV) estimator. The randomly assigned treatment assignment serves as an instrument for whether treatment was delivered or the dosage of treatment delivered, as indicated in the Figure 9. Figure 12. Causal process implied by instrumental variables estimation strategy. With the present randomized rollout design, there are several potential formulations of legal assistance. First, we use a simple binary measure of whether a detainee received treatment when assigned to receive (any non-zero dosage of) treatment. The second formulation follows Figure 6 and measures the number of weeks that a Assignment to Legal Assistance Legal Assistance Provided Liberated (yes/no) Nine months later DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 31 detainee received legal assistance (between 0 and 10). These formulations and the corresponding estimands/estimators are listed in Table 6. Table 6. Specification of the instrument (treatment assignment) and treatment as used in the IV estimation. Estimand Assignment Variable, Range Treatment Variable, Range Interpretation of Estimates CACE Binary, {0, 1} Binary, {0, 1} The average difference in probability of being released between a detainee assigned to receive any dose of legal assistance versus a detainee not assigned to receive legal assistance, among the complier causal type only. LATE Continuous (calculated by the quantile function as above) [0, 1] Discrete—weeks of legal assistance given, [0, 11] The marginal effect of an additional week of legal assistance on the probability that a detainee will be liberated nine months later. We utilize the same four specifications as in the ITT estimates above. The estimates are presented in Table 7. The top panel estimates the CACE. This is the treatment effect among compliers. As before, we estimate this effect only among the detainees in PN and CdB, as there is no variation in the instrument in PV with the binary formulation. As before, all estimates are positive, indicating that treated detainees are more likely to be liberated within nine months. The effect is particularly pronounced in CdB (Column 4). This effect is substantively large relative to the baseline rate of liberation and is statistically significant at the level. The second panel estimates the marginal effect of additional week of treatment. Here, the effects are consistent with the other findings that legal assistance increases the probability that a detainee exits pretrial detention within nine months of the start of the impact evaluation. Indeed, the estimated marginal effects of an additional week of legal assistance are quite pronounced. Moving from receiving no legal assistance to receiving ten weeks of legal assistance leads to a 12-percentage point increase in the probability of being liberated, nearly doubling the baseline rate of liberation. Moreover, the effect is much larger in Croix-des-Bouquets. In this jurisdiction, receiving ten weeks of legal assistance increases the probability of being liberated by 24 percentage points. These effects are substantively and statistically significant at conventional thresholds. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 32 Table 7. Estimates of the CACE (top panel) and a LATE of the legal assistance treatment. Taken collectively, these analyses indicate that legal assistance increased the probability that detainees were released from prolonged PTD nine months after the beginning of the impact evaluation. The estimated effects in Croix-des-Bouquets are particularly large. There exist many possible explanations for these findings. First, Figure 3 indicates systematic differences in the duration of PTD between different prisons. More recent cases in CdB may be easier to investigate and process. Second, the CFI in Croix-des-Bouquets may simply be more effective in processing cases than the CFI in Port au Prince. Third, the PROJUSTICE staff may simply be more efficacious in CdB. PROJUSTICE administrators, however, never indicated that the legal assistants in CdB were more effective. A wide range of alternative explanations exist, though qualitative evidence suggests that institutional differences between the jurisdictions may influence the efficacy of legal assistance in reducing prolonged PTD. Dependent variable: Liberated, Nine Months Later (1) (2) (3) (4) Detainee Treated (binary) 0.028 (0.035) 0.046* (0.032) 0.023 (0.036) 0.160*** (0.065) Mean DV, Treatment = 0 0.144 Treatment Range {0, 1} {0, 1} {0, 1} {0, 1} Instrument Range {0, 1} {0, 1} {0, 1} {0, 1} Observations 830 830 632 198 Subsample PN, CdB PN, CdB PN CdB IPW (by prison) yes yes no no (1) (2) (3) (4) Weeks of Legal Assistance (count) 0.010* 0.012** 0.009 0.024** (0.007) (0.007) (0.008) (0.013) Mean DV, Treatment = 0 0.136 Hypothesis Test Upper Upper Upper Upper Treatment Range [0, 10] [0, 10] [0, 10] [0, 10] Instrument Range [0,1] [0, 1] [0,1] [0,1] Observations 876 876 678 198 Subsample All All PN, PV CdB Quintet FE no yes yes yes Hypothesis Test Upper Upper Upper Upper DV Scale {0, 1} {0, 1} {0, 1} {0, 1} Notes: DV=Dependent variable. *p<0.1; **p<0.05; ***p<0.01. 1= Full sample, no covariate adjustment; 2= Full sample, quintet block adjustment; 3= Port au Prince sample, quintet block adjustment; 4= Croix-des-Bouquets sample, quintet block adjustment. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 33 The qualified assessment that there exist institutional differences that condition the efficacy of legal aid to reduce prolonged PTD posits several implications for future efforts to reduce the prevalence of prolonged PTD in Haiti. First, legal assistance alone is not likely to eliminate the incidence of illegal PTD alone. Legal assistance seems to interact with the institutional environment at the Courts of First Instance. To this extent, efforts to reduce absenteeism and corruption within the courts may improve the efficacy of legal assistance. The issue of limited legibility appears to pose a related obstacle to the reduction of prolonged PTD. To the extent that much of lawyers’ time was spent finding files in the CFI, efforts to improve the legibility of such records may improve both the efficiency and efficacy of legal assistance within the Haitian courts. To this extent, the efforts of PROJUSTICE to implement electronic case management information systems (CMIS) in some jurisdictions may substantially improve the efficacy of legal assistance. However, given the state of records in these courts, the start-up costs of inputting all backlogged files and assessing accuracy are formidable. Absent validation in both the court and prison, the records in the CMIS are unlikely to be highly accurate. Further efforts to harmonize records and practices at the courts and prisons would be well advised. The final barrier to the ability of legal assistance to reduce prolonged pretrial detention relates to the non-compliance experienced in the impact evaluation, specifically the limitations on the legal assistants’ ability to treat some of the individuals assigned to treatment. To this extent, two policies may improve the ability to provide legal assistance to all detainees. First, larger jurisdictional scope of legal assistance would ensure adequate representation for detainees who are held in prisons within a different jurisdiction from their case files. Second, a reform to policies for prosecuting cases with multiple defendants is necessary to accelerate the pace of such cases, particularly when some defendants have not been arrested. COSTS AND BENEFITS OF THE PROVISION OF LEGAL ASSISTANCE Using the results from the impact evaluation, we now conduct a cost-benefit analysis of the provision of legal assistance. This accounting of costs and benefits is unconventional because the costs accrue to USAID in the form of the provision of legal assistance while the benefits accrue to the government of Haiti (GoH) in the form of reduced burden for “room and board” of illegal detainees in Haitian prisons. The purpose of this cost-benefit analysis is to provide an estimate of the benefits that could accrue to the government of Haiti via the institution of a public defender’s office. As such, we consider only the costs of staffing of legal assistants, not PROJUSTICE administration costs. Costs of Legal Assistance The costs of employing legal assistants during the intervention program is easily quantifiable. PROJUSTICE employed three supervisors at a monthly wage rate of DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 34 $850 (USD) and fifteen “interns” at a monthly wage rate of $650 USD. The intervention period spanned nearly three months, including the trainings on how to proceed through the list of detainees. The costs of this staffing level for three months are thus $36,900. The estimated costs for housing a prisoner for one month in Haitian prisons is $40 per detainee. As such the “break even” point at which the staffing costs accrued to USAID are equivalent to the cost savings to the GoH is thus 922.5 detainee-months. Savings in Detainee-Months The central challenge of the cost-benefit analysis is to estimate the number of detainee-months reduced by the legal assistance intervention. In general, this is not possible without assumptions about the data generating process underlying the process of release. We assume that release in both control and treatment groups can be modeled by a stochastic process approximated by a binomial distribution. Among the detainees present in the first “census” at baseline, we know the proportion of detainees that have been liberated by the endline “census.” We assume that each detainee has a baseline probability of release in each week. The effect of treatment serves to augment this probability. The proportion of detainees that were present in the baseline and still detained at the endline can thus be modeled as: Where represents the probability of release in a given week and is the number of weeks between baseline and endline. In this case and is directly estimable from our regression estimates. We can thus solve for , the key parameter in our simulation, under both treatment and control using simple algebra: Our simulation utilizes the ITT estimates with a binary treatment variable (Table 5 panel 2), as a conservative estimate of the effect of legal assistance, effectively averaging over the varying dosages of legal assistance. We conduct 10,000 simulations of the following process: 1. Draw an estimate of under both control ( ) and treatment ( ) from Column 2 of Table 5, panel 2. Because is estimated, we must incorporate uncertainty into this simulation. By drawing a unique estimate in each simulation, we incorporate this uncertainty into the simulation of detainee months saved. 2. For both and , estimate the associated and Using these estimates, we simulate the week of release from detention for 1000 hypothetical detainees. This simulation consists of a draw of 260 samples from a binomial distribution with the parameter 𝑝𝑝 = 𝑝𝑝𝑐𝑐 or . The position of the first “1” in each DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 35 sample corresponds to the week of release. We censor at 260 weeks (5 years) as necessary (rarely). This is roughly consistent with the observed data. 3. Define the average week of release of the 1000 control and treatment simulations as and . The simulated “treatment effect” in terms of reduced weeks in detention is simply: . Where this quantity is negative, treatment reduces the average number of weeks spent in detention. 4. Convert this difference to months by dividing by The resultant outcome generates a distribution of the average difference number of months of detention saved per detainee over a horizon of five years. This distribution incorporates uncertainty over the estimates that generate and as well as variation induced by random draws from a binomial distribution. In order to make a final conclusion about the cost effectiveness of the intervention as implemented, we multiply each estimate of the difference in months by the $40 monthly cost per detainee and by the 324 detainees in the baseline sample that lawyers attempted to treat. From this quantity, we subtract the $36,900 cost of staffing to calculate the “net” benefits of the program. Figure 12 provides the results of these simulations. The top panel depicts the reduction in prison time. We estimate that the treatment that was assigned during the impact evaluation (any dosage) reduces the number of months a detainee spends in detention by an average of 3.7 months with a 95% confidence interval: [ -8.73, 1.67]. The mean is indicated in the graph with the red vertical line and the 95% confidence intervals are indicated by the blue dotted line. The bottom panel depicts the net savings to the GoH. This reduction in time spent in detention corresponds to an average net savings of $170,785 with a 95% confidence interval of [-130,498, 453,3912], with 87.49% of simulations producing net savings. The mean is indicated in the graph with the red vertical line and the 95% confidence intervals are indicated by the blue dotted line. These effects are purposefully calculated to be conservative. First, the ITT upon which this is calculated excludes PV since all detainees were assigned to treatment. Second, we calculate savings based on ITT estimates rather than LATE or CACE estimates that estimate effects among those that received treatment. Third, the analyses suggest that efficacy of legal assistance is increasing in the dosage. Given the short time period of the legal assistance program, the effects for those assigned to a week or two of treatment are smaller than those assigned a higher dose. In a longer-term legal assistance program, savings will be greater than those calculated here. We can conduct a similar exercise with the LATE estimates from Table 7 (column 2). While these estimates require some additional extrapolation from the estimates than in the previous simulation, they suggest even more substantial cost savings. These effects suggest that that accrued savings are substantially higher with longer treatment periods. With 10 weeks of treatments, this analysis suggests a reduction in detention of 11.41 months with a 95% confidence interval of [-20.73, 0]. With this average, the employment of a supervisor would hypothetically be cost-neutral with a DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 36 caseload of 6 cases. This caseload is much smaller than the caseloads in the present study, which averaged 20 during the experimental period. Figure 13. Simulated reduction in months of detention (top panel) and net benefits of legal assistance (bottom panel). The red line is the mean of the estimates and the blue dotted lines indicate a 95% confidence interval. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 37 Conclusion and Recommendations Illegal PTD represents a significant threat to human rights in Haiti. This impact evaluation examines the impact of USAID-funded legal assistance administered by the PROJUSTICE program on detainees’ case trajectories. By aggregating average effects estimated within the RCT framework, this impact evaluation allows for an assessment of the impact that a more widespread legal assistance program could have on prolonged PTD in Haiti. The impact evaluation quantifies the scope of illegal PTD in two Haitian jurisdictions, arguing that the government of Haiti and aid donors must urgently address the issues associated with PTD in Haiti. We conclude with four lessons and related policy recommendations stemming from this impact evaluation. 1. Widespread free legal assistance is critical to reducing PTD in Haiti. Survey responses show that detainees favor legal assistance (“hiring a lawyer”) as the most preferred antidote to prolonged PTD. Nevertheless, very few of those that are illegally detained have access to a lawyer. The impact evaluation documents that USAID-provided free legal assistance reduces the probability that a detainee remains in illegal detention after nine months. Nevertheless, to adequately represent all those that remain in illegal detention, a larger program with a larger jurisdictional reach is necessary to ensure that all detainees can access this representation. In principle, the most coherent way to organize such a service would be through government of Haiti-funded public defenders. 2. Legal assistance is necessary but not sufficient to eliminate illegal PTD in Haiti. While this report documents positive effects of PROJUSTICE legal assistance, the estimates suggest that legal assistance is not likely to overcome high rates of prolonged PTD alone. First, the majority of detainees imprisoned at the baseline were still imprisoned at the endline. While the short legal-assistance period (10 weeks) may be relatively “weaker” than a long-term administration of legal assistance, this rate is still revealing. Second, there appears to be some heterogeneity of effect sizes on the basis of jurisdiction (Port au Prince vs. Croix￾des-Bouquets). While the difference in ITTs/CACE/LATEs is not statistically significant at conventional levels, it is consistent with qualitative evidence characterizing the difference in jurisdictions. This suggests that institutional features may condition the effectiveness of legal assistance. 3. Investment in recordkeeping may complement legal assistance in addressing PTD. Poor recordkeeping in the CFIs and prisons may delay the timely processing of cases, thus increasing rates of prolonged PTD. This report quantifies some of the shortcomings of the recordkeeping in these settings. To the extent that CMIS systems can improve the legibility of such records, such investments may also help to reduce the prevalence of prolonged PTD. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 38 4. Reform of criminal law may help to alleviate excessive rates of illegal PTD in Haiti. One of PROJUSTICE’s other areas of work was advocacy for legal reform, particularly the reform of Haitian criminal law. While the present impact evaluation does not speak to the efficacy or content of those efforts, it suggests some areas for improvement. In particular, it is important to develop better practices for prosecuting and processing cases with multiple defendants. In a setting where capabilities to find and arrest individuals accused in a multi-defendant case is limited, there must be processes implemented to protect the rights of detained co-defendants. To the extent that these individuals were not treated during the impact evaluation on these grounds, legal reform must address these procedural difficulties. DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 39 ANNEX A, DESIGN REPORT Impact Evaluation Design: USAID Haiti PROJUSTICE Program PTD Companents Tara Slough∗ Christopher Fariss† July 29, 2015 Abstract Since its creation in 2009, USAID’s rule of law program in Haiti, PROJUSTICE, has worked to enhance justice delivery as a basis for establishing stability and security and improving citizens’ confidence in Haitian government institutions. One activity of this program has been the provision of legal assistance in order to reduce the rate of prolonged pre-trial detention (PTD) in Haitian prisons. Prolonged PTD constitutes a prevalent human rights abuse in Haiti: over 70% of individuals in Haitian prisons are held in this illegal form of detention. This impact evaluation seeks to evaluate the effectiveness of PROJUSTICE’s legal assistance program in prisons in Port au Prince and Croix des Bouquets. Given the large populations of pre-trial detainees in these jurisdictions, PROJUSTICE legal assistants will be unable to provide assistance to all detainees before the expiration of the program in 2016. By randomizing the order in which legal assistance is provided to current pre-trial detainees, we will be able to estimate the causal impact of PROJUSTICE legal assistance on the trajectory of the cases of those subjects held in prolonged PTD and quantify the impact of legal assistance on rates of PTD in Haiti. We also intend to track a number of secondary outcomes through baseline and endline surveys. Overall, the results from this impact evaluation will be of interest to local stake holders in civil society and the government, such as the Ministry of Justice and the Superior Council of the Judiciary (CSPJ), in addition to the academic community and the many human rights advocate, monitoring groups, and NGOs that work on similar issues of justice internationally. ∗Columbia University, tls2145@columbia.edu †Penn State University, cjf20@psu.edu 1 Introduction Since its creation in 2009, USAID’s rule of law program in Haiti, PROJUSTICE, has worked to enhance justice delivery as a basis for establishing stability and security and improving citizens’ confidence in Haitian government institutions. To date, the project’s efforts have been concentrated in target districts of Croix-des-Bouquets, Saint Marc, Port au Prince, Fort Liberté, and Cap-Haïtien.1 PROJUSTICE aims to improve the productivity and efficiency of the Haitian justice system. The program has four components: (1) access to justice through provision of legal aid, legal education, and mediation; (2) reduction of illegal pretrial detention (PTD) through provision of assistance to detainees in PTD; (3) promotion of an independent judiciary; and (4) advocacy for legal reform. One component of this program, the provision of legal aid to decrease illegal pre-trial detentions, will be the focus of this impact evaluation. International estimates suggest that, as of April 2013, 70.6% of Haiti’s prisoners nationwide, or 7,019 individuals, were held in pre-trial detention (Walmsley, 2014).2 In some districts, the proportion of prisoners in pre-trial detention is much higher than the national average. For example, in Port au Prince, the capital of Haiti, nearly 90% of pre-trial detainees are in pre-trial detention and lack the means to pay for a private attorney.3 The Haitian Constitution outlines the procedure for penal and correctional cases with maximum allowances of time for each step. For example, the Constitution requires that detainees receive a hearing within 48 hours of their arrest and be provided with a lawyer. The subsequent steps in criminal procedure have similarly specified time frames. In practice, however, these rights are frequently violated particularly for those prisoners that cannot afford a private attorney. In these cases, prisoners are in prolonged or illegal pretrial detention. These forms of detention represent fundamental human rights violations (Mayerfeld, 2016; de Zayas, 2005). Moreover, the volume of pre-trial detainees exacerbates overcrowding in Haitian prisons (United States Department of State, 2014). USAID’s PROJUSTICE program has provided free legal assistance to prisoners that cannot afford a private attorney in order to combat prolonged pretrial detention in two ways. First, lawyers employed at PROJUSTICE legal assistants provide legal assistance to individuals held in extended illegal pre-trial detention (typically for months or years) in an effort to procedurally advance their cases to final disposition. Second, PROJUSTICE legal teams provide services to those who have been detained in police stations or 1These districts represent five of Haiti’s 18 judicial districts. 2We do not have a more recent nation-wide figure, though we do have more detailed estimates in the five districts where PROJUSTICE is active. Fieldwork from the scoping mission suggests that this figure is likely a lower bound for current levels particularly given the accrual of additional backlogs over the past two years. 3 Interviews at National Penitentiary and the Women’s Prison in Petionville, June 25 and 26, 2015. 2 peace courts (les Tribunaux de Paix ) in order to prevent prolonged pretrial detention where possible. The impact evaluation will evaluate the effects of the provision of this type of legal assistance, focusing on the former activity by PROJUSTICE lawyers. The impact evaluation will ultimately serve two purposes. First, it will provide a rigorous evaluation of PROJUSTICE’s efforts to combat prolonged PTD through the provision of legal assistance. Second, the USAID mission will use the results of the evaluation to inform policymakers about the benefits of free legal aid in reducing PTD. The findings of the IE will be used to encourage government to establish a national public defender’s office to provide free legal services to individuals in PTD. Ultimately, the use of this information as an advocacy tool will aim to improve the human rights record of the Government of Haiti (GoH) with regard to pre-trial detention. 2 Theory The Haitian criminal justice system involves a number of distinct institutional actors. In principle, the involvement of different institutions should serve to create a safeguard against abuses of Constitutional and human rights inside the criminal justice system. However, in practice, limited capacity creates various coordination problems between these institutional actors. These coordination problems prevent the criminal justice system from functioning efficiently and adhering to the timelines for penal/correctional cases set forth by the Haitian constitution. These issues, broadly stated, therefore are responsible for the high rate of prolonged PTD in Haiti. In general, there are four institutional entities involved in the Haitian criminal justice system (below the appellate level) with whom PROJUSTICE legal aid staff interact: the Haitian police (Police Nationale d’Haiti), the Peace Courts, and the prosecutor’s office (Parquet), and the judges working in the Courts of First Instance (les Tribunaux de Première Instance, CFI). In order for the criminal justice system to function effectively, these entities must play complementary roles. When one entity cannot fulfill their responsibilities, the capacity of these institutions is imbalanced, or there is limited communication between these institutional entities. Under these conditions, penal cases tend to stall, which increases the prevalence of prolonged PTD. The Haitian police arrest/detain individuals and manage Haitian prisons. PROJUSTICE legal assistants interact with the police in both of these capacities, whether providing assistance to individuals after their arrest in order to prevent prolonged PTD where it is unnecessary, or in their work with individuals already in detention. The police are not within the judicial system like the other institutional entities with 3 whom PROJUSTICE legal assistants interact. Police records remain distinct from those used in the courts. As such, prisoner records are distinct from case files used in the CFI or Parquet. Various international entities and donors have supported efforts to strengthen the capacity of the Haitian police in recent years through the training of police officers and the construction of new prisons etc. While these efforts have generated a more capable police force, fewer efforts (PROJUSTICE being a notable exception) have been directed at ensuring that the judiciary can handle the higher volume of individuals arrested by a better-trained police force. Thus, capacity-building remains relatively asymmetric; that is, the capacity of the police has increased more than the capacity of the judiciary. In Haiti, the judges of the Peace Courts are the judges of proximity. Specifically, there are Peace Courts in each of Haiti’s 140 Communes in addition to a number of annexes, for a total of 179 courts. In penal cases, these judges play a judicial police-like role, investigating in cases of flagrant delicto. Within three days, the judge in the Peace Court must send information procured during the course of her investigation to the prosecutor in the Parquet. Due to limitations on practical resources, this process often exceeds the three day minimum. Given the visibility of these judges in their local communities, they face incentives to send all cases up the chain of command (to the Parquet) in order to preserve their reputation (and at times personal safety) in the communities where they operate. Limited resources including staffing, electricity, and security impede the efficient function of the Peace Courts in criminal proceedings.4 The prosecutors in the Parquets receive the cases referred by the Peace Courts. Upon receiving a case, the chief prosecutor (Commissaire) assigns each case to a prosecutor. Subsequent to a first decision hearing, the prosecutor determines the trajectory of the case. She could drop the case (classement sans suit), effectively releasing the detainee. If the crime or misdemeanor is committed in flagrant delicto, the case may be forwarded directly to the correctional court. If the crime or misdemeanor is not committed in flagrant delicto, the prosecutor must request an investigation by an investigating judge (IJ). For the remainder of the correctional and penal cases, prosecutors and judges must work in consort. Prosecutors and, by extension, the Parquet as a whole face challenges in undertaking these responsibilities. First, unlike judges, prosecutors have no mandates and are employed at will. This leads to high levels of turnover (at times arbitrary) in the Parquets. This fear of termination provides incentives for prosecutors to be conservative in their determinations. Practically, this corresponds to a low rate of release by prosecutors, which increases the number of individuals held in PTD. Second, in Haiti, prosecutors play a role in all trials: civil and criminal which tends to overstretch an already limited staff. 4 Interview with the Chief Judge of the Saint Marc Tribunal de Paix, June 30, 2015. 4 When a prosecutor submits a case for investigation by the IJ, the request (Réquisitoire d’Informer ) is received by the chief clerk of CFI. Once registered, the case should be forwarded to the Dean (Doyen) of the judges of the CFI for assignment to an IJ. The IJ subsequently has three months to investigate the case unless an extension is approved by the Dean. This three month period is frequently overrun, and absent an extension, an overrun constitutes prolonged PTD. Subsequent to the preliminary investigation, the IJ submits informs the Parquet via an Ordonnance de soit-comminiqué. The judges in the CFI face a number of challenges. First, in many cases, judicial mandates are unfulfilled, leaving the courts understaffed. In particular, when there are too few IJs, cases cannot be investigated and proceed in a timely fashion. Given the incentives of Justices of the Peace and prosecutors to send many cases to the IJ, these officials are often overburdened, even when mandates are fulfilled. Second, there is a lack of resources for trials–particularly those with jury. Beyond the hardships faced by these institutions individually, there is generally a lack of coordination between these offices which exacerbates the delays in the system. Three issues underscore these coordination issues. First, there is a lack of systematized records between the prisons and the CFIs. The first task of legal assistants after identifying a legal aid recipient is to find their identification number in both prison and then Parquet registers in order to track the case. This lack of record synchronization is problematic prior to trials in that it makes it hard to track the prevalence of prolonged PTD. It is also concerning subsequent to investigations and trials when release orders are not forwarded from the Parquet to the prison, inducing other forms of illegal detention. Second, the limited coordination between the Parquet and the judges of the CFI represent another source of delays that elongate the length and thus severity of PTD. In the jurisdictions where prolonged PTD has been reduced dramatically such as Saint Marc or eliminated as in Fort Liberté, Deans and Commissaires consistently credited the collaboration between their offices (or “leadership”) in addition to legal assistance and material support for trials from PROJUSTICE. Given that cases pass from the Parquet to the CFI multiple times, a lack of coordination between these institutions incurs the possibility of significant delays. Institutionally, the incentives for coordination between the Dean and the Commissaire are not particularly strong. The Parquet receives orders from the Ministry of Justice while judges are under the direction of the CSPJ. No official hierarchy exists between the two positions, leading to frequent power struggles and limits to coordination.5 Nevertheless, without coordination between these offices, there is not particularly strong oversight tracking the trajectory of cases, and thus the fate of the accused is compromised. 5 Interview, Dean and Commissaire of Fort Liberté, July 3, 2015 5 Finally, the asymmetric strengthening of some institutions over others limits coordination capacity between these institutions. Multiple informants attribute an increased number of arrests to the significant resources and capacity-building directed to the police by international agencies and donors. Without similar investments in judicial institutions, a backlog of cases has ensued, increasing the prevalence of prolonged PTD. Within the CFIs, when judicial mandates are not renewed, coordination between the Parquet and the judges becomes much more difficult due to the limited judicial staffing, slowing the speed of cases. 2.1 Theory of Change The provision of legal assistance for detainees in prolonged PTD reduces the duration and thus the prevalence of this form of illegal detention by alleviating some of the aforementioned coordination problems. PROJUSTICE legal assistants’ work with detainees bridges the divide between prisons and the CFIs. Conventionally these lawyers identify individuals in prolonged PTD in one of two ways. First, they identify individuals from prison or court files: if a case has not advanced appropriately, the associated files lack documentation of these advances. It is through this method that we will identify subjects for the purpose of the impact evaluation. The other method of identifying detainees is through the advocacy of detainees or their families. As PROJUSTICE staff are known in courts/prisons and certain communities (particularly those where legal clinics operate), staff regularly collect the names of detainees. In either case, by gathering both the identification number in the prison and the court, the legal assistants overcome some of the coordination problems between the prisons and the CFIs by gathering this information. This provision of information helps to inform the Parquet, in particular, which individuals are illegally detained. At the conclusion of a case–whether this results in release or conviction–the Parquet and prison must again interface so that the prison can carry out the court’s determination. If a prisoner is slated to be released, the release order signed by the Commissaire must be received by the prison in order to release the detainee. If the prisoner is convicted, the prison must have documentation of the conviction and sentence in order to release the detainee at the appropriate time. When coordination between the Parquet and prison fails at this level, illegal detention is again a threat. To the extent that a legal assistant can ensure that this document arrives in a detainee’s records in a prison, legal assistance further helps to alleviate the consequences of the lack of coordination between the prison and the Parquet. More critically, in CFI environments where coordination problems exist between the Parquet and the judges, PROJUSTICE legal assistants help to overcome these problems. We posit that legal assistance reduces the costs of coordination through the provision of information. Most practically, this type of advocacy 6 helps remind overburdened prosecutors and judges of the cases on their docket and the appropriate timeline for completion of each of the steps in the process. Given the extensive back-and-forth between the Parquet and the judges that is present in criminal (penal) cases, these informational cues help the process to run smoothly. While in principle the coordination problems inherent in these institutions could be addressed through multiple and complementary interventions–and indeed PROJUSTICE has invested in other coordination￾enhancing interventions–the provision of legal assistance is indispensable for several reasons. In addition to the informational role that it plays in enabling coordination between these institutions, it also serves as a means of material support of trials. Trials (with or without a jury) are necessary to determine a detainee’s innocence or guilt and under Haitian law, defendants must have a lawyer present for a trial to proceed. The provision of legal aid ensures the presence of a lawyer for the annual trial period, enabling more cases to advance through the trial, effectively limiting the population held in PTD. The provision of legal aid by PROJUSTICE remains distinct from other forms of material support or coordination-improving records systems (i.e. the implementation of electronic Case Management Information Systems) in that it provides an advocate for a population whose human rights are compromised. Beyond combating the day-to-day manifestations of the coordination problem between Haiti’s judicial institutions, the provision of legal aid serves normatively to promote human rights in Haiti’s penal system. Given that prolonged PTD constitutes a fundamental human rights abuse, we anticipate that detainees are apt to mistrust the Haiti’s judicial institutions and criminal procedure. Moreover, prolonged PTD deprives individuals of economic opportunity as they are detained inside a prison for extended periods of time. We posit that at the level of the individual detainee, the provision of legal aid increases trust in Haiti’s justice system and governmental institutions more broadly. Moreover, at the individual level, release from prolonged PTD can improve the economic situation of an individual and her family. For a schematic of the Theory of Change, please refer to Appendix II. 2.2 Hypotheses The preceding discussion of the essential role of legal assistance in Haiti’s criminal justice system yields five main hypotheses that we will assess in the impact evaluation. Within the criminal justice system, we hypothesize that: (i) The provision of legal assistance accelerates the procedural advancement of cases in Haiti’s criminal justice system. 7 (ii) The provision of legal assistance accelerates detainees’ release from prolonged PTD, whether this results in release from prison or conviction. (iii) At an aggregate level, the provision of legal assistance to detainees in PTD lowers the prevalence (rate) of prolonged PTD in Haitian prisons. We also intend to assess the effects of gaining access to legal assistance on individual attitudes and economic situations. (iv) Receiving legal assistance improves detainees’ trust in (attitudes toward) Haiti’s judicial institutions and the government more generally. (v) As legal assistance promotes earlier release from prolonged PTD, it has a downstream impact on the economic opportunities available to detainees. Namely, it increases the economic opportunities available to detainees. 3 Impact Evaluation Design The PROJUSTICE program is currently in its second (and final) one-year extension, operating at a lower capacity due to lower levels of funding. This impact evaluation will take advantage of the operational capacity and experience that the current PROJUSTICE program has cultivated in its legal assistance program. In the extension of the program, the geographic focus of the legal assistance program has changed slightly. PROJUSTICE will maintain a staff of eight (8) individuals that are focused on reducing PTD in Port-au-Prince (PaP) and Croix-des-Bouquets (CDB). In these jurisdictions, legal assistants have, in the past years, been devoted to preventing PTD through intervention at police stations and the Peace Courts. In the final year of the current programming, the legal assistants will turn their focus to individuals currently held in prolonged PTD in PaP and CDB. As such the impact evaluation will focus on these efforts. PROJUSTICE will also maintain a six-member mobile legal team that will provide support for the jurisdictions of Saint Marc (SM), Cap Haïtien (CH), and Fort Liberté (FL). This support will largely focus on providing legal assistance during trials, in order to augment the number of cases that can be heard. This form of assistance is critically important but not suitable for an impact evaluation. If these mobile teams are able to assist in preventative efforts in prisons and Peace Courts, however, the timing of visits to these entities could plausibly be randomly assigned given the limited staffing/resources in order to gauge the 8 impact of these efforts. Depending on the focus of these efforts, this could be integrated into a subsequent version of the impact evaluation design. 3.1 Subjects and Context 3.1.1 Subjects The subjects of this impact evaluation are the recipients of PROJUSTICE legal assistance to combat prolonged PTD, or individuals who are currently detained in prolonged PTD. Given the geographical focus on the districts of PaP and CDB, the subjects will be detainees in the following prisons: (1) National Penitentiary (); (2) Women’s prison in Petionville (Prison des femmes à Pétion-ville), and (3) and the prison in Croix-des-Bouquets (la Prison Civile de la Croix des Bouquets). We conservatively estimate that there are 4,000 individuals in PTD in these prisons.6 Two additional criteria will render detainees ineligible for PROJUSTICE legal assistance. First, as per USAID guidelines, legal assistance is not offered to detainees accused of rape, human trafficking, or drug trafficking so these potential subjects will be excluded from consideration in the impact evaluation. Second, inmates who are not presently in prolonged pretrial detention will be excluded from the experimental sample. Given resource constraints of the PROJUSTICE legal assistants and the difficulties associated with collecting data for so many individuals given the state of records in Haiti’s prisons and judicial system, we will randomly select 1,000 individuals that meet the eligibility criteria for inclusion in the experimental sample. This should ensure that the estimated treatment effects generalize to the population of interest defined by the above criteria. 3.1.2 Projected Timeline Conditional on funding decisions and Institutional Review Board (IRB) approval, the timeline suggested by PROJUSTICE and Mission proposes an October or November 2015 start date for the beginning of the impact evaluation. Legal assistance will be provided for approximately six months, or until the PROJUSTICE program begins to wind down. Endline data will be collected subsequent to the end of this period. We anticipate completing data collection and the endline survey in summer 2016. 6 In National Penitentiary there were 4,374 prisoners and in the Petionville Women’s Prison there were x as of June 25, 2014. 9 3.1.3 Data Collection Data will be collected from multiple sources since we plan to combine administrative data with survey responses. Prior to the beginning of the impact evaluation, we will collect baseline covariates from the prison registers. This information will help us to determine eligibility to receive legal assistance and will provide the covariates for blocking in the randomization process (see the next section for additional details). It is necessary to collect much of the information in the registers–some which identifies the subjects and some of which does not. The identifying information consists of a detainee’s name, place of birth, address, and the name and contact information of the detainee’s designated contact person. This information will be recorded separately from other covariates on the form which do not contain personally identifying information. The two records will be linked only by the the detainee’s prison identification number (numéro d’écrou). This number is informative as to the prison facility where the detainee is held and the month of his/her entry, as shown in the example below: |{z} SM Prison ID /H/ 15 |{z} Year / 01 |{z} Month | {z } Month of Imprisonment / 001 |{z} Order in month This prison identification number will be recorded alongside a number of covariates: gender, birth date, date of arrest, date of booking (in prison), the alleged infraction, a measure of education via evaluations of the signature (see the discussion in the next section), marital status, and profession. The primary outcome variables of interest will come from the case files at the CFI. These measures of the process of each case within the CFI come from the dates at which various orders/documents are filed. The dates that will be recorded are reported in Tables 1-3. These dates presume that the prisoner’s file has been sent to the Parquet and that the Commissaire has already assigned the case to a prosecutor. Because the procedural trajectory of a case is largely determined by a prosecutor’s original determination, Tables 1-3 consider the three possible scenarios. These dates are not completely comprehensive and exclusive, particularly in Table 3. There are some orders or correspondences that have little apparent relevance to our outcomes of interest (procedural advancement of cases to reduce PTD). While these may be requisite for subsequent steps in the process, the specific dates of these steps will not be analyzed in any detail. 10 Table 1: Case Progress if Prosecutor Dismisses Case Action Document/Source Description 1 Decision by Prosecutor Classement sans suite Prosecutor dismisses a case 2 Release from Prison [Recorded in Prison Records] Date of release from Prison Table 2: Case Progress if Prosecutor Sends Case to Correctional Court Action Document/Source Description 1 Decision by Prosecutor Citation Directe Citation for trial in Correctional Court 2 Dismissal of Case Exequatur Documentation of dismissal Trial in Correctional Court [Trial Date from Minutes] Date of trial 3 Decision of Correctional Court Disposatif du jugement Judgement outcome, sentence 4 Release from Prison [Recorded in Prison Records] Date of release from Prison Table 3: Case Progress if Prosecutor Sends Case to IJ for Investigation Action Document Description 1 First decision by Prosecutor Requisitore d’Informer Application for investigation by IJ 2 Assignment of Case to IJ Ordonnance de désignation du doyen Date of assignment to IJ 3 Conclusion of IJ’s investigation Ordonnance de soit-communiqué Communication from IJ to Commissaire 4 Commissaire approves investigation Requisitoire définitif Commissaire’s recommendation 5 IJ’s closing order Ordonnance de clôture Commital for trial or dismissal 6 Dismissal of Case Exequatur Documentation of dismissal Trial in Correctional Court [Trial Date from Minutes] Date of trial 7 Judgment Disposatif du jugement Judgement outcome, sentence 8 Release from Prison [Recorded in Prison Records] Date of release from Prison One other important consideration that is consequential for our analysis of the final case dispositions is the application of the Lespinasse Law loi Lespinasse in the case of convictions. This law allows judges to credit convicted individuals for the time that they have already served in PTD. Given that our sample includes only individuals in prolonged PTD, this law should be invoked in the case of many if not all convictions. In practice, however, it is not uniformly applied. Thus, it is important to understand whether or not this law has been invoked when assessing release records. The judgment as recorded on the Fiche de jugement de personne en détention (Form of Judgment for the Detainee) includes information about whether or not loi Lespinasse has been invoked or not. 11 We also plan to run a two wave (baseline and endline) survey in order to assess the effects of PTD on attitudinal and economic outcomes. Notably, surveys have been the principal outcome measures in impact evaluations of existing rule of law interventions in developing countries (Blattman et al., 2014; Sandefur and Siddiqi, 2013). The surveys will each include a number of sections as displayed in Table 4. The demographic questions are relatively standard on surveys of public opinion. Indeed, the questions are adapted from fields on the Latin American Public Opinion Poll (LAPOP) from 2014. LAPOP is one source of existing public opinion data from Haiti on a wide variety of issues. The second section will be implemented in both surveys and is borrowed directly from LAPOP is a battery of questions about trust in and attitudes toward various institutions of the government of Haiti. Respondents express their preferences on a seven-point scale. The third section will similarly be implemented in both baseline and endline surveys and seeks to assess respondents’ understanding of the Haitian criminal justice system. This component will be comprised of several (≈ 4-5) open-ended questions about criminal procedure and rights of the accused in Haiti to be developed in collaboration with PROJUSTICE staff. Fourth, the endline survey aims to assess the economic situation of subjects and (where relevant) their families. [Finally, in line with a growing literature in political science and economics, we will invite subjects to play one or two behavioral games intended to assess subjects’ behavioral responses to a number of situations.] Table 4: Components of Baseline and Endline Surveys Baseline Survey Endline Survey Demographic Information X X Attitudes Toward GoH Institutions X X Understanding of Criminal Justice System X X Economic Situation Assessment X Behavioral Games X The first wave of the survey will include all subjects who will consent to voluntary participation in a survey. These individuals will all be in prison during this wave. We will subsequently run a second survey wave following the conclusion of the intervention. This wave is somewhat more complicated: we anticipate that some individuals will still be detained (whether in prolonged PTD or subsequent to conviction). However, other individuals will have been released and considerably harder to track. Our interviews in Haiti suggest that the tracking of these individuals will be difficult. However, we plan to attempt to survey these individuals. Recent literature has included endline surveys of similarly transient populations (i.e. high risk youth in Liberia in Blattman et al. (2015)) or measurements further downstream (i.e surveys of 12 Ugandan ex-child soldiers after 10 years in Blattman (2009)). Using similar survey techniques (particularly in terms of offering incentives to enumerators), we aim for a sizeable response rate among those released from detention. Moreover, we will use resampling techniques such as those discussed in Gerber and Green (2012) on a subsample of missing respondents if attrition is significant in the second round of the survey. The final source of data will allow us to accurately provide a cost-benefit analysis of legal aid provision for further advocacy efforts used by the mission. This requires detailed cost per prisoner information, presumably from sources at the Ministry of Justice and Public Security. PROJUSTICE staff indicate that they can help us find this information to conduct this analysis. 3.2 Randomization Method Our design takes advantage of the resource and time constraints of PROJUSTICE’s legal aid staff as well as the wind-down of the program. In preparation for the impact evaluation, the designated impact evaluation M&E specialist will create a list of the detainees selected for participation recording five covariates that will be used for blocking in our random assignment procedures. These six covariates can be recorded directly from the prison registers as they are included in the Ministry of Justice-issued registers. These covariates include: 1. Prison: Prison where the detainee is held. 2. Gender: Recorded as Male or Female.7 3. Age: From birth date field. 4. Time in prison to date, in months: From Prisoner identification number or date of booking (both contain this information). 5. Infraction type: Coded categorically. 6. Education indicator (categorical): From the qualities of a prisoner’s signature. Haiti’s literacy rate (2008-2012) was 48.7% (UNICEF, 2013). The signatures in the prison registers vary dramatically in quality and, we argue, serve as a rough proxy for educational attainment/literacy. We will rely on multiple coders’ qualitative assessments of signature quality to code this variable.8 Ultimately, we are randomizing the order in which the PROJUSTICE legal assistants will provide legal assistance to subjects. Since legal assistants will proceed through as many of the 1,000 cases as possible prior to the conclusion of the PROJUSTICE program, we use blocking to ensure that the randomization yields maximal heterogeneity among the full schedule of potential subjects. Using these five blocking variables, we will generate 200 “quintets” of five subjects each. These blocks will minimize the multivariate distance 7Gender is not perfectly collinear with prison since the prison in Croix-des-Bouquets includes both male and female inmates. 8We have not validated this measure as a proxy for education. We intend to use it for blocking only, where the inclusion of a potentially non-prognostic covariate will not bias results. 13 between each “quintet.”9 The result of the blocking can be thought of as a 200×5 matrix, with the quintets in the rows. In order to eliminate any systematic patterns within the quartets, we will “shuffle” or randomize the order of the cells in each row. Subsequently, we will randomize the order of the columns. The resultant matrix contains the full schedule of treatment. The order of the cases will be prescribed by starting in the top left corner and proceeding through the first column before moving to the second, then the third etc. In order to assess the efficacy of the randomization procedure, ex-ante, we will compare each column (n = 200) for balance across all available pre-treatment covariates. In addition to those used for blocking, we have marital status, profession, and physical appearance covariates. Ex-post, we will report balance tests between those assigned to treatment and control using the same covariates, given the number of individuals the legal aid team was able to assist. 3.3 Treatment The treatment is based entirely upon the services already provided by PROJUSTICE legal assistance teams for detainees in prolonged pretrial detention. These lawyers first identify prisoners by their prison identification number (numéro d’écrou). In this task, the list and ordering of the subjects will make the legal assistants’ job slightly easier as they will be provided with this number from the start. They then must go to the Parquet to match the prison identification number to the identification number used in the CFI. This number allows the lawyers to track the files to the assigned prosecutor and gain access to the file. The lawyer will subsequently assess the state and progress of the case, meeting with the prisoner as well as with the relevant entities in the CFI to procedurally advance the case. Though bribery can sometimes be a strategy used in this process to expedite service, the PROJUSTICE staff are forbidden from paying these bribes as per USAID requirements. Because case histories, the assigned prosecutor/judge(s), and designated legal assistants assigned to each case vary in practice, the legal assistance “treatment” offered will vary somewhat across subjects. It is important to preserve this heterogeneity in order to maintain the authenticity of the treatment for the purposes of an impact evaluation (Gerber and Green, 2012). For this reason, we define treatment broadly as the provision of legal assistance. 3.3.1 CONSORT Participant Flow Diagram For a rough outline of the CONSORT Participant Flow Diagram, please see Appendix II. 9We will implement this blocking through the blockTools R package developed by Moore and Schnakenberg (2015). 14 3.4 Results 3.4.1 Outcome Measures and Covariates The covariates that we will collect and use in analysis are limited somewhat by the relatively sparse nature of the available records. Since the use of post-treatment covariates is apt to induce bias in estimates of the treatment effect, we will rely only on the covariates that we can collect prior to the randomization. These covariates are those available from the prison registers. A list of these covariates is listed above in the “Data Collection” section. These covariates will be used in blocking, randomization checks, and in our statistical analysis. As previously discussed, our outcome measures come from two distinct sources administrative records from the CFI and the survey responses. The prison and court records contain duration data about the process of each case in the judicial system. The challenge here in constructing appropriate outcomes is to operationalize these duration data appropriately and to take full advantage of the richness of this data. There are a number of challenges in estimation given the nature of the data which we will discuss in the next section. Case Outcomes and Procedural Advancement Our primary outcome of interest is whether cases have reached their final disposition–signaling the end of prolonged PTD. At the conclusion of the process, there are three possible states for each case: release from prison, conviction, or continuing prolonged PTD.10 Two of these outcomes indicate that a case has reached its final disposition, release or conviction. Pooling these two outcomes, we can create a binary variable to encode whether or not subject’s case has reached a final disposition or not. A more nuanced use of duration data is necessary to determine the effects of legal aid and the potential costs savings from the provision of these services. In this case, the duration of time that a detainee spends in prolonged PTD prior to a final disposition. The most basic specification would be to measure the number of days between the start of the experiment and the final disposition (where present). This, however, introduces the issue of censoring as the final disposition will not necessarily be observed for all cases. We will address this issue more comprehensively in the Statistical Analysis section. We will also use the duration until release (when release is the outcome of interest) in our analysis to determine the cost savings from the provision of legal aid. Finally, we will use the duration of various steps in the process to improve our understanding of the effects of legal aid within the criminal justice system, we will use the duration data for various stage. The 10This assumes no attrition, escape, or death. 15 duration data introduces the issue of selection, which will be developed more thoroughly in the Statistical Analysis section. Survey Measures The measurement of changes in attitudes, knowledge, and economic situation are somewhat more straightforward, at least statistically, than the case and duration data introduced above. In terms of general attitudes toward the government, we borrow some standard survey questions from existing surveys including the Latin American Public Opinion Project (LAPOP). These questions probe trust in/attitudes toward a variety of Haiti’s institutions. We also intend to assess subjects’ understanding of criminal procedure and their rights via a set of open-ended questions. Both of these measures will be measured in both the baseline and endline surveys. We also intend to measure economic circumstances of subjects and their families in an endline survey. It is somewhat challenging to gauge economic circumstances via survey measures in any environment, though environments with high degrees of informality present particular challenges. We are in the process of developing a questions to measure economic circumstances in order to create a latent measure of economic well-being. 3.4.2 Statistical Analysis Final Disposition of Cases The most straightforward and test of our hypothesis that legal assistance reduces prolonged PTD in Haitian prisons is to compare the proportion of cases that reach a final disposition in treatment (those that receive legal assistance) to those in control (those that do not receive legal assistance). Due to the randomization of the ordering of cases, differences in the proportion of cases that reach a final disposition can be attributed to the provision of legal assistance by PROJUSTICE. This difference-in-means can be estimated via ordinary least squares (OLS) as follows: yi = β0 + β1zi + ψXi In this specification, the observed potential outcome yi is whether or not a subject’s case reaches a final disposition (binary, as described above), zi is the treatment status of an individual (binary), and Xi 16 is a matrix of covariates. The regression will be reported with and without covariate adjustment. Because all subjects have an equal probability of selection into treatment despite the blocking, this regression can be estimated without employing weights. The dichotomous treatment variable obscures, to some extent, that individuals in the treatment group potentially receive different “doses” of treatment based on when their cases are addressed by legal aid attorneys. Nevertheless, this basic specification provides an easily interpretable result. Speed of Case Disposition Our hypotheses posit that the provision of legal assistance by PROJUSTICE accelerates the speed at which cases proceed to their final disposition. Thus, we will analyze the impact of the provision of legal assistance on the speed at which cases progress through the criminal justice system. The structure of our duration data, which we will use in this analysis, poses the challenge of censoring. We do not anticipate that all subjects will have been released or convicted by the end of the study in April 2016. The censoring in the data emerges from the fact that the end of prolonged PTD cannot be observed for all subjects. We propose two methods for analysis of this data: OLS and tobit-like modeling. Our OLS specifications will yield the most easily interpretable analysis of the treatment effect of legal assistance. However, given our beliefs/hypothesis that legal assistance is apt to speed the rate at which subjects’ cases reach their final disposition, we would expect more observations in the control group to be censored than in treatment. We anticipate that this will attenuate the estimates of the treatment effect. This will serve to provide a straightforward lower bound on the ATE. Our OLS specification follows the same format of the model presented above except that the outcome yi is the day at which the subject was released or convicted, or, in the absence of release or conviction, the day of the final data collection. The treatment zi is now a continuous variable representing the day at which treatment was initiated (if initiated). We will code this as the difference between the last day of data collection and the day at which treatment was initiated. The result is a discrete (but continuous for all intents and purposes) variable with a large mass of observations at “0” (the control group). We hypothesize a negative coefficient on zi . For the alternative approach, we intend to use a tobit model or derive a likelihood with similar properties but different distributional assumptions. In this model, observations that are censored contribute to the likelihood (function) in a different way from those that are not censored. In this case, the outcome 17 variable can be defined as follows, given that b is the final day of data collection (i.e. the censoring point). yi =    y ∗ i if y ∗ i ≤ b b if y ∗ i > b In the tobit model, outcomes that are censored contribute to the likelihood differently from outcomes that are uncensored. Thus, the cases presented above contribute to the likelihood function distinctly. The general tobit function assumes that  ∼ N (0, σ2 ). We may derive a tobit-like likelihood function following another distribution based on the properties of the survival function, given the well-documented weaknesses of the tobit model in the presence of heteroskedasticity. Procedural Advancement An intermediate outcome of interest is a case’s procedural advancement. Namely, we are interested in whether the provision of legal assistance accelerates specific parts of the process. For example, one source of delay is the IJ’s investigation. As previously mentioned, constitutionally, the IJ has three months to complete her investigation unless granted an extension by the Dean. Thus, one outcome of interest (procedurally) is the length of time for this investigation. However, this quantity is difficult to estimate in a principled manner given earlier procedural steps that are endogenous to treatment status. For example, if the provision of legal assistance changes the proportion (or type) of subjects whose cases are sent to the IJ in the first place, the potential outcomes of the IJ’s investigation are no longer equivalent in the treatment and control groups. The approach that we will utilize to address this problem is an application of the trimming bounds on ATEs proposed by Lee (2010) and advocated by Gerber and Green (2012). This assumes a form of monotonicity in terms of attrition/selection out of the sample. Specifically, in the case of the IJ’s investigation, we assume that those who receive legal aid are systematically more likely to be released by the prosecutor or sent to correctional court than those who do not receive legal aid. Conditional on a difference in the rate of those released/sent to correctional court, we can estimate said trimming bounds on the treatment effect. The resultant estimated treatment effects will be interval estimates rather than point estimates. For the early stages in the process (i.e. the length of IJ investigation), we anticipate that these bounds will be relatively informative as relatively few cases sort out of the sample prior to the IJ’s investigation, given the aforementioned prosecutors’ incentives. However, as the proportion of individuals that select out of the penal process increases (at various points) such interval estimates are bound to become 18 less informative. This is a limitation inherent to working with data of this class. Survey Outcomes The data from the survey will be more straightforward to analyze. We will use both a binary and the discrete measures of treatment discussed above to assess the effect of treatment on measures of (i) attitudes toward Haiti’s criminal justice system and governmental institutions more broadly; (ii) knowledge of rights of the accused and criminal procedure; and (iii) a latent measure of economic circumstances. These outcomes can be estimated via OLS without the challenges discussed with regard to the duration measures. We intend to estimate the effects of legal assistance with and without covariate adjustment in the interest of transparency. Given the nature of the surveys and the multiple outcome variables that come from the survey, we are acutely aware of the possibility of “fishing” and/or multiple comparisons problem that may arise in the analysis of this data. Once the surveys are in their final form, we will specify the operationalization of our outcome measures in a comprehensive pre-registration plan to be filed with EGAP prior to the beginning of the impact evaluation (Humphreys et al., 2013). Attrition in the form of missing survey responses, particularly in the follow-up round, poses a challenge to this analysis. If there is attrition of this form, it is unlikely that the consequent missingness could be said to be “independent of potential outcomes,” and it is seems doubtful that a lesser assumption of missingness independent of potential outcomes with covariate conditioning would be wise. Thus, this represents a potential source of bias. One way to overcome this challenge is to make an additional effort to follow up with a random sample of missing respondents. (Gerber and Green, 2012, 239) show via Monte Carlo simulations that this is a cost effective procedure to ascertain unbiased estimates of treatment effect in scenarios with attrition. We will reserve a portion of the survey budget to provide the resources for time-intensive resampling. Non-Compliance One issue that we have not yet addressed is that of non-compliance. In our design, non-compliance is defined as someone assigned to treatment (i.e. within the portion of the order for which legal assistants have the resources to provide their services) that does not receive this assistance or a person assigned to control that receives legal assistance from PROJUSTICE.11 The former is possible under three circumstances: (i.) a case 11Note that the presence of other organizations in the legal assistance sector is not problematic or a source of non-compliance in this study. The order in which participants will be offered services should not be systematically related (i.e., the relationship is orthogonal) to any selection criteria from other organizations, so the estimate of causal effects of PROJUSTICE legal aid will not be compromised. 19 is advanced/reaches its disposition before the PROJUSTICE staff is able to offer the intervention; (ii.) a subject rejects legal assistance; or (iii.) the PROJUSTICE staff accidentally pass over a subject that should receive treatment for any reason. The latter is possible only in the case of error by the PROJUSTICE staff. On the basis of our conversations with PROJUSTICE staff and legal assistants, we believe that the threat of non-compliance is relatively low. If there is some (low) level of non-compliance, we will pursue the same empirical analysis estimating ITT effects as opposed to ATEs. Additionally, we can estimate LATEs/CACEs for the OLS-based estimation strategies. These should not vary significantly from the ITT effect estimates if non-compliance is low. For the non-OLS based estimates, there is not an established estimation procedure for the LATE/CACE. Attrition We have discussed attrition above in terms of the survey responses, but not in terms of case trajectory. One potential source of attrition is the transfer of subjects to other prison facilities. We anticipate that any potential transfers will affect a very small proportion of subjects. To the extent that prison registers seem to effectively document those entering and exiting prisons, we anticipate that we will be able to gather outcomes even for those transferred to different facilities in the endline record collection to minimize the threat of attrition to the case outcome design. Extension: Integration of Experimental Data and Observational Data PROJUSTICE has collected data on their provision of legal assistance to reduce prolonged PTD in Haiti over the past few years. The mission expressed an interest in integrating this data into our impact evaluation if at all possible. While this existing observational data does does not have a counterfactual and is thus insufficient for an impact evaluation on its own, we can integrate it with the experimental results. The data generating process for the experimental and observational case history data is very similar. Building the Bayesian model requires a number of additional assumptions about these processes. However, it has the potential to yield richer results, particularly in the cost-benefit analysis. An integrated Bayesian model will allow us to estimate a posterior predictive distribution for the case trajectory of individual subjects (beneficiaries), which in turn will assist in the assessment of the benefits provided by PROJUSTICE legal aid which can be used in future advocacy. Our proposed integration of experimental data with existing observational data comprises an approach that is relatively new in the academic literature and, to our knowledge, has not yet been utilized in the impact 20 evaluation literature. However, related approaches are becoming more prominent in the academic literature. For example, Humphreys and Jacobs (2015) propose methods for the Bayesian integration of quantitative and qualitative research. In a working paper, Cooper (2015) discusses the integration of expert assessments with results from a randomized controlled trial (RCT). Developing the model to integrate both types of data will require further refinement and discussion, but we believe that this will contribute to the richness of the implications in the impact evaluation. 4 Conclusion This impact evaluation proposes to evaluate the legal assistance program offered by PROJUSTICE in two jurisdictions in Haiti from late 2015 through mid 2016. These results will provide estimates of the effect of this legal assistance on recipients’ case trajectories, attitudes, and economic well-being. Moreover, it will allow us to estimate the effects of the provision of this assistance on rates of pre-trial detention (PTD) in Haiti more generally. Ultimately, we will be able to use the estimated treatment effects to provide a cost-benefit analysis for use in future rule of law advocacy by USAID in Haiti, contributing to other objectives of the program. Our design takes advantage of the legal assistance services offered in the Port au Prince and Croix des Bouquets jurisdictions during this extension of the PROJUSTICE program. Ultimately we seek to identify 1000 eligible recipients of this legal assistance. We will randomize the order in which these recipients will receive this service, in what is arguably the fairest means of distributing the limited legal assistance resources that PROJUSTICE provides. The intervention allows the lawyers working for PROJUSTICE to provide the same services that they have been providing over the past several years. We will measure the outcomes of the legal assistance in terms of administrative data and survey outcomes. The administrative data will be taken from prison and CFI records about the progress and outcomes of subjects’ cases. The survey data will come from baseline and endline surveys that we intend to implement. This impact evaluation design provides an overview of the statistical analysis we will pursue to obtain a measure of the causal effect of USAID legal assistance on prolonged PTD in these jurisdictions. The next steps for this impact evaluation include review of this design by the USAID mission in Port au Prince, NORC, USAID-DRG, and an academic peer review. The other steps include obtaining IRB approval from Penn State University and Columbia University and funding for the impact evaluation. 21 References Blattman, C. (2009). From Violence to Voting: War and Political Participation in Uganda. American Political Science Review, 103(2):231–247. Blattman, C., Hartman, A., and Blair, R. (2014). How to promote order and property rights under weak rule of law? an experiment in changing dispute resolution behavior through community education. American Political Science Review, 108(1):100–120. Blattman, C., Jamison, J. C., and Sheridan, M. (2015). Reducing Crime and Violence: Experimental Evidence on Adult Noncognitive Investments in Liberia. Working paper. Cooper, J. J. (2015). How Robust is Institutionalized Extortion? A Field Experiment with Truck Drivers in West Africa. Working paper, Columbia University. de Zayas, A. (2005). Human rights and indefinite detention. International Review of the Red Cross, 87(857):15–38. Gerber, A. S. and Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. WW Norton. Humphreys, M. and Jacobs, A. (2015). Mixing Methods. American Political Science Review, Forthcoming. Humphreys, M., Sanchez de la Sierra, R., and van der Windt, P. (2013). Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration. Political Analysis, 21(1):1–20. Lee, D. (2010). Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects. Review of Economic Studies, 76(3):1071–1102. Mayerfeld, J. (2016). The Architecture of Human Rights: Why Constitutional Government Requires International Human Rights Law. University of Pennsylvania Press. Moore, R. T. and Schnakenberg, K. (2015). Package ‘blockTools’. Technical report, The Comprehensive R Archive Network. Sandefur, J. and Siddiqi, B. (2013). Delivering justice to the poor: Theory and experimental evidence from liberia. In World Bank Workshop on African Political Economy, Washington, DC, May, volume 20. UNICEF (2013). At a glance: Haiti Statistics. Electronic., UNICEF. United States Department of State (2014). Haiti 2014 Human Rights Report. Country Reports on Human Rights Practices for 2014, Bureau of Democracy, Human Rights and Labor, United States Department of State, Washington DC. Walmsley, R. (2014). World Pre-trial/Remand Imprisonment List (second edition). Policy report, International Center for Prison Studies, University of Essex, London. 22 Appendix I: Theory of Change Figure 1: Theory of Change. Inputs Staff: - PROJUSTICE PaP/CdB Legal Assistance team (8 individuals) - PROJUSTICE Administration Activities Intervention (treatment): - Matching of prison and CFI records for individuals in prolonged PTD - Provision of Legal Aid to individuals in prolonged PTD - Record keeping about services provided, case trajectory Outcomes Outcomes of intervention: - Procedural advancement of cases - Shorter time to final case disposition (release or conviction) - Shorter time in prolonged PTD Longer Term Outcomes Longer-Term Outcomes: - Lower rates of prolonged PTD. - Lower costs for the Government of Haiti (esp. comparing provision of legal aid to cost of imprisonment). - Improved economic standing of detainees in prolonged PTD and their families. - Improvement in attitudes (esp. trust) towards justice system and Government of Haiti by detainees. 23 Appendix II: CONSORT Diagram Figure 2: Flowchart of potential and actual participants. PT Detainees, PaP and CdB (n ≈ 4000) Excluded (n=χ+ξ): a) Accused of rape, human trafficking, or drug trafficking (n=ξ) b) Not yet in prolonged PTD (n=χ) Randomly Selected for Inclusion (n=1000) Excluded (n≈ 4000 − χ − ξ − 1000): a) Not selected for experimental sample Randomised Ordering (n=1000) Intervention group (n= ) Control group (n= ) Pre-intervention survey: - Responded (n= ) - Did not respond (n= ) Pre-intervention survey: - Responded (n= ) - Did not respond (n= ) Intervention Administered: - Received treatment (n= ) - Did not receive treatment (n= ) - Explanation... (n= ) Intervention not Administered: - Did not receive treatment (n= ) - Did not receive treatment (n= ) - Explanation... (n= ) End-line Survey: - Responded (n= ) - Did not respond (n= ), – Not located (n= ), – Chose not to participate (n= ) End-line Survey: - Responded (n= ) - Did not respond (n= ), – Not located (n= ), – Chose not to participate (n= ) 24 DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 40 ANNEX B, SURVEY INSTRUMENT(S) (Under separate cover) Impact Evaluation: PROJUSTICE Legal Assistance Questionnaire for Baseline Survey Impact Evaluation: PROJUSTICE Legal Assistance Baseline Survey Questionnaire v1 Preload Field Control ENUMERATOR: Name________________________ ENUMID: ID ________________ SURVEYID: Unique Identifier _____________________ PRISONNAME: Name _________________________ PRISONCODE: Code ________________________ GENDER: M/F TIMESTAMP: Papyrus Consulting, a Haitian data collection firm, in collaboration with PROJUSTICE, is conducting a survey as part of USAIDs Human Rights and Learning Strategy designed to assess the outcomes of participants in the PROJUSTICE program. You have been selected through your participation in the PROJUSTICE program. The purpose of this study is to better understand the condition of current and former pre-trial detainees in Haiti. During this interview, we will ask you a series of questions about yourself, your beliefs, and your time in [insert prison name]. All information you provide will be held as confidential and not shared with prison authorities. All records of this interview will be identified only with a number. In addition, your decision on whether to participate or not, as well as any statements you make, will not affect your status as participant in the PROJUSTICE program. It is also important to note that the outcome of your trial or the conditions of your detention will also not be impacted (positively or negatively) by your participation in the study. We anticipate that this survey will take no longer than 30 minutes to complete and provide information about the conditions of Haitian prisons and pre-trial detention to Haitian leaders, scholars, and the international community. You can refuse to answer any question that you do not want to answer and you can choose to stop participating in this survey at any time. If you have any questions about the survey, please call XXXXXX so that we may assist you. CONSENT. Are you willing to proceed? 1 Yes 2 No PROG: SKIP TO CLOSURE Demographic Information: To begin, I would like to ask you some questions about yourself. AGE How old were you on your last birthday? _______ MARITAL What is your current marital status? Impact Evaluation: PROJUSTICE Legal Assistance Questionnaire for Baseline Survey Single Married Separated Divorced Widowed Prior to being in pre-trial detention can you please let me know your place of residence? Department ___________________ Arrondissements __________________ Commune _______________________ Sector __________________________ ADULT Prior to being in pre-trial detention, how many adults age 18 or greater lived in your household? CHILD And how many children lived in your household before your arrest? CHILD2 How many biological children do you have? EDUCATION What is the highest level of education you have completed? Please only include levels you have completed fully: No formal education Some primary school Primary school completed Some secondary school Secondary school completed Post-secondary school Thank you for your responses. We know have some questions dealing with your attitudes towards different institutions in Haiti. Please remember your answers are confidential: TRUST1 On a scale of 1-7 where 1 is strongly agree and 7 is strongly disagree: To what extent do you think the courts in Haiti guarantee a fair trial? To what extent do you respect the political institutions of Haiti? To what extent do you think that citizens’ basic rights are well protected by the political system of Haiti? To what extent do you feel proud of living under the political system of Haiti? To what extent do you think that one should support the political system of Haiti? To what extent do you trust the justice system? To what extent do you trust elections in this country? 1= Strongly Agree 2= Agree 3=Somewhat agree 4= Neither agree nor disagree 5= Somewhat disagree 6= Disagree 7= Strongly disagree TRUSTMOST I’m now going to read you a list of groups. From this list, please tell me which group you trust the most? (Read options) - Courts - Political institutions - Political system of Haiti - Justice system - Parliament - Police (PNH) Impact Evaluation: PROJUSTICE Legal Assistance Questionnaire for Baseline Survey - President - Prime Minister - Local/municipal government - KASEK - Elections - TRUSTLST Using the same list, which group do you trust the least?(Read options) - Courts - Political institutions - Political system of Haiti - Justice system - Parliament - Police (PNH) - President - Prime Minister - Local/municipal government - KASEK - Elections TRUST2 On a scale of 1-7 where 1 is strongly agree and 7 is strongly disagree: To what extent do you trust the Parliament? To what extent do you trust the Police (PNH)? To what extent do you trust the President? To what extent do you trust the Prime Minister? To what extent do you trust the local or municipal government? To what extent do you trust the KASEK? PODIFF In your opinion, who is the most to blame for the political difficulties in Haiti? (Read options, select) - The president - The National Assembly - Politicians in general - The Haitian people in general - The international community - Didn’t read options/other - Experience in Criminal Justice System (open ended, can we record these responses?) a. In relation to your current detention, was there a moment that someone could have helped you and did not? a. Who? i. Family member ii. Friend iii. Another civilian (non-official) iv. Police v. Lawyer/legal assistance vi. Judge vii. Prosecutor viii. Other b. When? i. Prior to arrest ii. Between arrest and imprisonment (in penitentiary) iii. During imprisonment Impact Evaluation: PROJUSTICE Legal Assistance Questionnaire for Baseline Survey b. What is the most unfair part of this process? We would like to work with PROJUSTICE and the mission to develop 4-5 open-ended questions regarding criminal procedure or rights of the accused that can be used to gauge respondents’ understanding of the criminal justice process/their rights. *Note that these are prototypes and will be confirmed in further communication with PROJUSTICE.* 1. Who has the power to convict or acquit individuals in pretrial detention? a. Correct Answer b. Incorrect Answer c. Unknown d. Refused to Answer 2. What resources/help are individuals in your situation entitled to as per the Haitian Constitution? a. Correct Answer b. Incorrect Answer c. Unknown d. Refused to Answer 3. What is Projustice? What do they do? a. Correct Answer b. Incorrect Answer c. Unknown d. Refused to Answer Enumerators only: Please evaluate the respondent on the following statements: 1=Disagree strongly 2= Disagree 3=Somewhat disagree 4= Neither agree nor disagree 5= Somewhat agree 6=Agree 7= Strongly agree The respondent understood that prolonged pretrial detention is unconstitutional. 1 2 3 4 5 6 7 The respondent was knowledgeable about his/her rights as an accused person in prolonged pretrial detention. 1 2 3 4 5 6 7 The respondent was resigned to his/her current situation as a pretrial detainee. 1 2 3 4 5 6 7 The respondent was unhappy with his/her current situation as a pretrial detainee. 1 2 3 4 5 6 7 DRG-LER – GS-10F-0033M/AID-OAA-M-13-00013 IMPACT EVALUATION OF USAID HAITI PROJUSTICE PROGRAM PRETRIAL DETENTION COMPONENT | 41 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523