UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT DOMINICAN REPUBLIC PROGRAM FOR POLITICAL EDUCATION AND MANAGEMENT, 2 ND PHASE (FORMATION OF YOUNG LEADERS, FYL) DEVELOPMENT EFFECTIVENESS EVALUATION STUDY HUGO J. PIRELA, PH. D EMMANUEL SILVESTRE, PH.D. SANTO DOMINGO, D.R. NOVEMBER 2012 ACKNOWLEDGMENT The researchers wish to recognize the management of the Program’s executing Consortium of Unibe, Participación Ciudadana and Intec, who collaborated with the present study by providing extensive documentation on program operational data and reports. I TABLE OF CONTENTS PAGE Executive summary ……………………………………………………………………………………………………………... a Preface on Program Evaluability ……………………………………………………………………………………….….. i EVALUATION OF THE PROGRAM FOR THE FORMATION OF YOUNG LEADERS, 2 ND PHASE I. Introduction ………………………………………………………………………………………………………………….. 1 A. Goal of the study………….…………………………………………………………………………………………… 1 B. Focus of research…………………………………………………………………………………………………………. 1 C. Information & data collection……………………………………………………………………………………. 2 1. Statistical sampling ………….…………………………………………………………………………..……… 3 2. Field work……………..………….……………………………………………………………………………..…….. 3 II. Main findings and analysis……………………….…..…………………………………………………………..……… 4 A. Program intermediate outputs…………………………………………..……………………………………….. 4 1. Courses given…………………………………………..………………………………………………..……….. 4 a. Regular leadership & political management courses………….…………………..…………. 5 b. P.A.T. courses to train trainers in the regular course…………….…………………………… 6 c. Specialized courses…………………………………………………………………………………..……….. 6 2. Inter-party dialogue events…………………………………………………………….…………..…………. 7 3. Studies on modernization requirements………………………………………….…………..………… 7 B. Program terminal outputs……………………………………………………..…………….…………..…………. 7 1. Graduates from the regular courses……….………………………..…………….…………..……….... 7 2. Graduates from the other courses……….………………………..…………….…………..………….. 9 a. The PAT Courses…………………………….……………..……………………….…………..………….. 9 b. The specialized courses…………………………..………………..…………….…………..…………. 9 C. Program intermediate outcomes…………………………..……..………………….…………..….…………. 10 1. Academic performance scores…………………………..……………………….…………..…………… 10 2. Learning indexes……………………………………….………………..…………….…….……..…………… 11 3. Dispersion of graduates knowledge gain by cohort, sex and region………….……………. 11 a. Knowledge gain based on exit scores. ……………………………………………………………. 12 b. Knowledge gain based on final tests scores………………………....…………..…………..... 14 D. Program terminal outcomes……………………..…………………………..…………...……..…………..... 16 1. Comparability between Treatment & Control groups……………………………...……………. 17 a. Basic independent variables as confounding factors…………………………..……………. 17 b. Parallel training as a confounding factor.……………………..………….……..………………. 19 2. Outcome trends during execution.……………………..……………….……….……..………………. 19 a. Education of organizations’ grassroots.……………..…………………….……..……………... 19 b. Submission of proposals for organizational reform.………..………..…………………….. 22 c. Increased management responsibility for young leaders.………..………...……………. 24 d. Increased participation of young leaders in election processes.………..……………… 26 e. Increased young leaders’ participation in inter-organizations dialogue .….....……. 27 f. Increased young leaders’ transparency & accountability .………..…………..………….. 29 3. Outcomes observed after execution……………………..……….………………...……..……………. 31 a. Role of external factors……………………..…………………….………………..…………..………. 31 b. Main final outcomes. A quantitative perspective……………………..….…….....………….. 33 c. Main final outcomes. A qualitative perspective……………………..………….....………..… 36 II (i) Education of organizations’ grassroots……………………..…….…….......…………... 36 (ii) Proposals for organizational reform………………..………..……….…........……......... 38 (iii) Increased management responsibility for young leaders……….......……………. 41 (iv) Participation of young leaders in election processes……………………....……….. 41 (v) Young leaders’ participation in inter-organization dialogue…………............... 42 (vi) Young leaders’ transparency & accountability ……………….…………….............. 43 E. Program impacts………………………………………....……………………………..………...……..…..……. 43 1. General trend impacts………………………………………....………….………………………..…..….. 44 2. Specific trend impacts………………………………………....………….………………………..…..…… 45 a. Gender equity and youth participation………………………………....………….…..…..…… 45 b. Extended training for party members…………………………………………..….…..…..……. 46 c. Improved transparency and accountability……………………………...……….…..…..…… 46 III. Conclusions …………………………………………………………………………………………………………………. 47 A. Operational delivery ……………………………………………………………………………………………….. 47 B. Short-term changes induced ……………………………………………………………………………………. 48 1. Acquisition of knowledge ……………………………………………………………………………………. 48 2. Observed behavioral trends…………………………………………………………………………………. 49 C. Medium-term changes induced ……………………………………………………………………………….. 50 D. Contributions to long-term transformations …………………………………………………………….. 51 IV. Recommendations ……………………………………………………………………………………………………….. 53 A. On the extent and reach of the Program…………………………………………………………………… 53 B. On the recruitment and selection process ………………………………………………………………….. 56 C. On the program effectiveness focus …………………………………………………………………………… 58 D. On program metrics and evaluability ………………………………………………………………………… 59 E. On the effort’s long term sustainability ………………………………………………………………………. 60 STATISTICAL APPENDIX a EXECUTIVE SUMMARY This document presents an assessment of the development effectiveness of the USAID-sponsored program for the formation of young leaders (FYL) on its 2nd phase in the Dominican Republic, to the extent that such effectiveness can be gaged by the observable modernization changes induced by the Program in the way the political system does business in the country. The study logically follows the evaluation of the Program’s 1st phase, completed in 2008. However, of necessity and by design, the 2 nd phase established a more stringent methodological framework for the present study, including a better focus on evaluation metrics and the use of control and treatment groups to better establish attribution of results. These methodological precisions, plus changes operated in the intervention itself, created discontinuities in the evaluation premises between the first and the second phases that render impossible for this study to be valid for the Program as a whole, from a strict epistemological point of view. However, this circumstance has not precluded the reaching of important conclusions and lessons for the intervention as a whole; the same that are disclosed in what follows. The document is divided in three large sections. First, a Preface on evaluability is included for the more technically inclined reader, where the above mentioned design constraints and other methodological issues are discussed. Second, the principal body of the study is presented, discussing its main findings and conclusions about the results chain -outputs, outcomes and impacts- achieved by the operation, both from a quantitative and qualitative point of view, as well as a section on recommendations. Finally, a statistical appendix is included, where all relevant data have been compiled for the record. Relevant findings The main section of the report starts by showing that the program did submit its expected deliverables (outputs) well enough in terms of quantity, quality and opportunity, so that the ensuing results (outcomes & impacts) can be claimed to be at least time-correlated with, and attributed to, the Program. The rest of the main section focuses on probing whether the Program’s development hypothesis materialized through the following results chain: Young leaders trained Knowledge gained Individual attitudes changed Organizations’ practice changed, toward a more democratic, transparent and institutionalized political system. The section ends with a discussion of the relevant study conclusions and recommendations. Since the discontinuities above described precluded a coherently comparative research of the totality of changes that may have occurred since the program’s first phase, it follows that the time relevant for observation of the results chain by the present research is not the full eight years of the two phases, but only the time elapsed since the Program resumed activities in 2009. In turn, this time span cannot, generally, be considered enough for development impacts strictly defined -in this case: permanent changes in organizational structures and practices- to have fully materialized yet. Still, the available evidence of development outcomes, including intermediate behavioral changes observed in young leaders, is more abundant and robust now than was ever found in the Program’s 1st phase. b This provides enough grounds to expect the values and habits instilled by the Program in young leaders to eventually yield structural organizational changes, as the several cohorts of program graduates first promote them internally or, by rising through the rank and file of organizations, possibly even enforce them in the future. Specifically concerning the Program’s development outcomes, the study investigates in detail the attitudinal and behavioral changes observed in Program graduates, both as trend results measured during the execution of the Program’s 2nd phase (period 2009-2011) and as end results measured after its closing (2012). The field surveys allowed a systematic comparison between attitudes and behaviors of individuals from the treatment and control groups, as measured by the following six pre-defined results indicators, required by USAID: (i) knowledge gained; (ii) implementation of training courses for the organizations’ grassroots; (iii) submission of proposals for organizational reform; (iv) increased management responsibility for young leaders; (v) increased participation of young leaders in election processes; and (vi) increased young leaders’ involvement in mechanism of municipal participation or inter-party dialogue initiatives. Comparison with individuals of control groups was impractical in the case of indicators of knowledge gain for reason discussed in the text. Researchers added the measurement of a seventh indicator: increased participation of young leaders in practices of transparency and accountability. The data on the trend outcomes, measured at the base-line point, year-end point of each cohort and program-end point through surveys administered to the same set of individuals at each point, evidence how behaviors and attitudes of the graduates (treatment group) clearly start to differ after the courses from those of the control group in all cohorts, even while the Program was still under execution. The behavioral and attitudinal trend differences are more consistent and statistically significant -i.e. more attributable to the Program- in cohorts 2009 and 2010. This result does not reflect negatively on the Program’s effectiveness because, in the researchers’ opinion, the base-line – year-end – program-end series for cohorts 2009 and 2010 are better representations of a true behavioral evolution than that for the 2011 cohort. There are two reason for this: (i) the 2011 cohort includes only two points of measurements (base-line and program-end); and (ii) more time for change in behavior naturally elapsed for the 2009 & 2010 cohorts until the Program ended in 2012. Another clear trend result established by the present study, in what concerns the Program’s development outcomes, is the pronounced and statistically significant equalization effect the program produced between sexes, concerning all measured outcome indicators. Very frequently starting with lower values at the base-line, young female leaders consistently showed progress in all relevant indicators, and frequently end up with roughly equal or higher values than those of their male counterparts. Female participants in the Program also surpass the normal percentage of participation of women in leadership positions currently existing in the political parties participating. This constitutes another clear contribution of the Program toward equalization between sexes in the political system. Program impacts were expected to be visible structural transformations in the participating organizations, formally instituting: (i) gender equity and youth participation; (ii) extended training for party members; (iii) improved transparency and accountability. The study found no evidence that the Program has so far induced any but the most tentative modernization processes inside the political c organizations involved, concerning the mentioned institutional transformations. There is, however, clear evidence for early and localized evolutions on issues such as gender equity, upward movement of graduates -especially to local leadership positions- and a minimal, but visible, transformation process to expand political education and more transparent practices. This evidence, which in some cases is circumstancial but in most cases is supported by hard and statistically significant data, suggests the presence of what we may call “trend impacts”, both general and specific. For instance, there are reasons to argue that some of the behavioral outcomes found to be statistically significant in Program graduates may already be entrenched enough in their normal practice as to keep contradicting prevailing attitudes and be maintained in the face of strong currents in the opposite direction; which may very well be the mark of a true behavioral precursor of longer term, more structural changes to come. It can also be reasonably expected that the progresively increasing presence of Program graduates in positions of leadership at all levels of the parties will eventually result in their greater weight on institutional decisions an reform processes. So, because impacts usually require extended periods to mature in practice, the longer-term changes that apparently are currently brewing just under the surface as a result of the Program may just need additional time and care to become open institutional transformations. Therefore, as it was the case with the evaluation of the 1st phase, the present study is forced to conclude that young leaders educated in the modern democracy paradigm by the Program perhaps must still gain further access to power positions in order to be more effective at their institutional change initiatives. Main conclusions The facts that: (i) all intermediate and final products of the Program were actually delivered in the quantity generally expected, with appreciable quality, and -controlling for the fact that there was a delay at the start, for reasons specified in the main text- within the time stipulated at inception; and (ii) statistically significant differences have been documented between base-line and later measurements, and between treatment and control groups, provide grounds for attribution of results to the workings of the Program, both in knowledge gained and in the ensuing behaviors observed. The statistical significance test applied to the scores achieved by students in the Program’s courses proved beyond any reasonable doubt that participants did aquire new knowledge and skills that were relevant for the eventual development of their leadership and political management competencies; especially in the case of young female leaders, whose knowledge levels the Program clearly equalized with those of the male young leaders participating. Direct comparisons with the knowldege gain of individuals from the control groups were not feasible; but, because the study factored in the possibility that individuals of the control groups might have received separate, relevant training, independent of the Program during the same period, and because results showed statistically significant differences between the two groups on that variable, with a negligible weight of such equivalent training in control groups, the conclusion is reached with high degree of confidence that the observed differences in behaviors between the two groups are more aptly d correlated with the presence of political training in the Treatment Groups, and the lack thereof in the Control Groups, than with mere chance or other factors uncontrolled-for. The conclusion about knowledge gain based on the hard evidence of statistics and likelihood tests applied to scores must be complemented by the qualitative opinion of senior political leaders interviewed, who felt that the gain of knowledge in young leaders went beyond the sole acquiring of theoretical concepts and practical skills, to actually include “experiencing politics” in a way different than that imposed by the currently dominant political culture. This aspect of the knowledge gain has been generally associated with the sort of interfacing and networking with other political and social leaders that the Program allowed among its students during the courses: a kind of interfacing in which adversaries are not necessarily seen as “enemies”, and discussions can be based on principles, ideas, rational arguments and programmatic proposal, rather than on pure sectarian prejudices; and a kind of networking conducive to healthy primary relations, instead of the usual inter-party confrontation and rancor. These are all important qualitative insights suggesting that at least some evolution toward more constructive exchange practices and relations among political leaders, and between them and the citizenry at large, showed up during the Program courses. The conclusions concerning other more permanent behavioral outcomes, that would presumably be associated with the knowledge gained from the courses, are more complicated. The study found that Program designers did include in the expected results chain behavioral outcomes that could be heavily interfered by factors originating outside the Program’s influence domain, and made the implicit assumption that those factors would concur with the Program development goals. Cases in point are the outcomes measured by indicators such as: the promotion of young leaders to positions of higher responsibility, or the participation of young leaders as candidates in election processes; which chiefly depend on decisions basically made by the current senior leadership of the organizations involved, and do not necessarily have much to do with knowledge gain or, for that matter, with any behavioral decision by the young leaders themselves. Much less vulnerable to such extraneous factors are other behavioral outcome indicators, such as: Implementation of training courses for the organizations’ grassroots; the submission of proposals for organizational reform; the increased young leaders’ involvement in municipal participation or inter-party dialogue initiatives; and the increased young leaders’ transparency & accountability. The study has found sizable differences in results between the latter type of outcome -herein called type A- and the former type - herein called type B. Concerning type B indicators, the Program implicit assumption was that the current leadership of the participating political parties and C.S.O.s would promote the appointment -or the election- of graduates from the Program to positions of higher responsibility within each organization, in order to take advantage of the “investment” made in the training of those young leaders and to further the organization’s advancement and institutional strength. Since these expectations are rational and the resulting promotions would, presumably, contribute to the effectiveness of the Program by placing progressively increasing numbers of graduates in decision-making positions, the underlying development hypothesis is cogent and essentially acceptable. However, the fact remains that the necessary concurrent factor -the decision to promote the young leaders- ultimate lies with the e organization’s current senior leadership or the relevant election processes, and not within the Program’s influence domain. Therefore, not achieving this particular type of outcome cannot be necessarily held against the quality of the Program’s performance; but not having made the concurrence of such an important external factor explicit enough as a critical assumption in the intervention’s logical framework -so that an appropriate risk management strategy might have been pursued- can certainly be held against the quality of the Program’s design. Predictably enough, the research data shows, as a general rule, that there are much greater and statistically significant changes, both between base-line and later measurements, and between the treatment and control groups, in what concerns type A indicators than those found concerning type B indicators. The forced conclusion is, therefore, that type A outcome indicator differences are generally better correlated with the workings of the Program; while type B outcome indicator results suggest the presence and workings of other factors, external to the Program. Behavioral changes measured by type A indicators have been also found to be mostly positive (upward trend) and fairly pronounced in the Treatment groups during Program execution; while the trend in Control groups concerning the same type A indicators was generally much flatter or outright downward (negative changes) in many cases, during the same period. This forces the conclusion that the Program was successful in inducing those behaviors that were more clearly under its influence domain. The most salient conclusion concerning type A outcome indicators is that, although the research confirms positive changes in the treatment groups during the execution of the Program well above those of the control group, in behaviors relating to training directed to the organizations’ grassroots and submission of reform proposals, the most statistically significant behavioral changes occurred in the areas of inter-party dialogue and accountability practices. The upward (positive) behavioral changes concerning the dialogue of graduates with political adversaries stand out, particularly because they dramatically contrasts with the clear downward trend of that behavior in the control individuals during the same period. Because this last trend could be expected in times of high political tension -as was the period of Program execution, during which two major political elections occurred- the fact that Program graduates have behaved in the opposite direction, with high statistical significance during that period, is a very salient Program outcome. Also worth emphasizing are the statistically significant differences between the treatment and control groups concerning behaviors of transparency and accountability toward superiors, subordinates and the general public. These results are remarkable because they also contrast with an observed downward trend in the corresponding behavior of individuals in the control groups, during the same period. The positive trend in the treatment groups is even more remarkable in the case of disclosure toward the outside public, because this is neither a mandatory, nor a normally expected behavior -as, for instance, reporting to superiors might be in hierarchical organizations such as political parties- and arguably constitutes a truer sign of transparency. Behavioral changes have been generally found to be even more preponderant, stronger and more statistically significant in the case of young female leaders than in their male counterparts, in all cohorts and in all type A outcome indicators. Therefore, there are grounds to support the conclusion that these new Program-induced behaviors may very well be precursor signs of more permantent transformations to come, as such behaviors f get entrenched enough in the young leaders’ normal practice to even contradict prevailing attitudes; and the demostration effect of such behaviors generate some contagion inside their organizations; and as the clear gender equalization effect of the Program strengthen the young female leaders’ positions and their weight in future decision along the direction they are showing in their behavior; i.e.: one which is more inclined to education, institutional reform, inter-party duialogue and habits of transparency. However this conclusion is dampened by the more dismal results obtained in the type B outcome indicators, which show that the external factors required to concur in order for these outcomes to materialize, have not necessarily concurred; especially in the case of those associated with decisions by current senior leadership in the political parties. Not all type B indicators showed “dismal” results during the Program execution. Concerning the participation of young leaders as candidates in election processes the observed trend has been positive in all course cohorts, both in the individuals of control and treatment groups. The problem is, however, that not only changes in the value of that indicator are relatively small and not statistically significant, but that the trend is exactly the opposite in what concerns young leaders being promoted (appointed) to higher positions. This trend has been verified also both in control and treatment groups, with small and not statistically significant differences between the two, as well. Since statistical tests suggest a low correlation with the workings of the Program, the conclusion is forced that these results should be taken as consequences of factors operating outside the influence domain of the intervention. The fact remains, however, that the promotion decisions made by current senior leadership of the organizations appear not to have aligned with the Program development hypothesis, quite as expected. The evidence also gives ground to the conclusion that the relative lack of support by authorities, especially in political parties, has not only affected the promotion of Program graduates up the organizational ranks. It has also been manifest in the somewhat tepid reception of graduates’ initiatives inside the organizations. Surveys show that significant pluralities or majorities of graduates have developed and submitted proposal both in the area of education and in the area of institutional reforms, toward modernization in practices and procedures. But also significant pluralities or majorities have reported that their initiatives in the educational area did not necessarily meet with a corresponding proactivity on the part of the senior leadership in their respective political organizations. The relative resistance, or lack of attention, from senior leaders to the institutional reform initiatives of the Program graduates can also be inferred from the relative small proportion of young leaders reporting that such initiatives were approved or met with success. Even among those who felt that their initiatives succeeded, in both areas of concern, very small proportions of survey respondents cared to mention specifically that authorities in their organizations have lent support to the initiatives Additional complementary conclusions are afforded by data from field surveys done after the closing of the Program, which shed light on the medium-term persistence of behavioral changes observed during the Program execution, thus allowing some confirmation of changes, or providing important qualifications to those trend results; as well as additional qualitative angles for interpretation of the data. These ex post surveys have revealed an all important exception to the general lackluster g support from senior leadership in political parties, that we discussed above. This prominent exception relates, once again, to the performance of young female leaders participating in the Program. A first important general conclusion on female young leaders is that, not only they have consistently outperformed male young leaders in all type A behavioral outcomes, but they have done so in type B behavioral outcomes as well. For instance, despite the downward trend in the promotion of young leaders in general -treatment and control groups- the opposite has consistently occurred in the case of female graduates who, as a final result, have been promoted in greater proportions than their male counterparts; the majority of promotions occurring to provincial or muncipal levels and below in the organizations. Also, the ascend of young leaders to positions of higher responsibility does not only occur through direct promotion (appointment) but also through open election processes. In this sense, the ex post data confirms that a significant majority of program graduates have run for elective posts, and that also a significant portion of those running have actually won the elections they run in. This result is even more pronounced in the case of female graduates, with statistically significant differences with respect to the electoral succes of male graduates. The conclusion follows, therefore, that, if sustained, the more abundant presence of Program female graduates in leadership posts within the organizations, promoted by appointment or by election, can be expected to eventually result in a greater weight exerted by women on institutional decisions and reform processes, especially within the political parties. Finally, another important conclusion concerning female young leaders is that they have also been significantly more succesful than male young leaders in having their initiatives and proposals approved by the superiority in their political parties. All these results confirmed for female Program graduates restores much credence to the conclusion that long term impacts of the Program, although not openly evident yet, may indeed be just “brewing under the surface” and require additional time and care to be realized. Recommendations The section on recommendations include a number of sugestions based on the study analysys and conclusions, and designed to feed-back political parties, the private sector, civil society organizations and the USAID mission, about ways to build on achievements and keep improving performance in future political modernization efforts of this type. Recommendations run the gamut from issues of methodology and program design through ways to improve the intervention development effectiveness and sustainability. First, in the researchers opinion, the fact that the study yields enough precursor evidences to suggest that at least some of the final structural transformations intended by the Program (1st and 2nd phases) may be already in the offing -albeit not openly visible yet- argues in favor of maintaining this development intervention with the design improvements suggested; and this not only in order to ensure long term impacts, but also to avoid the wastage represented by the possibility that hard-fought-for changes, which are still budding, may be reversed at the end by the forces resisting modernization, for want of a more prescient and perseverant educational effort. h Researchers also recommend increasing the intervention’s potential to induce structural changes by better identifying possible threats to effectiveness originating outside the interventions’ curfew (external factors) that may be mitigated through risk management strategies; and better targeting factors retarding political modernization which are anchored in the social environment. Ways are suggested to adjust the course’s design to further help young leaders in sharpening the focus and enhancing the practical feasibility of their reform proposal within their respective organizations. These include inducing more systematically in the participants the practice of external outreach, especially vis-à-vis the community at large, and finding ways to strengthen the graduates’ ability to identify and handle issues of incidence and viability of organizational change, and to better flesh out their initiatives with colleagues and superiors. Other recommendations refer to ways of making the learning experience of participants in the Program courses more result-oriented, including the addition of practical activities, shared with other leaders, as part of the curricular design, and emphasizing the training on digital networking early on in the courses, in order to elicit in the participants the associated habits as soon as possible in the process. Recommendations are also included for better handling key methodological design aspects that, in the researchers’ opinion, produced difficulties that could be avoided concerning the measurement of Program development results; such as the decision to repeat each year the selection of control and treatment groups, which may have exacerbated sensitivities concerning the selection criteria and processes. It is also recommended to apply demographic proportionality criteria in programming courses per region, and a closer supervision of program monitoring studies, to preclude such practices as the application of excessively prolific survey questionnaires and undue changes in survey questions. Finally, some suggestions are also included concerning longer term sustainability of the effort, by better inducing and increased involvement of local resources, and ownership of the Program by the political leadership in the country. i PREFACE ON PROGRAM EVALUABILITY I. Conceptual approach and constrains 1.1 While methodological changes in the monitoring and evaluation framework for the Program’s 2nd phase have been thoroughly incorporated to the present study, in part based on recommendations of the 1st phase’s evaluation study, the evaluation herein presented keeps the main conceptual approach used in that previous evaluation. Specifically, this study assesses the development results of the Program based on the measurement of indicators, in line with the stipulations of the Log Frame methodology. These and other issues of evaluability are discussed below. A. Issues on the Program metrics 1.2 According to the study scope of work (s.o.w.) a technical assistance provided to executors before the start of the Program’s 2nd phase suggested changing the evaluation metrics from “benchmark indicators” to “pure” (non-directional) indicators. This implied a substantial conceptual change in the way the Program´s performance was to be judged, in comparison to what was done in the 1st phase.1 1.3 The present researchers recognize that there may be valid reasons to choose non directional indicators, not the least of which is that specific indicator targets may be hard to establish in the absence of enough proven theory or experience in particular interventions; especially in what concerns expected outcomes and impacts. However, by providing clear directionality to interventions, the use of benchmark indicators aligns well with the current USAID Evaluation Policy that, for purposes of accountability … requires comparing performance to ex ante commitments and targets… 2 . Also, the present researchers have confirmed that the Program designer stressed the need for setting targets on indicators reference sheets, developed for that purpose, as well as the importance of establishing methods for target setting. Therefore, as it was the case in the evaluation of the Program’s 1st phase, researchers approached the present evaluation by, whenever possible, first assessing whether the program under investigation achieved or not its own ex-ante stated operational targets and development objectives. 1 In the standard log frame model, metrics are not understood as just “indicators” in the common parlance sense of the word. The model requires “benchmark indicators”; i.e., metrics that, beside a precise formula for measuring changes in the relevant variables, also include a “standard” (target or reference value) against which to judge changes actually achieved and in comparison with a starting value (base-line). This approach directly associates metrics with program objectives and provides a clearer-cut gage for the “desirability” of actual results. For instance, as opposed to the indicator: Tons of produce/Hectare, the benchmark indicator: Yield to increase from 2 to 5 Tons of produce/Hectare in 3 years, allows program/policy managers to judge more precisely if actual yields achieved were acceptable or not. Conversely, “pure” indicators make evaluations to focus on: (i) providing a non-normative judgment on performance (i.e.: not based on comparison to any target); and (ii) judging the incidence and quantum of results, not their direction. 2 See Purposes of Evaluation USAID Evaluation Policy www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf , p. 2 ii Figure A: The Results Chain Impact Outcome Output Input        ACTION RESULTS = = 1.4 Other unintended or indirect program consequences, as well as the role of any external factor at play in shaping the actual end-results, were also to be studied; but only second to judging effectiveness against the program’s original intent. This brings to the fore the issue of evaluability at entry of the this 2nd phase of the Program, i.e.: the extent to which the Program’s performance can be judged against a pre-ordained set of rules and measures established in its own internal documentation, and/or whether those elements were present at the outset. B. Issues on the Program’s expected Results Chain 1.5 The modern conceptual approach to judging development effectiveness requires that projects identify ex ante the results chain they will pursue with their execution, as well as the set of associated performance metrics and envisioned factors that may represent risks to effectiveness.3 Based on the definitions by the Development Assistance Committee of the OECD, our operational understanding of the relevant result chain is illustrated in figure A4 . If a development program can be succinctly understood as the binomial: ACTION RESULTS the “results” member can then be though to include the directional and sequential set of outputs, outcomes and impacts expected from the deployment of the program’s inputs; set which normally is laid out in a program’s so called Logical Framework or log frame. For purposes of the present research we define the elements of the relevant results chain as follows: (i) Outputs: First tangible results. Clear throughput coming directly off program activities. Outputs are usually equivalent to the so-called program “deliverables”, upon which management has maximal direct control. (ii) Outcomes: Second tier results. Immediate changes taking place in the program’s target reality, imputable to a program’s output delivery or the output’s onset/workings at the end of execution. Program managers normally have no direct control on this category of results, yet outcomes form part of the causal hypothesis explicit or implicit in most program designs. From an ex post viewpoint, outcomes may be directly or indirectly attributable to outputs and dimmed positive or negative, intended or not. (iii) Impacts: Third tier results. Changes attributable to the program, taking place over a longer time span after execution has finished, and/or in a wider, more complex and farther reaching context surrounding the program’s target reality. Control over this category of results is even more problematic and attribution more difficult to establish or argue; usually requiring from the outset the establishments of control groups and experimental designs and settings. Yet, 3 See proceedings of the International Conference on Financing for Development, Monterrey, México, 2002; and the Marrakech International Roundtable on Results, 2004. 4 See OECD DAC Working Party’s Glossary of Key Terms in Evaluation and Results Based Management, p. 33. iii impacts also form part of the causal hypothesis explicit or implicit in most program designs. From an ex post viewpoint, impacts may be directly or indirectly attributable to outcomes and dimmed positive or negative, intended or not. (iv) Intermediate results: Earlier outputs, outcomes or impacts, attributable to the program. Intermediate outcomes or impacts can possibly occur while the program is still under execution. From an ex post viewpoint, intermediate results may be directly or indirectly attributable to the related elements in the results chain and dimmed positive or negative, intended or not. (v) In “ex ante” log frame parlance, impacts correspond to the professed goal of a program; outcomes relate to its intended purpose; and outputs are directly associated with the program’s components. 1.6 Although by its scope of work the present evaluation’s purported focus is the high-end portion of the results chain (possible impacts) the research does not ignore precursor elements in the results chain for two main reasons. First, the exclusive emphasis on establishing impacts without verifying in the same breath the concomitant occurrence of precursor outputs and outcomes would leave the final argument without proper grounds for attribution (i.e. to be able to impute the former to the latter, by at least arguing time correlation thereupon). Secondly, given the short time elapsed since the program 2nd phase’s start, impacts -as strictly defined- may very well have not had time to fully mature yet, while other important elements of the results chain -especially outcomes- might arguably be more feasible and currently visible. The present evaluation, therefore, studies the program’s whole result chain to the extent that is evaluable at this point in time. 1.7 Beyond the refining of the program´s Results Framework, major modifications made for the 2nd phase of the Program included the introduction of control and intervention groups, and changes in field measurements to be made by the program´s executors in the 2nd phase. The present researches have adapted their study to these conditions; however, the following conceptual precisions were adopted in order to maintain the research´s consistence with standard evaluation theory and practices. C. Issues on the evaluation design 1.8 As noted above, and complying with the new USAID Evaluation Policy, the study´s s.o.w. indicated that the Agency has mandated an “experimental” design for the Program´s impact evaluation (IE), whereby control and intervention groups have been established to ensure that measured changes in the relevant variables are clearly attributable to the intervention -on the basis of the “theory of change” (cause and effect model) for the Program- and to provide … control for factors other than the intervention that might account for the observed change.5 Researchers have followed these 5 See USAID Evaluation Policy, in www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf, p. 2 iv directives in the present evaluation; but, in doing that, they have also taken into consideration the following important caveats to the conceptual approach outlaid by the USAID policy, in order to comply with standard Log Frame conceptual approach and best practices. 1. Attribution hypothesis in the program’s expected results chain 1.9 The present researchers recognize that the issue of “attribution” (in other words: the assertion of cause-effect) not only concerns the impacts of an intervention, but is actually relevant to all elements of the Results Chain. The standard Log Frame approach to evaluation of development projects postulates that the expected results chain of an intervention is, in fact, a cause-effect model (program theory) in which each element of the ordered set would be attributable to the element just below in the sequence, as was depicted in figure A of the previous section. 1.10 But the standard Log Frame conceptual approach also asserts that this attribution may always be limited, qualified or otherwise affected by factors originating outside the control domain of the intervention (external factors) that generally act as disturbing vectors, negatively or positively influencing results (all results, not only the impacts). Notice that, beside the upward vectors depicting the causation (attribution) sequence expected in a generalized results chain, figure B depicts such external factors as horizontal vectors on the right-hand side of the figure. 1.11 Furthermore, the control that the intervention (i.e. the program itself, through their managers, etc.) can exert on the causation process would only be really strong in the “early” portion of the sequence, below what in Log Frame parlance is called the Management Responsibility Frontier. Notice, in figure B, how this early portion is depicted by the darkest upward vector linking inputs and outputs below the frontier. Notice also how this control grows weaker (fainter vectors) the farther we go upward toward the “late” portion of the sequence of results, implying that beyond that frontier managers and executors of an intervention don not exert effective control on results. 1.12 By the same logic, the opposite is expected to be true for the external factors, which would exert the strongest possible influence (see darker leftward vectors in figure B upper right-hand side) on the “late” portion of the chain (expected outcomes and impacts) and a weaker influence below the Management Responsibility Frontier, where control of factors by the intervention is maximal (see the faintest leftward vector on figure B lower right-hand side). Figure B: Attribution and External Factors in the Results Chain v 1.13 Also, the same logic is applicable if the Results Chain is further subdivided into a more detailed sequence, as depicted in Figure C. The increasingly darker shade of the leftward vectors in the figure illustrates that the section of the results chain where outside factors may really start to be significant -adding or subtracting- for results is the section beyond the delivery of outputs onward. Thus, it is for this section of the Results Chain that the use of control and intervention groups becomes significant to cancel out the effects of chance or of any other unknown external factor that may add to, or subtract from, “net” program results. Consequently, the researchers have used this approach to gage results attributable to the intervention not only in terms of impacts (final), but also in terms of every other result for which the changes can be compared between control and intervention groups; thereby strengthening and clarifying the attribution argument for intermediate outcomes, final outcomes, and intermediate impacts as well. So, based on the above discussion and terminology, the researchers understand the expression Impact Evaluation (IE) to really mean: Development Results Evaluation, for which the “experimental” design mentioned in the study´s s.o.w. has been adopted. 6 1.14 By the same token, and in terms of the accountability lessons to be learned from the present study, the present researchers recognize that the attribution of development outcomes and impacts to the FYL program are primarily a function of the quality of the program’s design and theory, and not the direct responsibility of the intervention´s management -provided outputs are delivered in time, quantity and quality by the executing agency in charge of implementation, as expected. This is also in line with language in the current USAID Evaluation Policy that has essentially recognized the nuanced distinction between control and influence.7 Control -however not absolute- can only be associated with outputs, while it is completely lost in the case of outcomes (and impacts). Also 6 It is also appropriate to clarify in passing that, strictly speaking, full control over all variables except for the ones under study (dependent variables) is only possible in the realm of the “closed” systems of physics and chemistry, where the number of variables in a particular systems can be said to be finite; and, thus, truly controlled “experiments” can be devised for research. In “open” systems with possibly infinite variables and factors influencing behavior -as human systems are- truly experimental research or intervention designs are not generally deemed possible. Therefore, the researchers adhere to standard technical language and practices, and the present research is presented as based on a “quasi-experimental” project design, an expression more adequate for social sciences. 7 In its section on Accountability the Policy advocates measuring …outputs and outcomes that are under the control or the sphere of influence of the Agency. (My emphasis) See USAID Evaluation Policy, in www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf, p. 3 Figure C: Attribution & External Factors, nuanced Results Chain vi interventions can only claim influence on outcomes and impacts: and one that would chiefly depend on the correctness of the cause-effect hypothesis adopted in the program´s theory of change. 1.15 Therefore, to enhance the feed-back to stakeholders involved in the FYL program, the present researchers went beyond the sole statistical control for outside factors afforded by the quasi￾experimental design of the present evaluation, and complemented the study with specific, qualitative identification of such external factors that may have affected the effectiveness of the program in terms of outcomes and impacts (intermediate and final). Finally, while understanding the quasi-experimental design is not relevant to the section of the results chain below the Management Responsibility Frontier, the researchers have identified factors outside the control domain of management that may have affected outputs (deliverables) through direct research of the operational conditions of the program´s execution. 2. Field measurements 1.16 The study’s s.o.w. stipulates that the evaluation should analyze the quantitative data already compiled by the program´s executor, who were expected to have made: (i) three yearly base-line surveys; (ii) three year-end surveys; and (iii) a program-end survey with intervention and control groups. This implies, therefore, a “moving base-line”. The researchers also understand that the present evaluation has been requested to produce a separate, independent research on results that may confirm, qualify or otherwise complement results found by the Consortium’s measurements during program execution. Certainly, independent corroboration of program results falls in line with current Agency policy that calls for evaluations that … are not subject to the perception or reality of biased measurement or reporting due to conflict of interest or other factors. 8 1.17 The present study, therefore, includes field measurements of program results in samples from intervention and control groups additional to, and separate from, the measurement previously made by the Program itself during execution. In these new measurements, the present researchers have applied rigorous techniques to reach a high level of statistical significance and power in the comparison between intervention and control groups, commensurate to those applied by the executors (hereinafter: “the Consortium”). When sub-group comparisons have been made along criteria such as gender, organization and region, statistical significance of results has critically depended on the researchers’ ability to maintain a proper balance of sample size not only between the general intervention and control groups as a combined whole, but in the corresponding subdivisions of such samples as well. This has been normally possible, with the exception of the East and North-East regions of the study. In the case of these two regions the subdivision of the statistical sample in a matrix by gender and organizations yielded too many “empty cells” in the matrix, forcing researchers to lump the two regions into a single one, to obtain proper representation, by sub￾categories. 8 USAID Evaluation Policy, www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf, p. 6 vii 1.18 Also, since the present researcher’s field surveys had to be based essentially on the same general population participating in the program, both as “eligible” candidates (control group) and “actual” participants (intervention group) in the Program, consistency of the researchers’ survey results with the stipulations of USAID has depended on the extent to which the Consortium has been able to faithfully apply the strategy and procedure for assignment of target population individuals to available “seats” in the Program´s courses, as agreed with the Technical Assistance given to the Consortium and referenced in the s.o.w. Researchers found that executors did change the selection criteria used in the 1st phase of the Program, essentially based on aptitudes within the boundaries of a quota system agreed on with the parties, replacing it with a random assignment procedure within the said quota system, as requested by USAID for the 2nd phase. 1.19 The s.o.w. pointed out the existence of potential threats to the balance between intervention and control groups. These threats were: (i) restrictions in the anticipated application pool, (ii) the rigid quota system agreed on with the political parties; and (iii) the policy of accommodating re-applicants to the program course in a year by extracting them from the control group of the previous year. It was thought, specifically, that these threats might make the control group biased towards the region, party and gender submitting the most applications to the program. This bias was expected to appear especially during the first year of the Program 2nd phase, and the Consortium was to find ways and means to avoid this bias in the actual selection, whenever it might appear. Researchers found that, despite the strain that random selection exerted on the political parties’ expectations concerning the Program, executors did make their best efforts to avoid bias and maintain proper statistical balance in the selection process. II. Is this Program results chain clearly identifiable and evaluable? 2.1 Within the approach outlaid above, the researchers’ first order of business has been to identify clearly the program’s results chain, benchmark indicators, and possible ineffectiveness risk factors identified at the start. On this score and with the exemption of risk factors -which were not identified ex ante- the documentation review renders improved results in comparison to those of the Program’s 1st phase. In effect, the requirements of evaluability at entry, as defined above in paragraph 1.4 of this section, have been better fulfilled in the Program 2nd phase’s design. Designers clearly made an effort to adopt a more streamlined, prudent, less ambitious and more measurable expected chain of results for this stage of the effort; as well as to establish a more balanced and concise set of indicators, thus heeding recommendations #6 and #7 of the 1st evaluation study. They also identified useful metrics to gage the program results. 2.2 According to the scope of work of the present study the general structure of the 2nd phase of the Formation of Young Leaders Program is presented on the next page. That structure presents the Program’s objectives as narrated in the Logical Framework and as associated to the expected Results Chain. The corresponding metrics expressed in a generalized way are also included. Minor imprecisions and deviations from standard concepts and language have been corrected or explained in a footnote. viii PROGRAM’S 2 ND PHASE LOGICAL STRUCTURE OBJECTIVES IN LOG FRAME EXPECTED RESULTS CHAIN METRICS Indicators of success in GOAL institutional change (IMPACT) Indicators of initiatives taken toward institution- nal change (OUTCOME) - Indicators of knowledge gain by political leader- ship and management (INTERMEDIATE OUTCOME) PURPOSE - Indicators of increased capacity for Inter-party dialogue (INTERMEDIATE OUTCOME)** - Indicators of knowledge gain on course facilitation (INTERMEDIATE OUTCOME)** - Indicators of knowledge gain on requirements for internal democracy, transparency, and institutionalization (INTERMEDIATE OUTCOME)** - Indicators of courses in leadership & political management given (OUTPUT) - Indicators of people trained in leadership and political management (OUTPUT) COMPONENTS - Indicators of PAT courses given (OUTPUT) - Indicators of PAT graduates (OUTPUT) Number of studies Number of events (*) Items that were somewhat misplaced by the s.o.w. in the “component” (outputs) row of the log frame, when in fact they are outcomes. (**) Indicators that would have been necessary, but that were not considered in the Program 2.3 Accordingly, significant changes in political party behavior by virtue of initiatives undertaken by young leaders who had been exposed to the FYL Program, was the main expected result chain stipulated by the Program from the beginning. In other words: the deployed education and technical assistance effort was supposed to have made development sense to the extent that it had induced transformations towards modernizing the political system and leadership in the country. Unfortunately, target values for this results chain were only identified clearly for final output indicators, and were “moving” and inconsistent for the indicators in the rest of the results chain. Modernization of the political system and leadership in the Dominican Republic Young leaders of political parties and CSOs contribute to the modernization of their organizations - Young leaders strengthen their knowledge of political leadership and management* - Young leaders strengthen their capacity for interparty dialogue* - Young PAT graduates strengthen their knowledge of course facilitation* - Increased knowledge among party leaders and CSOs of their requirements in terms of internal democracy, transparency, and institutionalization* 1.1 Training provided in political leadership and management 1.2 PAT training provided 2. Requirement studies done 3. Inter-party dialogue events held ix A. Comparability with evaluation of the 1st phase 2.4 The above specified expected results chain is essentially equivalent to that of the reconstructed Results Chain of the program’s 1st phase, as is shown in the following table. As before, minor deviations from standard concepts and language have been corrected and explained in a footnote. EXPECTED RESULTS PHASE 1 EXPECTED RESULTS PHASE 2 INDICATORS PHASE 1 INDICATORS PHASE 2 ORGANIZATIONS’ PRACTICES MODIFIED (Impact) Modernization of the political system and leadership in the Dominican Republic - Indicators of induced Institutional change - Number of political parties that institute mechanisms for strengthening both gender equity and youth participation in their parties. - Percentage of political parties that institutionalize training programs - Number of political parties that institutionalize mechanisms for transparency and accountability in accordance with the Electoral Law INDIVIDUAL ATTITUDES CHANGED (Outcome) Young leaders of political parties and CSOs contribute to the modernization of their organizations - Indicators of reform initiatives undertaken - Percentage of young leaders providing accountability of their acts or encouraging accountability within t heir organizations - Number of young leaders who implement courses for the party base* - Number of institutional reform proposals presented by young leaders that are implemented by their organization* - Number of young leaders who have increased their level of management in their organization* - Percentage of young leaders who increase their participation in electoral processes* - Percentage of young leaders who increase their participation in mechanisms of municipal participation or in inter-party dialogue initiatives * - Number of participants who implement courses within their organization through the PAT project ** - Network established by graduates with representation from political parties and CSOs ** - Percentage of participants who increase the knowledge on political management and leadership* ** KNOWLEDGE GAINED (Intermediate Outcome) -Young leaders strengthen their knowledge of political leadership and management -Young PAT graduates strengthen their knowledge of course facilitation -Increased knowledge among party leaders and CSOs of their requirements in terms of internal democracy, transparency and institutionalization -Learning & academic indicators YOUNG LEADERS TRAINED (Output) - Young leaders participate in leadership and political management courses - Young leaders participate in PAT training - Political leaders participate in training on requirements for internal democracy, transparency and institutionalization - Rates of graduation, desertion - Number of people trained in political management (graduates disaggregated by course type) - Number of participants trained in facilitating the political management course (graduates of PAT) - Number of representatives of political parties and CSOs that participate in meetings and socialization events on the topic requirements for internal democracy, institutionalization and transparency TRAINING PROVIDED (Intermediate Output) - Leadership, political management & PAT training - Studies on modernization - Inter-party dialogue - Indicators of courses & seminars given - Courses given on leadership and political management*** - Courses given on facilitating the political management course*** Percentage of graduates and students participating in program webpage and virtual forum - Number of studies created on the modernization requirements of the political system including gender equity and women empowerment - Number of events to strengthen inter-party dialogue NOTE: Indicators marked with (*) are the ones the s.o.w. indicates should be measured and compared in Control and Treatment groups. Indicators marked with (**) appear originally misplaced as pertaining to outputs, instead of outcomes. Indicators marked with (***) are not included in the s.o.w. but are required to measure intermediate outputs 2.5 However, despite this congruity, the extent of changes adopted for the 2nd phase has created discontinuities in the evaluation premises between the two phases, rendering impossible the goal of making the present evaluation epistemologically valid for the Program as a whole. Particularly, changes in the Input Output section of the program’s extended Results Chain arguably modified the underlying attribution hypothesis, making the outcomes and impacts of phase two not directly x comparable to those of the first phase. First, while the treatment effort in phase one concentrated largely on “regular” political training courses, the number of these decreased while other components (outputs) were added to help induce institutional change. Components such as Modernization Studies and Interparty Dialogues were important complementary interventions in phase two, effectively diversifying the lower tranche of the Results Chain, as depicted in figure D. 2.6 Second, the inputs of the program also changed. The contents of the regular course underwent appreciable modifications through a curriculum redesign and more teaching hours, as well as the addition of a special tutorship effort to help students along in the preparation of their subsequent institutional proposals. Participants were trained to facilitate these regular courses down the line in their organizations through the so-called P.A.T. component, which was a much more important output than it ever was in the first phase. Also, specialized courses were added. Therefore, beside the fact that the evaluation framework of the 1st phase of the Program is not methodologically consistent with that of the 2nd phase; results of the Program’s 2nd phase cannot be attributed to inputs qualitatively and quantitatively comparable to those of the 1st phase of the Program either. B. Comparability with surveys made during execution of the Program 2nd phase 2.7 The present researchers have also encountered other issues of comparability, relating to the base￾line, year-end and program-end surveys made by the Consortium during execution. The first of these issues is the profusion of variables in the resulting data bases, suggesting that a number of measurements were made well beyond the set of indicators stipulated in the program monitoring and evaluation system. Also most of the variables appeared to bear little direct pertinence or significance in terms of what the present evaluation aims to establish. A second issue relates to fact that the number and narrative of the resulting variables actually changed between the various base￾line surveys, and even between the base-line survey and the year-end survey of the same period. This compromised the comparability of results, not only between the several survey cohorts but also between the base-line and the year-end results within a given year. Thirdly, even within the same variable, the way actual questions were posed to participants also changed between surveys, adding uncertainty about what was being measured and further compromising comparability of results. 2.8 In making comparative analyses, the present researchers resolved the limitations above indicated, by ignoring most variables in the surveys data base and restricting the comparison only to those variables that complied with the following conditions: (i) were mostly common to all surveys –i.e.: save for minor stylistic variations in the narrative-; (ii) were present in the base-line, the end-year and the program-end surveys; and (iii) were pertinent to the limited set of indicators slated in the s.o.w. to be measured in the field, on both the control and the treatment groups. Political system and leadership modernized Young leaders contri￾bute to modernization Modernization Training given studies done Interparty dia- logues done Regular course, PAT, Special courses IMPACT OUTCOME OUTPUT INPUT Figure D: 2 nd phase’s more diversified Results Chain 1 EVALUATION OF THE PROGRAM FOR FORMATION OF YOUNG LEADERS, 2 ND PHASE I. Introduction A. Goal of the study 1.1 The present study investigates the degree of success (development effectiveness) of the USAID￾sponsored FYL Program in its 2nd phase in the Dominican Republic, as measured by the observable transformations in the way the political leadership and system do business in the country (toward modernization), and inasmuch as these transformations are attributable to the changes in competencies and behavior patterns induced by the intervention in its target population (young political and social leaders). The overarching goal of the evaluation is to identify sustainable results and practices that could be shared with political parties, the private sector, civil society organizations and the USAID mission, to successfully further the modernization of the political system and leadership in the D.R. B. Focus of research 1.2 The present researchers endeavored to measure the program´s effectiveness, analyze its implications, identify lessons and provide feed-back on the Program’s operational and strategic guidelines. The research addressed, among others, the following key issues: (i) the modernization and democratization of political parties; (ii) the adoption of training programs as a standard practice by political parties; (iii) the importance given to training programs by political parties; (iv) the increased participation of young leaders on these training programs; (v) the promotion of young leader to party leadership positions; and, (vi) the training imparted to young leaders in the political parties´ schools The researchers applied mixed tools to answer the above key research questions, among others, using quantitative data on results, as well as qualitative information from interviews; yet also avoiding conclusions based on anecdotes, hearsay or sole compilation of people´s opinions. 1.3 Seeking to frame the study on the wider possible international consensus about the relevant underlying theory and notions, the Researchers performed the evaluation within the conceptual approach to development effectiveness as laid out in the Monterrey Consensus and in the Marrackech Memorandum; and focused on the notion of Results Chain as established through international agreement by the OECD.9 Particularly, the researchers concentrated on assessing the Program’s sequence of its outputs, outcomes, and impacts; critically investigating the associated 9 For a summary of these consensus-based concepts, see: (i) proceedings of the International Conference on Financing for Development, Monterrey, México 2002; (ii) proceedings of the Marrakech International Roundtable on Results, 2004; and (iii) the OECD, DAC Working Party’s Glossary of Key Terms in Evaluation and Results Based Management 2 metrics and giving prime attention to the attribution issues that may arise between the means deployed by the Program and its imputed results. 10 1.4 Attribution is inferred by carefully comparing changes observable in the program´s treatment groups, with changes observed in the control groups; thereby establishing that any variations between the two were correlated with the program intervention (deployed inputs and delivered outputs). In order to guaranty a scientifically acceptable level of confidence in the attribution of results, researchers applied adequate statistical techniques to systematically minimize the effect of random errors in the measurement process, as well as that of inherent variations in the measured variables. 1.5 The evaluation purports to shed light on “later” elements of the results chain (impacts), yet not ignoring precursor elements -which are arguably more feasible, and currently visible, given the time elapsed since the program’s start- such as outcomes. Particularly, the correct delivery of outputs will be also ascertained in order to establish the plausibility of links and maximize attribution in the whole chain of results: outputs-outcomes-impacts. In this process researchers also endeavored to discover and substantiate intermediate results imputable to the Program, as well as to identify external factors that might have affected causal processes and explain away ineffectiveness. 1.6 Finally, attribution is argued herein not in the strict sense of causality, but in the sense of statistical correlation between the Program’s deployed means, on one side, and imputed results on the other. The researchers probed the underlying hypothesis through high power tests for statistical significance, so that the probability of results occurring by chance is minimized and confidence in the resulting inference is maximized. C. Information & data collection 1.7 The researchers used: (i) content-analysis of the documental evidence, including that from intermediate field surveys performed during program execution; (ii) direct field research of target populations, based on appropriate statistical samples; (iii) interviews with key non target actors and stakeholders; and (iv) qualitative analysis of data compiled. 1.8 Measurements already available were reviewed to establish evidence for such results as courses given, complementary activities done, learning achievements, etc. and the changes induced in the relevant target reality, such as modifications in behavior patterns by political and social leaders and internal practices and demeanor of political parties and organizations of the civil society in the areas of interest; as well as their relations with the body social beyond the organizations’ boundaries. New 10 Inputs, as well as other elements, are also sometimes included in the definition of a generalized Results Chain (See, for instance, OECD, DAC Working Party’s Glossary of Key Terms in Evaluation and Results Based Management, page 33). However, not being “results” of the intervention in any strict sense of the word, inputs are not included among the subjects of the present research as a relevant part of the Results Chain. For further clarification of the notion of Results Chain and other technical concepts used in the present evaluation see the Preface on Evaluability. 3 field measurements were done for the set of indicators mandated by the s.o.w., both in the Control and Treatment groups, paying special attention the appropriateness of the statistical samples used. 1. Statistical sampling 1.9 The Researchers carefully selected an appropriate sample of the treatment group (Program´s target population) and the control group combined, to collect primary data during the study´s field work. Because establishing the after-program difference between the treatment and control groups is essential to establish results attributable to the Program, confidence was maximized in the capacity of the study to ensure that any such difference were not due to random errors arising from the measuring process or from chance variations in the relevant variables. The Researchers maximized this confidence by increasing the power of the study´s statistical test, establishing a sample size, a level of statistical significance and anticipated effect size that minimized possibly confounding factors, including the selection bias. 1.10 The Researchers used the specialized software application Statistical Package for the Social Sciences (SPSS) to determine the minimum sample size required to reach 80% power in the statistical test, and estimated the parameters consistent with this upper bound of the required sample size; namely: 0.4 standard deviations for the effect size; 0.05 for statistical significance level; and 0 for the variations explained by co-variants (R2). While maintaining the resulting sample size of the combined population constant, the study endeavored to select an equal number of individuals from the treatment group and from the control group for the field research, in order to increase the power of the statistical test as much as possible. These were general “study goals” that the researchers tried to achieve, whenever the data available permitted. It is important to note, however, that a different situation was encountered between, on one hand, the analysis of the Consortium’s databases on the indicators measured during program execution -which was based on the universe of the population surveyed- and, on the other, the analysis of the ex post evaluation survey, which was done by the present researchers based on a statistical sample of the population. 1.11 By establishing the above parameters for statistical significance, the Researcher sought to guarantee that there was a minimum chance (below 5%) that research results occurred only by chance. Also, since the effect size is usually expressed in terms of standard deviations of the variable under study, the Researchers reviewed and analyzed previously collected data, including the base-line measurements, to establish the mean and variance of the relevant variables. 2. Field work 1.12 Beside the program direct beneficiaries; i.e. the set of alumni who received the program’s courses and are the expected “main vectors” of change, and the members of the control groups in their different subdivisions, non-target stakeholders who were also studied during the field work included: (i) the political and social leaders under which the direct participants operate routinely; (ii) lecturers and academics who dictated the program courses; and (vi) program managers in charge of coordinating and supervising program activities. 4 II. Main findings and analysis A. Program intermediate outputs 2.1 The first line of products delivered by the Program includes: (i) Training courses; (ii) Inter-party dialogue events; and (iii) Studies on modernization requirements for the political party system. These products included several sub-products as summarized in table 1 below. PRODUCTS SUB-PRODUCTS Training courses Regular course on Leadership & Political Management P.A.T courses for training trainers in the Regular Course Specialized courses Inter-party dialogue events Regional encounters Regional dialogues Young Leaders Congress & Network Studies on modernization requirements Focus group meetings Events for socialization of focal group meetings results Diagnosis of the party system 2.2 The delivery of these early products were the basic means towards achieving the Program’s main output, namely: YOUNG LEADERS TRAINED on proper concepts and practices to improve internal democracy, transparency and institutionalization of organizations; and to further promote the rest of development outcomes and impacts, up the Program’s expected results chain; namely: actual KNOWLEDGE GAINED by these young leaders on values and skills for modern leadership and political management; which should in turn induce INDIVIDUAL ATTITUDES CHANGE in them, leading toward reform initiatives in their respective organizations; which should eventually contribute to the ORGANIZATIONS’ PRACTICE CHANGE toward a more democratic, transparent and institutionalized political system in the country.11 1. Courses given12 2.3 In this area, as well as in what concerns other concurrent events or products associated, such as inter-party dialogues and studies on modernization requirements, the Program’s performance is mixed if compared to the reference number of courses and other concurrent events/products that were to be delivered, according to the Consortium’s original Proposal for 2008 - 2012. In some areas the Program exceeded such numbers and in others fell short, as it is explained in detail below. It 11 For a more detailed specification of the Program’s expected development results chain, see the present study’s Preface on Evaluability, p. ix. 12 The researchers did not find any unambiguous, “official”, ex ante documentation on program targets for this level of intermediate outputs. Therefore, when targets are mentioned herein, in connection with these intermediate outputs, they should be taken only as reference values, intended to give intermediate outputs a general context, as provided by the Consortium document: PROGRAMA PERMANENTE DE FORMACIÓN JÓVENES LÍDERES DE PARTIDOS POLÍTICOS Y ORGANIZACIONES DE LA SOCIEDAD CIVIL DE REPÚBLICA DOMINICANA. PROPUESTA PARA LA SEGUNDA FASE 2008-2012 (Proposal for 2008-2012). Table 1: FYL Program 2 nd phase’s first line of deliverables 5 must be mentioned that two factors not entirely under the full control of executors affected the organization and execution of program activities. First, preparatory actions and interfacing with the USAID prior to execution of the Program took longer than expected, delaying the actual start and effectively reducing the execution time estimated in the original Proposal for 2008 – 2012. Secondly, the execution of the Program partially coincided with two major national election processes (legislative and presidential); which in the political context of the Dominican Republic represents powerful diverters of energy and attention of political leaders and militants; and, in this case, tended to hinder arrangements and attendance to events. a. Regular leadership & political management courses 2.4 During the three year execution of the Program’s 2nd phase, 18 of these courses were organized and dictated -3 in each of the 6 regions of the Program- with a total of 839 young leaders actually participating13. This represents not only a fraction of the courses originally intended in the Proposal for 2008 – 2012, (75% and 74% of targets in courses and participants, respectively) but, most importantly, it also represents a noticeable decrease in the number of courses given, in comparison to the 1st phase, when this “regular course” was by far the most important output of the intervention.14 This, together with the presence and added importance of other components in the Program, that were not present or were not so prominent in the 1st phase, make for a somewhat diminished quantitative weight of this particular component in the general structure of the Program during the 2nd phase. In contrast, however, the regular course underwent a major internal overhauling intended to increase its qualitative focus on, and contribution to, the learning outcomes and transformation practices expected from the Program in this new phase. 2.5 In order to better align its contents with the intended result indicators, the regular course was subjected to a curricular reform whereby new relevant subject matters were included along the three main axes of: (i) organizations’ internal democracy; (ii) transparency in management; and (iii) institutional strengthening. This resulted in an increase in academic hours from the original 72 to 88 in the new curriculum15. Also, the inclusion of a special tutorship program, designed to aid students in preparing subsequent initiatives in their organizations, was expected to increase the focus of the course’s inputs on inducing the actual behavioral changes and practices expected by the Program. 2.6 Finally, learning resources not present in the course during the 1st phase were added, such as a course web-page and a virtual classroom, to help strengthen both the teaching and tutorship efforts, and to afford the participants digital access to course materials and bibliography; therefore expanding and diversifying their learning experience.16 So, despite the fewer courses, these innovations towards a more flexible teacher-student relation, and the other changes indicated above should have made the course actually more effective in inducing the expected program results. 13 See: Memorias. Programa: Formación de Jóvenes Líderes Políticos en la Republica Dominicana. Segunda Fase. Diciembre 2008 – Julio 2012. (Hereinafter: Memoirs) pp. 12, 21, & 37. 14 Ibid. p. 12 15 Ibid. p. 25 & 27 16 Ibid. p. 29 6 b. P.A.T. courses to train trainers in the regular course 2.7 The P.A.T. (Spanish acronym for Technical Assistance Program) was a program component already present in the 1st phase, but that experienced a substantial conceptual transformation and gain much significance and importance in the 2nd phase of the Program.17 However, having turned essentially into a multiplier of the regular course, albeit in the form of workshops of less complexity at the grassroots level of each organization, the P.A.T. component instead of diminishing the importance in the Program of the regular course, should have effectively enhanced its effectiveness to shape and induce the expected program results. 2.8 One P.A.T. course was given in each of the three years of the Program 2nd phase, with a total effective participation of 118 graduates from the regular courses, who were trained as facilitators of workshops down the organizational line. Having been selected with the concurrence of the Political Education Schools of the respective organizations involved in the Program, these graduates participated in a series of training activities designed to equalize the level of knowledge for all chosen graduates, both in the contents of the future workshops and in adequate facilitation techniques. The training also included the preparation of the workshops themselves, the evaluation of lessons learned, and the identification of possible improvements. The resulting multiplication process amply exceeded the targets envisioned in the original Proposal for 2008 – 2012, of 20 workshops with 800 participants. By the end of the Program in 2012, graduates reported having organized and facilitated 193 workshops with a total participation of 4,637 individuals at the grassroots level. c. Specialized courses 2.9 Finally, specialized training courses were also given to participants both inside and outside the main target population of the Program, focusing on matters generally associated with the modernization of the political systems, such as: (i) Gender in the legislative process; (ii) Public information & political journalism; and (iii) Negotiation & conflict resolution. One course was given on each of the first two subjects, attended by 26 and 50 participants, respectively; with the second in the form of an actual Diplomat particularly directed to journalist actively working for important media outlets in the country. On the third subject, a total of 6 courses -out of 7 planned- were given during the three years of the Program; yet only 4 of those were actually directed to the originally intended population, that is: the delegates representing political parties in the country’s electoral body. As compensation, the Negotiation & conflict resolution course was given twice to members of the young leaders’ network.18 The number of specialized courses given was 8, which is below the number for these courses set out in the original Proposal for 2008 – 2012, whereby the Consortium had planned 11 of these courses for the whole duration of the Program. 17 The P.A.T. was originally included in phase one of the Program with the idea of helping strengthen and further institutionalize the political education schools in each participating political party. Having achieved meager results in the prior phase, for the present phase the component was modified into a training course specifically designed to train the best graduates from the regular courses, for the eventual facilitation of related workshops to other members inside their respective organizations. 18 See Memoirs, pp. 47, 49, 52, 53 & 55. 7 2. Inter-party dialogue events 2.10 After a national encounter of participating young leaders at the start of the 2nd phase in 2008, the Program organized 6 regional encounters -one in each of the Program’s regions- with an attendance of 461 participants. These encounters, in turn, lead to the realization of 6 regional dialogues between political parties, attended by 430 young leaders representing the various organizations. All this activity finally led to a National Young Leaders Congress attended by 395 individual, as well as the consolidation of the Young Leaders Network in which 712 individuals participated. 19 3. Studies on modernization requirements 2.11 During the execution of the Program 2nd phase, the study: Diagnóstico del Sistema de Partidos Políticos was updated up to 2010 in its chapter on the Dominican Republic. The updating was based on the organization of 7 focus group meetings, attended by 44 representatives of the political parties’ leadership in the country. Results of these focus group meetings were subsequently divulged and further discussed in other 6 meetings convened for that purpose, with participation of a wider audience of 461 members of the political parties involved. B. Program terminal outputs 2.12 Beyond the participants in inter-party and study activities, the Program expected final outputs mainly consisted of the graduates from the training, tutoring and technical assistance effort; specifically from: (i) regular courses; (ii) PAT courses; and (iii) specialized courses. They are considered the main vectors for the eventual behavioral and institutional changes intended in the political system. 1. Graduates from the regular courses 2.13 Based on Program basic selection criteria, during the three years of execution 1,433 young leaders were chosen -out of applicants from a potential list of 21 political parties and 105 civil society organizations- as candidates eligible to take the Program regular course. Of these, 839 young leaders were actually selected randomly to participate in the courses, for a participation rate of 58.5% (See figure 1.) In accordance with the Program’s quasi-experimental evaluation design, this group constitutes the treatment population. Conversely, the 594 eligible Young Leaders that were not chosen through the random selection procedure constitute the control population, for purposes of the Program evaluation. As also depicted in figure 1, a preponderance of participants in the course eventually did graduate from it; with a desertion rate of only about 6%. 19 See Memoirs, pp. 12, 73, 74, 77, 78 & 79 Figure 1: Regular Course, Participation, Graduation & Desertion. Source: FYL 2nd phase Memoirs, p. 37 8 2.14 Concerning this line of final outputs the Program’s performance was just shy of its general target of graduating 270 leaders a year; a number of graduates which was actually achieved only in the last year of the execution (See figure 2). However, this concerned only the target of total graduates per year; for, as can be also seen in figure 2, the annual targets set out for female graduates (100) were consistently exceeded throughout the three years of execution. 2.15 This latter output is remarkable in itself, for the mentioned proportion is way above the actual share of women in real positions of political leadership in the country. In this context, and as we mentioned in the evaluation report for the 1st phase, 33% is generally considered by political parties in the country as a target rate for women in representative and leadership positions; yet, in actual fact, this rate is not complied with in many cases. Therefore, the present Program’s performance in both targeting and incorporating young female leaders in their regular training activities has been very beneficial to women. 2.16 Of all young leaders who graduated from the regular course, the proportion of graduates from each of the 6 regions of the Program does not vary significantly, making the distribution of graduates by regions remarkably uniform as can be seen in figure 3. On the contrary, and due to the fact that, by design, the vast majority of young leaders chosen to participate in the Program has come from political parties, a significant majority of graduates are young political leaders, as opposed to young leaders coming from Organizations of the Civil Society (OSC). 2.17 In effect, despite the large list of OSC potentially participating in the Program, the distribution of graduates between Political parties and OSC as depicted in figure 4 reveals a commanding 87% of graduates belonging to political parties; result which, as indicated, is in line with, and was to be expected from, the Program own design and purpose. Also see in figure 4 how, of this last percentage, a significant portion of graduates (77%) belongs to the two largest parties in the country, PLD (41%) and PRD (36%), with the remainder 23% belonging to 19 other minority parties that also participated in the Program’s 2nd phase. This distribution arguably represents a generally good cross section of the comparative electoral “size” of the different organizations at the moment of the Program, and Figure 2: Regular Course graduates, actual & target Source: FYL 2nd phase Memoirs, p. 37 Figure 3: Regular Course graduates, distribution by regions Source: FYL 2nd phase Memoirs, p. 38 Figure 4: Regular Course graduates, by organizations Source: FYL 2nd phase Memoirs, p. 38 9 therefore constitutes a mostly acceptable approximation to the number of leaders, militant and sympathizers as a whole within each organization. 2. Graduates from the other courses a. The PAT Courses 2.18 From the executors’ data it is understood that all 118 reported participants in the PAT courses graduated, and that during the three years of the Program 64 of those young leaders eventually went on to organize and dictate 193 related workshops at their respective parties’ grassroots level; reportedly training in the relevant subject matters 4,673 militants in 6 of the 21 parties present in the original list of potential participants. This, as depicted in figure 5, constitutes a significant multiplier effect of the Program; especially in view of the fact that the organizations involved in this multiplication process included the 4 largest political parties in the country at the moment of training.20 b. The specialized courses 2.19 Finally, on what concerns the specialized courses given to the target population of the Program, as well as to other relevant individuals, a total of 229 participants took those courses during the 3 years of the Program, of which 208 graduated (33% women) for an overall desertion rate of 10%.21 However, this average rate really masks what in fact happened during the course of Gender in the Legislative Process, where the desertion rate reached 41%. (See figure 6) Also, graduates from the Negotiation & Conflict Resolution course given to members of the Young Leader Network basically made up for the under-performance concerning the target population which this line of training was really intended to. The case of this and the Gender in the Legislative Process course are instances in which external factors associated with the nature of the country’s political process itself played a role in shaping Program results. The energies and time consumed during the long concurring electoral processes, by the authorities and officers which were the targets of these courses, hindered attendance and graduation rates. Desertion of congressmen and women has been particularly sensitive, as it may have diminished the potential effect on including the gender perspective in legislation. 20 See Memoirs, pp. 62 & 63 21 See Memoirs, p. 12 Figure 5: PTA courses, graduates & multiplier effect Source: FYL 2nd phase Memoirs, pp. 62 & 63 Figure 6: Specialized courses, graduates by subject Source: FYL 2nd phase Memoirs, pp. 47, 49, 52, 53 & 55 10 C. Program intermediate outcomes 2.20 The first line of outcomes expected from the Program is the knowledge actually gained by the graduates, from the training, tutoring and technical assistance provided. Accordingly, the percentage of participants who increase the knowledge of leadership and political management is the particular indicator associated to the Program intermediate outcomes that the USAID has specifically required to be measured in the present study. We first present below the relevant measurements published by the Program executors, concerning: (i) graduates academic performance; and (ii) index of knowledge gain, as presented in the Consortium’s documentation.22 1. Academic performance scores 2.21 The academic evaluation system established by the Program included an entry test to determine, on a scale from 0 to 100 points, the level of basic knowledge in the relevant subjects of all students enrolling in the regular course –the Leadership and Political Management course. Also students were subject to a composite evaluation during the course, yielding a total exit score. Exit scores had a passing grade of 70 points on the same scale of 100 points, and the two markers were used by the Program as a measure of learning by students. 2.22 Based on these first metrics, the Executors present average scores of graduates both at entry and at exit, suggesting in general an increase in knowledge for each of the cohorts (2009, 2009 and 2011 courses). An improvement of entry scores is observed from an average 51.2 in the first cohort (2009), to little more than 65 in the last cohort (2011); as well as a slight decrease in the average exit scores from 78 in the first cohort, to about 77 in the last.23 Despite the fact that the middle cohort (2010) increased its average exit score to 83.2, the improved entry scores throughout the program execution and the relative dip in the final cohort exit score make up for a reduction and flattening of the distance between entry and exit scores during the three years of program execution, as depicted in figure 7. 2.23 Average scores produce a first approximation to estimating knowledge gain by graduates; but the academic figures published by the Program does not provide a measure of the variance of these scores, so they cannot help ascertain the dispersion of the figures and its effect on averages. 22 Knowledge gain is traditionally measured through indirect means. Specifically, grades achieved by students in tests and other forms of academic scoring are normally taken as proxy measures of actual learning. Since there was no practical or meaningful way of re-making those tests again to graduates in the context of the present research, the study bases its conclusion on the scores obtained by students in tests and other academic evaluations administered by the Program during its execution, and as presented in the executors’ documentation and data base. As it turns out, then, this indicator of knowledge gain is the exception among the 6 indicators required to be directly measured in the field by the present study. 23 See Memoirs, p. 33 Figure 7: Regular courses, average students’ scores Source: FYL 2nd phase Memoirs, p. 33 51.2 66.2 65.18 78 83.2 76.92 11 2. Learning indexes 2.24 As a composite grade, the exit score discussed in paragraph 2.21 of this section measured the following six items: attendance, intermediate tests, tutorships, final test and participation in class. Of these constituent grades, executors singled out the results of the final test and compared them to the results of the entry test to build what they call Learning Index I. Specifically this index is calculated as the arithmetical difference between the grades from the entry and final tests. A second index (Learning Index II) is related to the average scores discussed in the previous section, and is calculated as the arithmetical difference between the grades from the entry test, and those of the composite exit score. 2.25 Average figures for these indexes as depicted in figure 8 in general suggest a greater knowledge gain of graduates when the learning is estimated based on the composite exit score (general average difference of 26.0 points) than when it is estimated based just on the final test score (general average difference of 19.3 points). It is worth mentioning, as well, that these indexes consistently show greater knowledge gain among women that among men graduates of the Program regular course. In effect, as can be also seen in figure 8, female graduates score higher point differences than male graduates in both indexes. Here again, the average index estimates discussed above represent a first rough approximation to the knowledge gain achieved by graduates; but the absence of data on the variance of these indexes between students, in the available documentation published by executors, precludes an analysis of how dispersed the underlying scores were in the population. Also the information available does not provide evidence that similar efforts were made by the Program to measure the knowledge gain in the other courses provided. 3. Dispersion of graduates knowledge gain by cohort, sex and region 2.26 In order to complement the analysis of knowledge gain based on the average figures presented above, the present researchers subjected the Program’s data base on academic scores and learning indexes to several forms of variance analysis, in order to establish how correlated and statistically significant those average measurements of Program outcomes were.24 Specifically, Student t tests for paired observations were applied, comparing individual difference between entry scores, on one hand, and final test scores and exit scores, on the other, for each graduate in the three cohorts of the 24 Like any other “central trend” measure, average score values can be influenced by how the individual scores are actually distributed across the relevant value scale, even to the point of sometimes obscuring their real underlying meaning. Think, for instance, of the case of a course with a few gifted individuals getting extremely high scores, thus “pulling upward” the average grade for the whole class, and maybe so much so that the resulting average does not really reflect the real scores of the vast majority of individuals in the cohort. Therefore, central trend measures should be complemented with estimates of the “dispersion” (variance) of those individual scores and of how this dispersion influences the central trend figures. Figure 8: Regular courses, knowledge gain estimates Source: FYL 2nd phase Memoirs, p. 34 12 study. How the knowledge gain figures varied depending on particular independent variables of the population was also separately measured, through the application of a general linear model (GLM) for multifactor variance analysis. In what follows, measurements are presented on the variables sex and region, which are the only variables for which statistically significant results have been found. a. Knowledge gain based on exit scores 2.27 Results of the Student t two-tailed test applied on the individual differences between the entry and exit scores (equivalent to Program’s Learning Index II) for each of the cohorts are depicted in figure 9. Columns in the figure show the average grades obtained by graduates, and the table below the figure presents the variance values resulting from the Student t tests. Notice that the r values in the table are consistently low for all cohorts, indicating that individual scores are highly dispersed among the population. At the same time p values at the extreme right of the table are also noticeably low, indicating that results are very statistically significant.25 These results suggest that the measurements are systematically related to each other and confirm that the knowledge gain thus measured is certainly attributable to the workings of the Program. 2.28 As depicted in figure 10, the variance of the knowledge gain index II, analyzed by sex, shows that female young leaders consistently start out at lower grades as compared to male young leaders. However, they always gain enough knowledge as to mostly catch up with male leaders at the end of courses. Figure 10 shows this clear progress of female leaders surmounting average differences of between 4 and 5 points in the 2010 and 2011 cohorts. Note also that for these two cohorts, where 25 Note that p represents the probability that the observed results are due to mere chance or unknown factors. Figure 9: Correlation & significance of knowledge gain. Difference be- tween entry test scores and exit scores. Student t tests results Entry Test Score Exit Score Entry Test Score Exit Score Entry Test Score Exit Score Figure 10: Knowledge gain by sex. Multifactor analysis based on differences between entry test scores and exit scores. 13 the equalizing effect is clearer, the GLM yields statistics for high significance (.001 and .002) and large observed power (.924 and .871). Finally note that the observed relative effect size is large or very large for female leaders as measured in percentage points (see last row of the table below figure 10). All of the above means that the effect of the Program regular courses in the knowledge gained by young female leaders who participated was a clear and substantial development outcome of the Program in all cohorts. 2.29 Similar equalizing trends among the regions of the Program are observed in all cohorts; especially in cohorts 2009 and 2011. The analysis based on the GLM yields that not only the participants from all regions clearly gained knowledge as measured herein, but also that starting from very disparate scores -a difference between regions of about 10 to 30 points in the entry test scores- to notably uniform scores at the end of the regular course; with differences generally under 3 points between regions in the exit scores. From one cohort to the other, no particular region consistently starts below or above the rest, in terms of entry scores; as they all seem to interchange scoring positions randomly at the entry point. See in figure 11 how, for instance, the Metropolitan region starts way above the rest in the courses for year 2009, while in the courses for 2011 the Metropolitan region starts as the one with lowest entry scores. Regardless, all regions end up with very similar higher scores in these two cohorts, which strongly suggests that the Program has in fact contributed to equalize the knowledge of the skills and theoretical contents it imparted to young leaders throughout the whole country. Actually the converging trend is observed in all regions in the great majority of cases, except two cases that do not converge: the North region in cohort 2010, and the East region in cohort 2011. Also note, in the table below figure 11, that the tests results from the GLM show strong statistical significance for this equalizing trend in all cohorts (Sig. = .000; .014; and .000. for the three cohorts respectively) and very high observed power (1.000; .849 and .999 for the three cohorts respectively): a testament that the “leveling off” of the field in all regions, in terms of Figure 11: Knowledge gain by regions. Multifactor analysis based on differences between entry test scores and exit scores. Entry Test Score Exit Score Entry Test Score Exit Score Entry Test Score Exit Score 14 leadership and political management knowledge, was also a clear and substantial development outcome of the Program.26 b. Knowledge gain based on final tests scores 2.30 As a matter of interpretation, the present researchers feel that the measure of learning stipulated by the Program as Learning Index I (difference between the entry test score and the final test score) is a more apt gage of the true gain in knowledge by graduates, because it directly measures the difference from one test to another test -since the entry scores also come from a single test. Learning Index II, on the other hand, has the complicating characteristic of including in the exit grade other scoring components that are not directly comparable to what was measured in the entry score, as discussed in paragraph 2.24 of this section above. Therefore, in what follows, the Program data base on Learning Index I is also subjected to Student t tests for paired observations and a multi-factor variance analysis based on a General Linear Model (GLM), in order to establish criteria for correlation and statistical significance of the tests and other academic evaluation measurements done during the years of program execution. 2.31 As in the case of figure 9, columns in figure 12 show the average grades obtained by graduates, and the table below the figure presents the variance values resulting from the Student t statistical tests applied on the data by researchers. Notice that, judging by the low correlation values (r), the difference between the two scores involved in Knowledge Gain Index I are, again, fairly dispersed among the students taken the regular courses. The exception to this general result shows up in cohort 2009, for which the correlation is moderate (r = 0.493) Also the very low p values at the extreme right-hand side of the table (0.000; 0.038 and 0.0007 for the three cohorts respectively) suggest that the results are very statistically significant. Therefore, statistical tests do confirm that the knowledge gain as measured by Learning Index I can be strongly attributed to the participation of young leaders in the Program’s regular course. 2.32 The variance analysis of differences between entry test scores and final test scores done through the GLM also shows that, while always starting at lower scores, female young leaders taking the 26 Note that the value Sig. is the General Linear Model’s equivalent to the Student t test’s p value; i.e. an estimate of statistical significance, as measured by the probability of attributing results to the intervention when, in reality, they are due to mere chance or unknown factors other than the intervention. In turn, the value of Power is the probability of not making the opposite mistake, i.e.: not recognizing attribution of results, when in fact they can be attributed to the intervention. The reader will notice that the Power and Effect Size estimators are included in the present report only when and if statistical significance is found (p or Sig.  0.05) Figure 12: Correlation & significance of knowledge gain. Difference between entry test scores and final test scores. Student t tests results Figure 12: Correlation & significance of knowledge gain. Difference be- tween entry test and final test scores. Student t tests results 15 Program’s regular courses consistently gain enough knowledge as to equalize that of male young leaders by the end of the training. The progress of female young leaders is especially noticeable in cohorts 2010 and 2011 of the regular courses, where they are shown to have actually caught up with their male counterparts (see figure 13). Notice that with statistical significance values of 0.031; 0.002; and 0.001 respectively for cohorts 2009, 2010 and 2011, and effect size estimated to be large or very large for all the population, the results of this test portray a strong Program outcome in leveling of the playing field between male and female young leaders in terms of knowledge. This is especially true for cohorts 2010 and 2011, where the statistical significance and the large effect size, are coupled with a very high observed Power too, as depicted in the table under figure 13. Results are equally significant from a statistical point of view when the GLM is applied to investigate the correlation between knowledge gain measured by Learning Index I and the variable “region”. The statistical significance values range between 0 and 0.002, and the observed power ranges between 0.872 and 1, with noticeable differences between entry test scores and the final test scores.27 2.33 Finally, and in order to verify the comparability of the three different scores, (Entry test score, Final test score and Exit score) a Pearson bivariate correlation test was applied to the relevant data. Results show a positive and low to moderate correlation between the scores, as well as high statistical significance of the data. So, it can be said that this test, as well as the other variance analysis tests presented herein and discussed at length in the sections above, provide complementary verification of the dispersion and the statistical significance of the results found; thus generally confirming results from the analysis based on average values published by the Consortium 27 The important caveat concerning these results is that, curiously, while the knowledge gain is clearly verified in all regions for all the course cohorts studied, the equalization of knowledge between the regions that appears to have been induced by the Program in all cohorts concerning Learning Index II, does not show up again in any cohort when the same statistical analysis is applied to the data corresponding to Learning Index I. Since no basic course characteristics, such as curricular contents, academic hours, etc. suffered any changes during the Program execution, these results have no apparent explanation. Figure 13: Knowledge gain by sex. Multifactor analysis based on differences between entry test and final test scores. 16 in the Program Memoirs and discussed in paragraphs 2.21 through 2.25 of this section. The evolution of average scores is portrayed in figure 14. 2.34 A particular situation is worth mentioning concerning the average exit score for cohort 2011. When the Program’s data base was independently used in calculations and subjected by the present researches to the several statistical tests explained above, the average value that resulted for that particular score (82.96) widely contradicted the figure published in page 33 of the Program Memoirs (76.92). Also, while figures published in table 2, page 34, of the Memoirs show that average knowledge gain based on exit scores (learning index II) appear generally to be higher than that based on final test scores (learning index I) the variance analysis show that the opposite is true: i.e. the analysis confirms that the differences from test to test are actually greater than those between the entry tests and the final composite grades represented by the exit scores. Now, since the entry test and the final test are more directly comparable to one another -because they are free of the additional scoring elements represented by the composite exit score- in our view the test to test difference (learning index I) is a better gage of knowledge gain. Consequently the mentioned average difference, as calculated by the present researchers, actually argues better in favor of the Program. D. Program terminal outcomes 2.35 The final line of outcomes expected from the Program is the attitudinal and behavioral changes presumably induced by the knowledge gained by graduates, and evidenced by the observable actions and initiatives exhibited by the young leaders after graduation. As requested by the USAID, the elements to be documented as evidence of the program final outcomes include: (i) implementation of training courses for the organizations’ grassroots; (ii) submission of proposals for organizational reform; (iii) increased management responsibility bestowed on young leaders; (iv) increased participation of young leaders in election processes; and (v) increased young leaders’ involvement in mechanism of municipal participation or inter-party dialogue initiatives28. Additional actions included in the present study as evidence of relevant attitudinal and behavioral changes are: increased habits of transparency and accountability. 2.36 Final program outcomes are herein studied from two different angles. First, we study the observable evolution during the Program execution of the relevant indicators on those individuals in each cohort 28 It must be noted that changes iii and iv depend so heavily on factors outside the influence of the intervention (decisions by the organization’s current leadership) that they can just minimally qualify as true Program outcomes. That is, there are cogent reasons for these outcomes to be legitimately included in the Program’s development hypothesis; but, because the development hypothesis in the present case is based on such critical assumptions (i.e.: the concurrence of external factors) the actual achievement of those outcomes do not necessarily depend only on how well the Program has been executed. Figure 14: Average score values, bivariate correlation analysis 17 (2009, 2010, and 2011) who participated in the base-line and year-end surveys for their cohort, and were also chosen to participate in the final survey at the end of Program.29 Second, we study the total, ex post change in behavior patterns of a statistically significant sample of population, based on a new field survey done after the Program closing (ex post survey). Both analyses focus on the five indicators enumerated in paragraph 2.35 of this section above, and discuss significant differences in the associated dependent variables between the Treatment and Control groups, in what concerns the results of training activities. 2.37 Given the quasi-experimental framework set out for the program evaluation, the validity of conclusions to be drawn from these two analyses depends heavily on the fact that treatment and control groups were chosen randomly at the Program start, from a general population of equally eligible individuals, as has been reported by the Consortium. Also the control group -defined in this program as those generally eligible individuals who were not chosen for Program’s regular course￾has not been precluded from receiving any other separate political training, from any other source that might induce comparable behavioral changes. Therefore, for proper comparison, we should also control for these independent factors as they might influence the control group independently. So, before discussing results, a section is included below to gage these issues of comparability. 1. Comparability between Treatment & Control groups a. Basic independent variables as confounding factors 2.38 Pearson Chi-square and Student t tests, applied to both groups for the three cohorts of the Program’s regular courses, confirm that there is no statistically significant differences at the start between the individuals of the treatment and control groups in what concerns basic independent variables, such as: age, incomes, and seniority; the latter both in the sense of time belonging to their particular organizations, and in the sense of time in the position individuals were holding at the moment they were surveyed. The fact that no such differences have been found between the two groups on these variables represents an independent, ex post confirmation of the randomness of the original selection of individuals to conform the Treatment and Control groups, and argues in favor of the significance of results discussed below in the present report. This is particularly important in the case of the variables: (i) time in the organization, and (ii) time in the position; since seniority may presumably influence the chances for a young leader’s promotion and increase in management responsibility over the short run, within the organization he or she belongs to, as well as in his/her ability, power or influence to promote institutional changes in the organization; both of which constitute important behavioral patterns where changes are expected as a result of the present intervention. Therefore, since no statistically significant differences have been found between the two groups before the start of the Program’s intervention, any difference in these variables measured after the Program’s regular courses ended in each case may not, arguably, be attributable 29 Notice that this procedure allows the comparison of indicators on the same persons at each measurement point, thus tracking the evolution of the same group in each cohort, from the point that they entered the Program’s regular courses, through the end of the courses, all the way to the Program’s closing. 18 in general to a previously acquired condition of seniority, as being more typical in individuals belonging to one or the other group, within the respective organizations.30 2.39 The tests performed by the present researchers on both Treatment and Control groups generally bear the conclusion that they were, in fact, chosen at random. The exception to this result shows up in the variable: affiliation -to a party or a C.S.O. The present researchers understand that this constitutes a design feature of the Program, whereby even if the participation of organizations of the Civil Society in Dominican Republic was expected and encouraged by Program designers and executors, the principal population target of the political and social training effort was indeed those young leaders from the political parties, at the different organizational levels and in the different regions of the country where the Program was executed. Sure enough, Chi-square tests show in all measurement points that young leaders who are members of political parties consistently are in the majority within the Treatment Group; while members of C.S.O. consistently are in the majority within the Control Group, with p values of .044; 0.0003; and 0, for cohorts 2009, 2010 and 2011 respectively. 2.40 The very low values of p in this particular test confirm that this consistent difference is most probably not due to mere happenstance and that, certainly, a bias was present during the selection process toward choosing proportionally more political leaders than social leaders for participation in the training activities of the Program; and in particular in the regular courses (See figure 15) 30 Chi-square tests applied on other general available variables, such as occupation and sector of employment, yield no conclusive results for all cohorts, or confirm the absence of statically significant differences between treatment and control groups (as in the case of the 2011 cohort). On the other hand, t (2 tailed) tests on the distribution among the two groups of the variables: age, income, time in the organization and time in the position consistently give high p values, (with the exception of the base line for 2011) with the following ranges: .160 - .840; .173 - .873; .066 -.764 and .012 - .723 respectively. Figure 15: Affiliation of individuals to political parties or CSO in Treatment and Control groups. Chi-square test results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 19 b. Parallel training as a confounding factor 2.41 Survey data exists about to what extent both Control and Treatment groups received training on political/social issues during the relevant period. This data is of little value added in the case of the Treatment Group, for the training received by this group generally refers to that of the Program itself. Yet, the data is useful to find the presence of equivalent trainings that the Control group might have received. As shown in figure 16, surveys confirm a wide difference between the groups, with high statistical significance in all cohorts (Chi-square p values of 0.011; 0.013; & 0.006, respectively)31 2.42 The statistically significant low incidence of equivalent training received by the Control group, as compared to the Treatment group, points toward a negligible weight of such equivalent training as a confounding factor for comparability of results, and argues in favor of attributing observable behavioral differences to the effects of the Program’s training. 2. Outcome trends during execution a. Education of organizations’ grassroots 2.43 If compared to the 1st phase of the Program, the regular course in the 2nd phase should, by design, be expected to generate a greater multiplier educational effect -if measured by the training activities directed toward the organizations’ grassroots as a result of the Program- because specific Program inputs and outputs were deployed to promote such particular effect, through the P.A.T. component. In paragraph 2.18 of this section above, we already discussed this multiplier effect as observed from the point of view of Program outputs. Now, the field surveys generally confirm also behavioral changes in the treatment group well above those of the control group, in terms of the percentage of individuals increasing their activities of training directed to the organizations’ grassroots. In each of 31 It can be argued that, in general, the base-line – year-end – program-end series for the cohorts of 2009 and 2010 are a much better representation of a true behavioral evolution, than that for the 2011 cohort. This is not only because the latter includes only two points of measurements (base line and program end), but because more time for change in behavior was allowed to the 2009 & 2010 cohorts until the Program ended in 2012. Figure 16: Percentage of young leaders who received political training. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 20 the different cohorts, between 10% and 20% more individuals of the treatment group engaged in those activities when compared with the base line values, as depicted in figure 17. 2.44 The above results are notable because on this score the treatment groups consistently start below the control groups in each of the cohort, as can also be appreciated in figure 17. See that the trend difference between the treatment and control groups on this particular behavioral outcome is most noticeable in cohort 2011, with a jump of 20 percentage points of those young leaders engaging in education multiplication activities, while the corresponding increase in the control group was only 5 percentage points. This difference is particularly significant from the statistical point of view, as the p value coming out of the Chi-square test was 0.016, indicating a low probability that this diverging trend may have appeared just by chance. Also notice that the value of Power resulting from the test in the case of the 2011 cohort is extremely high (0.984), adding strength to the argument that the differences found are closely correlated with the Program intervention in the case of cohort 2011. 2.45 Now, as shown in figure 18, the most significant trend that can be associated with this particular outcome is the clear tendency towards equalization between sexes observed within the Treatment Group in cohorts 2009 and 2010, as measured by the percentage of young leaders in each sex Figure 18: Percentage of Young leaders of the treatment group giving training to the organizations’ grassroots, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 17: Percentage of young leaders who gave training to the organizations’ grassroots. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 21 engaging in training multiplication activities after graduating from the regular course. This equalizing trend is quite pronounced, with the percentage of female leaders engaged in such activities starting way below that of male leaders at the base line, and increasing significantly after the regular course of those years. This general trend is contradicted by the results for the 2011 cohort, for which the tendency is actually reversed, as differences on this score between the sexes widen in favor of male leaders. However, results for this cohort are not only somewhat less statistically significant than in the case of the other two cohorts, but also less representative of the true behavioral evolution of a particular cohort of graduates, for the reasons discussed in footnote 31, above. 2.46 As it was shown in figure 18, the percentage of female leaders in the treatment groups engaging in education multiplication activities consistently start way below that of male leaders at the base lines for 2009 and 2010, and then experience a sharp increase of about 30 percentage points. Also, the values of p are shown to be extremely low (0 and 2.00 x 10-8) as are high the values of Power yielded by the test in cohort 2009 and 2010 (0.815 and 0.96 respectively), which adds statistical significance to the results obtained for those two cohorts. So, despite the odd case of cohort 2011, the longer trend evolution represented in cohorts 2009 and 2010 appear to confirm, with a higher degree of confidence, that the Program has indeed induced equalization of behaviors between sexes in what concerns the education multiplier effect. 2.47 It is worth noticing that the same trend towards equality can be generally seen to occur between male and female leaders in the control groups for the 2010 and 2011 cohorts; although much less pronounced than in the case of the Treatment Group, as can be seen comparing numbers and charts of figure 19 with those of figure 18. This result, therefore, can be attributed to the general equalizing trend between sexes experienced by society at large and also reflected, however in much more limited way, in the normal practice of political parties and other organizations. However, the observed trend differences in cohort 2009 -one which, incidentally, shows high statistical significance, with a Chi-square p value of 0.004- is actually a diverging trend; i.e.: the percentage of young female leaders engaged in multiplication training tends generally to decrease with time, with respect to the base line value, and in comparison to that value in the young male leaders in the Figure 19: Young leaders of the control group giving training to the organizations’ grassroots, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 22 control group. Yet the trend is converging between male and female young leaders participating also in cohort 2010, with high levels of statistical significance as well (p value of 0.002) b. Submission of proposals for organizational reform 2.48 Beyond an increase in the active promotion of political and social education for other members of the organization at the grassroots level, another important behavioral outcome sought by the Program was an increase in reform initiatives taken by the Program’s regular course graduates, to the extent that those initiatives were aimed at democratizing and modernizing their organizations. The 2nd phase of the Program should also be expected by design to have induced enhanced results on this score, as compared to those of the 1st phase, because a special tutorship effort focused on this outcome was included for the 2nd phase. As seen in figure 20, the trend generally observed during the Program’s execution certainly points toward such increase in the young leaders’ pro-activity toward organizational reforms after taking the regular course, as measured by the percentage of graduates promoting initiatives of that sort. This percentage increased between about 10 and 25 percentage points from the base line values, depending on the cohort. Also, the behavioral changes observed in the treatment groups are significantly more pronounced than, and generally in the opposite direction of, those of the control groups. This certainly suggests an outcome associated with the young leader’s participation in the Program’s regular course. Nevertheless, the values of p from Chi-square tests consistently above 0.05 for all cohorts suggest, to the contrary, that such differences are not statistically significant for this particular outcome indicator; which argues against the said correlation (see values in the table under figure 20). 2.49 As it was the case with the indicator concerning the educational multiplier effect of the Program, the evolution of the indicator on proactivity of young leaders in terms of organizational reform proposals also shows a tendency toward equalization between sexes within the treatment groups, in all cohorts studied. In terms of the percentage of individuals effectively making reform proposals this converging trend clearly occurs between the base-line and the year-end measurements, for the young leaders participating in cohort 2009. Then it actually turned into a diverging trend between Figure 20: Young leaders proposing reform initiatives within their organizations. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 23 the year-end and the program-end measurements (see first chart in figure 21). Yet, the program-end measurements still show a shorter difference between female and male young leaders, than that measured at the base-line survey. This difference is statistically significant, judging from the value of p yielded by the Chi-square (0.011). Also as in cohort 2009, young female leaders in cohort 2010 and 2011 start out way below the level shown by young male leaders at the base line; and then, after the course, experience a visible increase in pro-activity (see figure 21) 2.50 Specifically, out of the total of young leaders engaging in promoting organizational reforms in their respective organizations, the percentage of young female leaders increase between 5 and 12 points, while that of young male leaders decrease between 4 a 22 points, depending on the cohort. Chi￾square p values of 0.003 for cohort 2010 respectively suggest that the observed behavioral trend differences among graduates in this cohort are even more statistically significant and, thus, correlated with the participation of the young leaders in the Program’s regular course. 2.51 A somewhat similar trend toward equalization between sexes in terms of proactivity toward institutional reforms can also be observed in the behavior of the control groups, (see figure 22); a trend that can be attributed again to the general gender equalization process being experienced in political organizations. However, in the case of cohort 2009 the trend is virtually flat (neither converging nor diverging). Also in the cases when convergence occurs it is much less pronounced than that the leaders who participated in the Program and shows no statistical significance, except in the case of the 2010 cohort (see middle chart in figure 22) Figure 21: Young leaders of the treatment group proposing reform initiatives, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 22: Young leaders of the control group proposing reform initiatives, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 24 c. Increased management responsibility for young leaders 2.52 As alerted in footnote #28 above, this particular indicator measures one of the two terminal outcomes included in the Program’s expected results chain whose achievement would be heavily interfered by a factor essentially out of the sphere of control and/or influence of the Program; i.e.: the decision of the current senior leadership of the organizations to actually promote young leaders up the organizational ranks. Arguably, Program designers have implicitly expected that the current leadership of the participating political parties and C.S.O.s would promote young leaders who have graduated from the Program to positions of higher responsibility within each organization, as a way of taking advantage of the “investment” made in the training of those young leaders and for the sake of the organization’s advancement and institutional strength. These expected promotions would, presumably, multiply the development effects of the Program, by placing progressively increasing numbers of graduates in decision-making positions wherein they could put in practice the values, concepts and practices taught by the Program to improve internal democracy, transparency and institutionalization of their respective organizations. While this development hypothesis is cogent and essentially acceptable, the fact remains that the necessary concurrent factor -the decision to promote the young leaders- ultimate lies with the organization’s current senior leadership or election processes, and not within the Program’s control or influence domain. That is why we argue in the present study that such outcome can just barely qualify as truly belonging to the development result chain that can be expected from the Program. 2.53 Judging by the responses of those surveyed, the incidence of young leaders’ promotion has actually fallen through time, in both treatment and control groups, during the Program execution years (see figure 23) suggesting that the external decision factor discussed above has not concurred as expected, thus hindering Program outcomes by this indicator.32 The fact that differences between treatment and control groups’ trends are not great or statistically significant, further suggests that results are note necessary correlated, either, with the Program, and may be due to other factors. Yet, 32 It is difficult to argue that the decision to systematically promote program graduates up the organizational ranks will not eventually take place, or predict when it would take place, if it does. Present results may just reveal a case in which the expected concurrent external factor just need more time to materialize. Figure 23: Young leaders being promoted to positions of higher responsibility. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 25 despite all this, an increase in the promotion of female young leaders is observed as a percentage of all individuals within the treatment group. This occurs in the 2009 and 2010 cohorts, along with a corresponding decrease in the percentage of male leaders promoted, as seen in figure 24. 2.54 The same general tendency toward a greater percentage of young female leaders being promoted, at the expense of the percentage of young male leaders, is observed in the control groups for each of the years; with the differences between male and female individuals being only statistically significant in the case of the year 2009 (see figure 25) d. Increased participation of young leaders in election processes 2.55 This is the other terminal outcome included in the Program’s expected results chain whose achievement may be heavily interfered by a particular external factor, necessarily concurrent but essentially originating outside the sphere of influence of the Program, namely: the senior leadership’s commitment to open internal election processes; and this is so for similar reasons as discussed in paragraph 2.52 of this section, in relation to the indicator: increased management responsibility for young leaders. The argument to include the indicator of young leaders’ participation Figure 24: Young leaders in treatment groups being promoted to higher positions, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 25: Young leaders in control groups being promoted to higher positions, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 26 in election processes in the Program expected results chain is also similar. Presumably, the democratizing process by which Program graduates are able to get elected to positions of authority might be viewed as a signal that the modernizing influence of such leaders might intensify progressively. Now, field survey data show a clear upward trend in the percentage of young leaders running in internal elections (see figure 26). Yet, the same trend and very similar percentages are shown in both treatment and control groups; so these changes can hardly be attributed to the Program. Therefore, results suggest the possible presence, and at least partial influence, of other external concurrent factors possibly associated with decisions of the current organizations’ senior leadership. 2.56 Once again, an equalizing trend between sexes within the treatment groups is revealed by the field data for 2009 and 2010 on this particular outcome. Figure 27 depicts how female young leaders, starting from much lower levels of electoral participation, after graduation tend to participate more as a percentage of total individuals engaged in such activities. The converging trend shows up clearly in the survey results between the base-line point and the year-end point for cohort 2009; and then the trend turns diverging between the year-end measurement and the program-end measurement (see the left-hand chart in figure 27) However, at the point of the program-end survey the percentage of female young leaders participating in election is still noticeably higher, and that of male young leaders lower, than the percentage at the base-line point. Therefore the general trend in cohort 2009 between the two populations was still generally converging, despite the mentioned Figure 26: Young leaders running in internal electoral processes. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 27: Young leaders in treatment groups participating in electoral processes, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 27 divergence. Still, the trend differences between male and female leaders on this score are not statistically significant; so they cannot be necessarily viewed as correlated with the Program either. 2.57 A similar trend is observed in the control groups, with gender differences close to those found in the treatment group. With female young leaders starting once again from much lower levels of electoral participation, the converging trend with male young leaders shows up very clearly in the survey results between the base-line measurement and the year-end measurement for cohort 2009. Then the trend flattens out between the year-end measurement and the program-end measurement. Still, the female’ gain is clear at the end (see the left-hand chart in figure 28) and with results statistically significant, judging from the p value of 0.036 from the Chi-square test. A similar evolution, but in reverse, occurs in cohort 2010 with still higher values at the end for women than for man, and a clear converging trend showing up between the measurements of those in cohort 2011 (see middle and right-hand side charts of figure 28) e. Increased young leaders’ participation in inter-organizations dialogue 2.58 This is an outcome of the Program that is quantitatively considerable, clearly verifiable and statistically correlated with the intervention, as can be seen in figure 29. Individuals in the Program’s treatment group appear dramatically more engaged in activities of dialogue beyond their organization’s boundaries as compared to those of the control group whose percentage of Figure 28: Young leaders in control groups participating in electoral processes, by sex. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 29: Young leaders participating in inter-organizations dialogue. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 28 individuals engaging in such activities actually seemed to have consistently diminished during the period of the Program (see red line in all charts of figure 29). This outcome clearly observed in all cohorts is also statistically significant for cohorts 2009 and 2010, with Chi-square p values of 0.04 and 0.012 respectively. Also observe how in the case of cohort 2010 the Power reaches a high value of 0.866. This diverging trend is of particular importance because the Program was executed during a period of heightened political partisanship –two almost back-to-back national election process occurred while the Program was executed– a period where diminished inter-party dialogue would generally be expected. This is precisely the trend observed in the control group, and the one clearly contradicted by the treatment group after graduation from the Program’s regular courses. 2.59 A gender equalizing trend is observable as well in the treatment group concerning the pro-activity of individuals toward inter-organizations dialogue, similar to the one discussed above in connection with other indicators, although restricted to the case of cohort 2009 only. In that cohort, female young leaders also start from very low values at the base-line point in terms of the percentage of individuals engaged in those activities, and then increase their involvement to the point of actually overtaking the corresponding percentage of young male leaders. This trend of 2009 is also statistically significant with a p value of 0.003 resulting from the Chi-square test (see figure 30). 2.60 The same is also marginally observed in the control groups, as depicted particularly in the left-hand side and middle charts of, as well as the numbers under, figure 31; although with much wider Figure 30: Young leaders in treatment groups participating in inter-organizations dialogue. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 31: Young leaders in control groups participating in inter-organizations dialogue. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. 29 differences and a lot poorer statistical significance,. Of particular notice is the very wide starting difference between female and male young leaders in this indicator. f. Increased young leaders’ transparency & accountability 2.61 Although not explicitly required in the evaluation s.o.w., the present study includes the analysis of indicators measuring young leaders’ behavior patterns concerning transparency and accountability, both within and without their respective organizations, as relevant measures of Program terminal outcomes. The surveys done during the Program execution provide data on habits of disclosure toward three different publics; namely: (i) superiors within the organization; (ii) subordinates within the organization; and (iii) general public outside the organization. Also, based on responses from questions concerning these behavior patterns, the Program constructed an Accountability Index, which, together with the other specific disclosure indicators already mentioned, can be comparatively studied between treatment and control groups. All these indicators were analyzed, although it should be noticed that disclosure activities of type ii and iii are better indicators of change toward increased transparency; since, contrary to that of type i, they do not constitute behavior generally considered mandatory to members of an organization. 2.62 Survey data reveals that the behavior in treatment groups for cohort 2009 and 2010 sharply trend toward increased accountability towards superiors, and that this trend clearly contrasts with that of the control group which exhibits behavior in the exact opposite -downward- direction (see figure 32) This is a particularly noteworthy result in and of itself, in the case of the control group, because a behavior of disclosure is to be expected out of mere compliance with internal rules within any hierarchical organization such as, for instance, a political party. Also, this trend cannot be explained away by mere chance or by other extraneous factors acting thereupon, judging by the low p values coming from the Chi-square tests applied on the surveys data for those cohorts (0.0059 and 0.008 respectively). Therefore, the trend appears as strongly attributable to the Program’s regular course. The same general trend is also exhibited by the 2011 cohort, although in this case with a weaker statistical significance. 2.63 A strongly diverging behavioral change is observed, as well, between treatment and control groups in what concerns the young leaders’ actions of disclosure toward subordinates within their Figure 32: Percentage of young leaders doing disclosure to institutional superiors. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Cohort 2009 Cohort 2010 Cohort 2011 30 organizations, in all cohorts except for that of 2011. However in the case of cohort 2011, as was also the case with the previous indicator, the trend is locally converging, however slightly, between the base-line point and the year-end point, only because of the fact that the control group started out with higher values for the indicator at the base line (see how the right-hand chart of figure 32 is virtually identical, in terms of trend, to that of figure 33) Despite this, the behavioral trend concerning this indicator is still upward for the treatment group and downward for the control group. Also notice that these results are really statistically significant only in the case of cohort 2009, with a p value coming out of the Chi-square test of 0.03. 2.64 Finally, in what concerns young leaders’ actions of disclosure toward the public outside the organization -arguably the strongest measure of real change toward transparency and accountability that can be expected from the Program- the same trend is observable in the data coming from the surveys done during the execution of the Program. Figure 34 shows specifically to what extent young leaders have engaged in publicizing -in this case through bulletins- their organizations’ performance. Notice, also, how these trends are statistically significant for cohorts 2010 and 2011. In effect the p values of 0.0019 and 0.031 coming out of the Chi-square test applied to the available data for those cohorts give a strong indication that this outcome is very much attributable to the workings of the Program, particularly through its regular courses where the values of transparency and accountability ranked highly in the curricular contents. Figure 33: Percentage of young leaders doing disclosure to their subordinates. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Figure 34: Percentage of young leaders disclosing performance through bulletins to the public. Chi square tests results. Source: Tests performed on statistics from the Program executor’s surveys data bases 2009, 2010 & 2011. Cohort 2009 Cohort 2010 Cohort 2011 Cohort 2009 Cohort 2010 Cohort 2011 31 3. Outcomes observed after execution 2.65 The researchers also measured the final outcomes of the Program after it was closed in 2012, for a complementary comparison with the trends found during the Program execution. For that purpose, a random sample, structured by sex, region and type of organization, of all individuals participating in the Program’s Treatment and Control groups was chosen, and a new field survey (ex post survey) was done to complement the same measurements made during execution. This ex post survey, whose results are presented in what follows, generally confirms that significant percentages of the young leaders that graduated from the Program regular course have engaged in the expected behaviors and attitudes of promoting modernization and democratic political practices in their organizations. This pro-activity has also been independently corroborated by senior leaders responsible for political education and youth issues in the organizations, who were interviewed for this study. However, results in the treatment group are not generally comparable to those of the control group, because a very significant portion of individuals from the control group chosen in the random sampling manifestly refused to participate in the survey, greatly diminishing the number needed to make a statistically strong argument about results in the control group.33 a. Role of external factors 2.66 Also, of special significance in the present evaluation of outcomes after termination of the Program is the role that factors outside the control and/or influence of the intervention may have played in the actual achievements. As we have already suggested, there is a visible and consistent difference between those outcome indicators whose values can be mostly attributable to the workings of the Program, and those who cannot. In footnote 28 above, we already advanced the point that there are outcome indicators more sensitive than others to the influence of external factors. A clear example of this distinction is the difference between the question: did young leaders promote reforms? and the question: were young leaders promoted to higher positions? Affirmative answers to both questions promote the development impacts sought by the Program; but the answer to the second question does not predominantly depend on the inputs and outputs of the Program, but on decisions adopted in other centers of cost and responsibility; in this case: the current senior leadership of the relevant organizations. 2.67 For purposes of the present study we shall call Type A outcomes those corresponding to the first kind (the ones that lie predominantly within the influence domain of the Program), and Type B outcomes 33 Problems with control groups are not infrequent when dealing with human individuals in quasi-experimental evaluation designs, since such populations require special handling, particularly when groups are porous and heavily charged with expectations. The specific problems confronted by the present study generally relate to negative feelings and attitudes toward the Program’s selection process developed during execution, and are discussed in detail in the recommendation section of the present report. Also specific recommendations for better handling control groups are included as lessons for future interventions. It is appropriate to advance here the researchers’ opinion that some “control individuals” that in the end did answer the ex post survey, replacing those first chosen randomly who refused, were more “interested” individuals, whose natural inclination to pro-activity may set them apart from other more “average” control group individuals; thus, perhaps yielding higher values of outcome indicators that could otherwise be generally expected. 32 those corresponding to the second kind (the ones not predominantly lying within the influence domain of the Program). Table 2 classifies the main indicators measured by the ex post survey depending on whether they are type A outcome indicators or type B outcome indicators.34 For the reasons just discussed, type A indicators can be considered a better gage of outcomes directly attributable to the Program; while type B indicators better reflect the concurrence of factors stemming from the control domain of participant organizations. 2.68 If, as argued in footnote 33, control-group participants in the final survey were more pro-active than the more “average” randomly chosen ones, they could be expected to show relatively high levels of initiative throughout; maybe even yielding comparisons with the behavior of the treatment group unfavorable to the latter. The high values of p out of the Chi-square tests done on survey results (low statistical significance) indeed show the consequences of having participants in the survey not necessarily chosen at random; i.e.: the number of control-group participants who voluntarily -i.e. proactively- replaced those individuals who were originally chosen at random but refused to answer the survey, for the reasons already mentioned. Despite all this, survey results still show consistently higher values in the treatment group for type A indicators, as compared to those of control groups. 2.69 Indeed, in spite of the fact that results may have been skewed against the treatment group, this group still presents higher percentages of individuals engaged in behaviors such as: promotion of education for militants; promotion of organizational reforms; public disclosure of matters concerning internal operation of organizations; declarations of wealth under oath; and participation in leaders networks; with differences of 5.3; 7.3; 10.2; 13.4 and 17.2 percentage points above those percentages in the control groups (see these results in the Statistical Appendix). 2.70 On the other hand, the ex post survey shows treatment groups consistently performing below control groups in those results generally associated with type B indicators; such as promotion of young leaders; approval of proposals both for reforms and for increased education of the organizations’ militant grassroots; appointment of young leaders as candidates in electoral process; and young leaders electoral success. Therefore, the present study is not able to cite conclusive evidence that current senior leadership of the participant organizations has so far significantly 34 Notice that in the present context we speak of sensitivity to external factors in relative terms and do not refer to just any external factor. For instance, the values of all outcome indicators included in table 2 depend also to good measure on the fact that graduates decide to act in a certain way after the course; decisions which, strictly speaking, also fall outside the control domain of the Program. But the classification of Type A and type B indicators does not mean to control for factors depending on decisions by the graduate themselves. OUTCOME INDICATOR TYPE Implementation of training courses for the organizations’ grassroots A Submission of proposals for organizational reform A Increased management responsibility bestowed on young leaders B Increased participation of young leaders in election processes B Increased young leaders’ involvement in municipal participation or inter-party dialogue initiatives A Increased young leaders’ transparency & accountability A Table 2: Types of main Program outcome indicators, based on their relative sensitivity to external factors 33 promoted Program graduates to positions of higher responsibility within the organizations, or significantly tended to approve graduates proposals, or their participation as candidates in electoral processes. Important exceptions to these general results show up in the cases of small parties. Data and interviews of senior leaders from small or emerging political parties done for the present study indeed confirm that these organizations tended to take better advantage of the educational effort represented by the Program; both in the sense that: (i) the top students of the regular courses frequently came from those small organizations; and (ii) the number of graduates subsequently promoted to positions of higher responsibility tended to be proportionally larger than those from the bigger organizations.35 2.71 Given the nature of final or “late” results, that usually require extended periods to mature in practice, the above mentioned situation maybe a case of terminal outcomes for which the three years of the Program’s 2nd phase is not enough for maturity, and that they may just need additional time to come to fruition. The low alacrity in the senior leadership’s decision processes that the Program expects to concur as a factor to promote its intended development outcomes is also a factor affecting the achievement of the expected impacts; i.e. the longer term structural changes sought out by the intervention. b. Main final outcomes. A quantitative perspective 2.72 Besides the already mentioned ratification that young leaders who graduated from the Program regular course engage more in the behaviors and attitudes expected by the intervention in type A outcome indicators, the ex post survey also strongly ratifies that, judging from the percentage results, female young leaders adopt the behaviors and attitudes promoted by the Program to a significantly greater extent than their male counterparts. The values of trend outcome indicators we discussed previously in the present report had already shown that women participants in courses appeared to have taken better advantage of the training. They performed better in terms of knowledge gain, and starting from lower values relative to men, tended to achieve on a par with them at the end. Measurements also showed that, percentage-wise, young female leaders also tended to outperform young male leaders in observed changes toward the behaviors and attitudes promoted by the Program. Now, the ex post survey has found that female young leaders maintain this high performance in both type A and Type B outcome indicators after the closing of the program. 2.73 The above results also mean that the knowledge and skills gained, as well as the behavioral and attitudinal changes adopted by young female leaders of the treatment group so far have been generally taken better advantage of by the organizations that sent them for education in the Program, through more frequent promotion of those leaders up the organizational ranks; more frequent success of those leaders in internal electoral processes where they have participated as 35 Of course, this latter result is also surely a function of the sheer number of young leaders from the small organizations actually taking the courses, as compared to those from the big political parties. Because a full 77% of all graduates came from the two largest political organizations (see par. 2.17) it would be expected that the young leaders promoted to positions of higher responsibility within these large organizations will always represent a much smaller percentage when compared to those promoted by a much smaller parties. 34 candidates; and more frequent approval of the proposals they have made to their organizations, both in the areas of political education and institutional modernization reforms. 2.74 Regardless of the comparison between treatment and control groups, the figures below demonstrate how female young leaders of the treatment group consistently show better values in type A outcome indicators, such as those relating to pro-activity towards reforms, participation in dialogues with organizations other than their own, and involvement with the young-leaders’ network. As depicted in figure 37, evidence shows that the percentage of female young leaders graduating from the Program that have promoted reforms in their organizations are about 20 points higher than that of male young leaders with the same initiative in cohort 2009. Notice that the p value of 0.059 from the Chi￾square test shows this result to be moderately significant. However, the 22 to 26 percentage points advantage of women over men graduates that participated in discussions with political or social organizations other than their own, show a strong statistical significance with a p value of 0.033 from the Chi-square test (see Figure 38). 2.75 A comparable advantage of women over men is also observed in the percentage of graduates actively participating in the young leaders’ network developed as a result of the Program. Figure 39 shows that difference of about 23 percentage points to be even more statistically significant, with the Chi-square test yielding a p value of 0.002. See in all figures of the series above how in the case of the control groups for every one of the three cohorts, female young leaders maintain consistent lower percentages concerning this outcome than their male counterparts. 2.76 The same kind of statistically significant differences between female and male Program graduates are observed concerning type B outcome indicators, such as those associated with the promotion of young leaders up the organizational hierarchies; the approval of proposals made by young leaders and the success of young leaders in election processes where they run as candidates. If this observed trend is to continue, especially within the political parties involved, not only an increasing gender equality is to be expected within the organizations in the future, but also the participation of female leaders in decisions toward institutional reform processes might also be expected to increase in the future as a result, at least in part, of the Program. Figure 39: Young leaders participating in networks. 2 results. Source: Ex post survey 2012 Figure 38: Young leaders doing external debates. 2 results. Source: Ex post survey 2012 Figure 37: Young leaders promoting reforms. 2 results. Source: Ex post survey 2012 35 2.77 Indeed, even if we ignore the comparison between results from the treatment and control groups, the percentage of female young leaders of the treatment group that have been promoted to higher levels of responsibility -a type B outcome indicator￾is, in itself, remarkable: more than 20 percentage points above that of the male young leaders (see figure 40). Notice that in this result the difference between male and female young leaders is statistically significant, judging from the value of p below 0.05 coming out the Chi-square test performed by researchers on the survey results. If sustained, this more abundant presence of young female leaders in posts higher than that of mere militants within the organizations is sure to eventually result in a greater weight exerted by women on institutional decisions and reform processes, especially within the political parties. 2.78 So, in what may be more important as a precursor of possible things to come, the ex post survey has also revealed a better performance of female young leaders in terms of getting their reform proposals approved within their respective institutions. This particular indicator shows the percentage of women having success in the approval process of their proposals to be more than 42 points above that of men. This important difference has also high statistical significance as depicted in figure 41. Notice, in effect, that the p value coming out the Chi-square test is only 0.002 2.79 In the next section of this report, a closer, qualitative analysis of the nature and object of these proposals is presented, and the corroborative evidence available from the interviews of leaders within the organizations is also discussed. From that analysis, as well as from other qualitative aspects of the data available, a more detailed profile will be drawn on the final outcomes that the Program might have actually induced in the reality of the political system in the country, and the possible future direction of changes up the expected Results Chain. However, the sheer amount of the difference between the sexes provided by the ex post survey data on this particular indicator constitutes a very significant result in and of itself. Figure 40: Young leaders promoted to posts of higher res- ponsibility. 2 results. Source: Ex post survey 2012 Figure 41: Young leaders getting reform proposals approved. 2 results. Source: Ex post survey 2012 36 2.80 Finally, and at least equally important, are the outcomes observed concerning the success that female young leaders in the treatment group have had in election process for which they were postulated as candidates. In this particular indicator the percentage of women graduates that won the elections in which they ran for a position of responsibility is a full 46 points above that of men graduates. This very large difference is also highly significant statistically, as can be seen in figure 42, with a p value from the Chi-square test of only 0.033 c. Main final outcomes. A qualitative perspective 2.81 The field survey of young leaders done after the closing of the Program has provided valuable specific information on the observed behaviors, to afford a qualitative complementary analysis of the Program outcomes discussed above. Meetings and interviews with program managers, academics and, specially, with senior political leaders, also have yielded corroborative information to provide a contrasting background on the data obtained and a more detailed context for the analysis. (i) Education of organizations’ grassroots 2.82 The multiplier effect on education of organization’s grassroots we have discussed in some detail in the present report have been made possible by the actions of Program graduates, who have designed and actually dictated conferences and workshops to organization members. These actions, which have been numerous and widspread as measured by the surveys during execution and after the closing of the Program, have also been confirmed independently by political leaders of the main parties involved, interviewed for the present study. The ex post survey collected information not only on actual conferences or workshop personally given by graduates, but also on the latter’s participation in organizing and generally promoting this multiplication effort, as well as any other proposals made by graduates on the area of education within their organizations. Among these proposals it is important to mention the initiative of program graduates to contribute beyond the sole direct multiplication of workshops, promoting the actual creation or re-vitalization of Political Schools within the parties. This effort is already more salient and visible in the case of some small, emerging political parties, where its possible impacts might be more significant as well for the future of these organizations. 2.83 The ex-post survey confirms that a significant majority of graduates pro-actively engaged in the promotion of education in this broader sense of varied educational proposals made within their respective organizations; including, but going beyond, the direct multiplication of conferences and workshops. According to those surveyed, proposals included the creation of political education Figure 42: Young leaders winning elections, internal or external, in which they ran. 2 results 37 schools in those parties which do not have one, and the revitalization of the schools in the parties which do have them. Senior leaders as well as course lecturers interviewed have coincided in that there is no permanent institutional offerings of political education in the country, neither by the political parties, nor by the educational system, to meet the observed demand for such education. So, these initiatives revolving around the parties’ political education schools, although not abundant compared to other types -8% of all proposal (see figure 43)- do show the preocupation of young leaders with a longer term solution to the issue, beyond the episodic organization of courses now and then. Initiatives included proposals to establish courses on specific issues of interest such as: party norms; party ideology; gender and youth issues within the party; conflict resolution; and electoral training among other themes. Figure 43 shows that more that 60% of graduates have engaged in the mentioned variety of educational initiatives, and that almost a third of these initiatives were related to the direct educational multiplication effort. Notice also that issues such as ideology and gender in the party are represented in relatively small percentages (6% and 3% respectively) as proposed educational issues, while the issue of training for the elections was comparatively more abundant among the proposals (12%). 2.84 Now, given this pro-activity on the part of the Program’s regular course graduates, an important qualitative issue is whether the initiatives taken by graduates had succeeded or not in practice; so that the associated development impacts can be expected up the Program’s results chain. In this sense, the graduates’ initiative in the educational area apparently did not necessarily meet with a corresponding proactivity on the part of the senior leadership in their respective political organizations. In fact, a full 62% of those individuals of the treatment group surveyed after the closing of the Program did not respond with any precision, or at all, to the question on whether their proposals had met with success; or said, flat out, that they had not succeeded (See figure 44). It is worth noting the very high percentage of those surveyed who would Figure 44: FYL Program Graduates’ opinion on success of educa- tion proposals. Reasons for failure. Source: Ex post survey, 2012 Figure 43: FYL Program graduates making education proposals. Types & issues of proposals. Source: Ex post survey, 2012 38 or could not clearly reffer to any success obtained by their educational proposals. Even more telling, as depicted in figure 44, 30% of those surveyed who said their educational proposal did not suceed mentioned specifically the lack of leadership support in their organizations. It is understood that aquiescence of the senior leadership about the initiatives taken by young leaders are the sort of terminal outcomes which already border the realm of actual impacts; i.e. they might be the precursors of long term institutional changes, that are in the process of taking place. 2.85 The above terminal outcomes also can be viewed as external conditions required for the impacts to materialize. In the latter sense, we have no evidence that this condition is significantly present as yet in the process. Notice, for instance, that even among those who felt that their initiatives have met with success, only 12% cared to mention specifically that authorities in their organizations have lent support to the initiatives (see figure 45). Yet, this is no evidence that such condition will not eventually appear, either. Remember, for instance, that back in figure 44 more than a quarter of respondent felt that positive results are still -or just- pending; meaning that they may be in the process of being realized in practice. There is even the case of young leaders saying that some results have come out of their proposals even though they have not been formally approved by authorities in the organizations (see figure 45). These are cases where, for instance, workshops or conferences replicating the teachings of the Program’s regular course have been taken up by young leaders of their own accord, without necessarily been formally sanctioned by the superiority. Although different “practical priorities” diverting the formal attention of senior leaders -for instance, during periods of elections- may have made this possible, we always have to presume the authorities’ at least implicit aquiescence. (ii) Proposals for organizational reform 2.86 The ex post survey also shows a significant majority of graduates engaging in proposals for institutional reforms and changes in practices and procedures. 60% of the young leaders from the Programs treatment group surveyed said that they have made specific proposals for reform in their organizations (see figure 46). The purpose of the proposed reforms included: increasing internal democracy in the political parties; improving the parties’ organizations from the municipal level on down the different levels where the parties grassroots operate; achieving greater gender equality within the party leadership; improving and extending the political parties Figure 45: Graduates’ opinion on reasons for success of education proposals. Source: Ex post survey, 2012 Figure 46: FYL Program Graduates making reform proposals. Source: Ex post survey, 2012 39 external links, both with other parties and with the civil society at large; furthering the participation of the youth in the organizations’ operation; and advancing the agenda of transparency within the organizations. The relative weights of the issues addressed in the submitted reform proposals, as declared by graduates, are depicted in figure 47. 2.87 Political leaders interviewed for this study have generally confirmed the visible proactivity of Program graduates found by the ex post survey in the area of institutional reform. They cite actions including internal proposals as well as proposals made by the party for changes in the political system at large. As an example of this, a case was reported of proposals for the Law on Political Parties -currently pending for national approval- in whose formulation Program graduates participated as members of one party body, to where they had been previously promoted. Political reform proposals have included suggestions for changes at local and intermediate levels of the parties’ organization and procedures; for instance: the formation, operation and composition of councils at the municipal level, and their relations to national, political and executive organs within the party. Proposals also have shown a focus on issues such as decentralization, merit-based promotion and party discipline. Also interestingly, organizational change proposals sometimes have been interwoven with issues of education, thereby promoting the role of knowledge in decision-making. See for instance in figure 47 how 17% of the proposals submitted as reform initiatives revert once again to the issue of political education. Other examples of these are: the motion that, as condition for advancing at the militant level, a minimal political formation process be required; or that, before the start of meetings, all party bodies discussed a relevant political or social issue of the day. On the other hand, in the opinion of at least one course lecturer and one senior party leader interviewed, there have been proposals too “academic” for practical application; or that, having been a requisite for graduation, did not necessarily reach the party instances for consideration. This is also in line with the sense of at least one academic interviewed, that theoretical and methodological considerations during the regular course’s tutorship may have prevailed too much over practical considerations, or issues of impact and feasibility, in articulating the proposals. 2.88 Also in this case, close to two thirds (62%) of those surveyed did not respond with any precision, or at all, to the question on whether their proposals had been approved by the leadership within their organizations; or explicitly said that they had not been approved (see figure 48). This is a more direct indication of the relative resistance, or lack of attention, by senior leaders to the initiatives of the Figure 47: Issues addressed by FYL Program graduates’ reform proposals. Source: Ex post survey, 2012 Figure 48: Approval of reform proposals made by FYL Program graduates’ Source: Ex post survey, 2012 40 young leaders graduating from the Program, that we already found in relation to the educational proposals. This indication is re-inforced by the response of the young leaders surveyed about their opinion on the success or lack thereof of their reform proposals. The same proportion (62%) did not answer the question about the perceived success or failure of the young leaders’ reform proposals; or they answered it ambiguously; or they indicated directly that their proposals had failed, as depicted in figure 49. Even more interesting are the reasons the young leaders surveyed cite for the failure of their reform proposals. A full 29% of them explicitly mention the lack of leadership support in their organizations. It is also noticeable that another 29% of those surveyed could not or would not speak, or do it unambiguosly, about the reasons for the lack of success, as can be seen in figure 50. As a counterpoint to this, 26% felt that the results from their proposals are just “pending”; an indication that they still expect an eventual success of their initiatives. Once again, external factors outside the control domain of both the Program and their graduates, such as, for instance, the occurrence of the mentioned national election processes in the country, may have diverted the attention of senior leaders on, or simply delayed, transformation processes that might be currently brewing, but have not materialize openly in practice yet. Also, among those who felt that their initiatives had met with success, 28% cared to explicitly mention the presence of support by the authorities within their organizations for the proposed reforms, as a reason for the success (see figure 51, on this respect). Finally, as can also been seen in figure 51, 14% of those surveyed who felt their proposals have suceeded also mentioned that complete results are still pending; indicating the presence of expectations of further succes in the future. An interesting additional point is the fact that 19% of those surveyed mentioned internal stakeholder mobilization –i.e. grassroots and intermediate leadership agitation and participation in favor of the reform proposals as a reason for success- suggesting that some contagion and/or demonstration effect on other members of the organizations might have also taken place as the young leaders who graduated from the Program went ahead with their reform proposals. Figure 49: Success of reform proposals made by FYL Program graduates. Source: Ex post survey, 2012 Figure 50: Reasons for failure of reform proposals by FYL Program graduates. Source: Ex post survey, 2012 Figure 51: Reasons for success of reform proposals by FYL Program graduates. Source: Ex post survey, 2012 41 (iii) Increased management responsibility for young leaders 2.89 Program graduates who have been promoted internally within the participating organizations have been generally appointed or elected to positions of higher responsibility at municipal or intermediate party levels. A full 54% of those promoted have indicated that their promotion has been to a municipal-level position or below, as it is depicted in figure 52. However, interviewed senior party leaders have confirmed also some promotions of graduates to more central leadership positions, such as, for instance, member of the party’s Political Committee, National Directorate or Central Committee. The ex post survey has established that 20% of Program graduates have been promoted to a national-level position, which represent an important Program outcome (see figure 52). A few also have reached positions as Vice-ministers in the National Government or as Congressmen and Congresswomen. This qualitative precision add focus to the quantitative findings already discussed above, on the proportion of young leaders who graduated from the Program and have been in fact promoted to positions of higher responsibility, especially within the participating political parties. The expectation is that these young leaders, mainly female, who have been promoted up the different levels of the parties’ leadership, will continue rising in the future, and apply the skills and values learned during training to further promote modernization processes within their organizations. (iv) Participation of young leaders in election processes 2.90 Beyond the measurable results on the participation of Program graduates in election process in general terms, already presented in this report, the ex post survey collected specific information on whether the young leaders participated as candidates or in any other way in elections, and what level of the party hierarchy was involved in the election (see figure 53) The reason for this is that the participation of graduates, for instance, as simple electors or organizers in the different levels of the organization is also important in terms of furthering and spreading the political modernization effects sought out by the Program. Figure 53 also shows that most participation took place in internal elections at the municipal level (41%). Figure 52: Level to which FYL Program graduates have been promoted. Source: Ex post survey, 2012 Figure 53: FYL Program graduates’ participation in elec- tions. Level of elections. Source: Ex post survey, 2012 42 2.91 The ex post survey found as well that a significant portion -although not the majority- of young leaders from the Program treatment group has run for elected positions internally in the political parties they belong to, and that an important majority of those running has actually won the election they entered in. Figure 54 depicts ex post survey results as to those Program graduates who participated or not as candidates in internal elections and the subsequent results. Also, there exists qualitative evidence provided by party leaders interviewed that not only Program graduates have increasingly participated in electoral processes, as candidates and otherwise, but that they also have become involved in the general promotion of democracy, both within the parties as well as, sometimes, when parties have taken public positions in issues of democracy. Examples of this involvement cited by party leaders range from a generally more active advocacy on democratization issues; more contribution with ideas to the internal debates by Program graduates, and specific comparative analyses of how democracy is practiced in party organisms; to a proposal that all internal positions be elective, or that specific representation quota be given to the youth in the political leadership instances of the party. 2.92 This is in line with the general recognition by senior party leaders that the Program graduates’ attitudes and proactivity toward the internal debate of issues have changed; and that young leaders that before had a retracted attitude, after the Program have participated much more actively in the theoretical discussions, analyzing and reflecting on issues, as well as taking more responsibilities and making more commitments than previously. Through this proactivity, Program graduates certainly have shown a better understanding of their rights and duties and an improved comprehension of their future role and potentials in party politics. (v) Young leaders’ participation in inter-organization dialogue 2.93 Interviewed senior political leaders do not generally confirm visible changes in institutional activities of outreach to civil society organizations, interparty-dialogues or external linking activities of any sort, but do not deny Program graduates pro-activity in networking and external interchange with young leaders of other political parties and CSOs. This probably means that the measurable external links and outreach activities of graduates as observed in the field surveys occur mostly by personal initiative and not as part of a concerted institutional effort by the organizations themselves. Some senior political leaders interviewed have even recognized that the outreach and linkage with social organizations is a weak area of their organization’s agenda and practices; as well as one where much improvement is needed. That does not mean, however, that the organizations do not recognize the Figure 54: FYL Program graduates’ running for, and winning internal elective positions. Source: Ex post survey, 2012 43 importance and advantages of the outward relations with other political and social organizations, developed by the Program graduates. 2.94 There are important qualitative insights provided by party leaders that reveal, at least, a minimal evolution toward a better kind of exchange practice and relations among political parties and between them and the community and citizenry at large; even in the midst of a hard fought political campaign. One political leader saw as very important that graduates of the Program visibly improved their participation in the recent campaign in qualitative terms; i.e. with better political arguments and more emphasis on programmatic proposals and the debate of ideas, rather than on mere political pandering and agitation. The sharing of young leaders form different ideologies/parties promoted by the Program in collective efforts such as, for instance, diagnosing the country’s present situation, has made possible for these young leaders not only to acquire useful cognitive knowledge, but also to “live” politics in a different way; i.e. one in which adversaries are not necessarily seen as “enemies”, and negotiations can be based on principle and rational arguments, rather than on pure sectarian prejudices. A consensus is observed among the political leaders interviewed on a positive assessment of the “primary relations” developed by Program participants with their counterparts from other parties and CSOs; as well as the development of “extra-party networks”, as desirable learning experiences and ones that may contribute to eventually change the political culture in the country. (vi) Young leaders’ transparency & accountability 2.95 The present study has demonstrated a visible and positive evolution in the behavior of participant young leaders seen during the Program’s execution in what concerns transparency and accountability. This positive evolution has been important in itself and statistically significant in comparison with that of young leaders who did not participate in the education process. The ex post survey found very high and statistically significant differences between the behavior of female leaders who graduated from the Program and their male counterparts, towards more transparency and accountability; which constitute a clear and also very important development outcome attributable to the intervention. E. Program impacts 2.96 The development impacts expected from the Program refer to the longer term more structural transformations it may have contributed to in the organizations involved, and in their regular practices, by virtue of the behavioral and attitudinal changes induced in the several cohorts of young leaders that graduated from the Program. According to the metrics stipulated in the Program documentation, impacts were to be gaged by the number or percentage of political parties that institutionalize mechanisms and permanent actions directed to: (i) strengthen gender equity and youth participation in the organization; (ii) extend training programs for the party members; and (iii) improve transparency and accountability within the Electoral Law. 44 2.97 Being formal, operational and structural in nature, the changes mentioned above may be subtantiated by direct observation and factual corroboration of organizational decisions. In that sense, the present research has found no hard, conclusive evidence that any of the mentioned mechanisms have been formally adopted and operationalized in practice at the national level by the political parties involved. Generally speaking, organizational and procedural changes of that nature are adopted in the context of general assembly meetings (the so-called Congresos) of the political parties; and these events are relatively infrequent. Also decisions by the Congresos both take a considerable amount of political haggling and consensus building, and often are not binding; so their practical application can be obstructed or reversed, or even not materialize at all. Several political leaders interviewed for the present study have indeed mentioned that reform proposals advanced by Program graduates, or the posibility of their promotions up the hierarchical ladder, will not be decided upon before the next party assembly convenes. Therefore, to the extent that the structural changes toward a more democratic and transparent political system sought by the Program would effectively materialize, it may still perfectly require additional time for these changes to mature. The above does not mean, however, that there is no evidence of early and localized evolution in areas such as gender equity, upward movement of graduates -especially to local leadership positions- and a minimal, but visible, transformation process to expand political education and more transparent practices. This evidence, which in some cases is circumstancial but in others is provided by statistically significant results, suggests the presence of what we may call both general and specific “trend impacts”. 1. General trend impacts 2.98 Budding long term transformations can already be inferred from some terminal outcomes discussed aboved which “touch” the border of preliminary impacts, mainly associated with the behavior of female program graduates and their relative further promotion by current leaders within participant organizations. There is also circumstantial evidence that the demostration effect of Program graduates inside those organizations may be inducing at least some behavioral contagion in the population at large. Whether this contagion is due only to the 2nd phase of the program or to the cummulative demostration effect of all the cohorts graduated by the Program since its inception in 2005 -six in total- we do not have a way to prove statistically. Yet, whether initiated more or less recently, this apparent trend can be circumstantially seen in the declared behaviors of individuals of the control group that actually participated in the ex post survey. The gender equalizing trend also seen in the behavior of control groups may be an indication that the movement toward equality between women and men being generally experienced by all organizations may be getting strenghtened, by more profound, longer-term changes in the same direction, induced by the Program but still brewing under the surface. 2.99 There are also circumstantial reasons to argue that some of the behavioral outcomes found to be statistically significant in Program graduates may already be entrenched enough in their normal practice as to keep contradicting prevailing attitudes and be maintained in the face of strong currents in the opposite direction. A case in point is the pro-activity of Program graduates to engage peers 45 outside of their orghanizations in meanigfull political dialogues. In the Dominican Republic and to the extent that these dialogues differ from mere partisan warfare, it can be reasonably expected that the practice of dialogue with political adversaries would diminish considerably during periods of heightened electoral confrontation, as it was the period of the Program’s execution. Indeed, this expected trend has been confirmed by the surveys in the different control groups of all cohorts of the Program, with no exeptions and high statistical significance of differences with the treatment groups. So the exact opposite and strongly diverging trend observed in the Program graduates, during exactly de same period, might just be the mark of a true behavioral precursor of longer term, more structural changes to come. 2.100 It can also be argued that, with their more abundant presence in positions of leadership at all levels of the parties, young leaders will eventually exert greater and greater weight on institutional decisions an reform processes. So, because impacts usually require extended periods to mature in practice, the longer-term changes that apparently are currently budding as results of the Program may just as well need additional time and care to come to fruition in the open. 2. Specific trend impacts 2.101 Apart from the general preliminary trend impacts that we may infer from the behavioral data available, we may also comment on trend impacts in the specific areas stipulated in the present study s.o.w. of: (i) gender equity and youth participation; (ii) extended training for party members; (iii) improved transparency and accountability. a. Gender equity and youth participation 2.102 In the present report we have presented conclusive evidence of the equilizing effects that the Program has induced between female and male young leaders in a number of significant observable effects and behaviors. Through the years, most political parties of consideration in the country have come to recognize women’s rights to participation in political decision-making, as well as the desireability of women’s participation, through holding at least 33% of the leadership positions. Now, even if this benchmark is not always complied with, it points toward a transformation that may be considered as already adopted and irreversible in principle. Such trend, of course, may not be viewed as a Program impact, but the Program has certainly contributed to expand even further the participation of women in politics, with young females participating in the courses in excess of the 33% mark and with better results than in the case of male leaders. A final outcome that has also been substantiated, and that comes even closer the actual impact expected in what concerns a gender equity mechanism, is the fact that considerably more women graduates of the Program have come to be promoted or elected to positions of higher responsibilities. Whether this trend will eventually induce the formal adoption of a mechanism to enforce gender equity is, of course, still an open question. But more women factually in positions of power, even if so far they involve just local o intermediate leadership positions, is arguably a precursos outcome that raises the probability that such institutional mechanism may eventually be adopted in practice. 46 b. Extended training for party members 2.103 The evidence available over recent years concerning structural instutions dedicated to political education reveals a tendency opposite to that sought out by the Program, and there is little that the present intervention has been able to do to counter this tendency, apart from the callings from a portions of its graduates for the creation or restoration of the parties’ political schools. Generally speaking, the political schools in the country have been minimized or discontinued, as the weight of political doctrine, education and ideology has considerably diminished in favor of the all consuming electoral efforts by the political parties. The result is a loss of systematicity in the educational endeavor, which does not constitute any more a steady effort -even in those parties where it used to be- and the downplaying of political education to a purely episodic, side activity. On their part, public and private educational institutions in the country have apparently not filled the gap, either, between the supply and the manifest demand for a professional, modernizing political education. Despite the efervescence toward education induced by the Program, the fact remains that leaders in charge of political education are normally not positioned close to the main decision centers of the different parties. Also, whatever limited -or abundant- resources parties have at their disposal normally have gone to stregthen their electoral machinery, and not the ideological or political education structures that may remain. So, the visible education multiplier effects and enthusiasm with education infused by graduates from the Program, plus the several educational proposals submitted such as the creation of “concept documents” for political schools, the preparation of curricula, the suggestion that a minimum of periodical courses be given to militants, etc. may constitute precursor movements toward the revitalization of the now deflated political schools, and/or creation of new ones; although the impact is not yet visible anywhere. 2.104 Finally, some of the political leaders interviewed have expressed their hope that the recently enacted enlarged period between elections will allow political parties more time to concentrate on necessary internal reforms and activities, among them: the education of their militants. c. Improved transparency and accountability 2.105 This is the area of concern of the Program where the least obvious progress can be observed in terms of permanent institutional transformations, despite positive changes in the observed behavior of Program graduates. Therefore, such a clear lacking in verifiable institutional response to the behavioral and attitudinal changes in young leaders does not bode well for short term expectations about future structural changes by the institutions involved. 47 III. Conclusions A. Operational delivery 3.1 The summary overview of operational performance presented in section II.A and II.B of the present report attest to the fact that the Program did delivered its expected products. In the case of the Program intermediate outputs, the lack of consistency in the documentation as to what exactly were the targets agreed on beforehand, , makes difficult to pass a clear-cut judgment on operational performance concerning those specific deliverables, such as courses given, events held, etc. Concerning final products, for which targets were more clearly set, performance was mixed: with the Program sometimes falling short of targets, as in the case of the number of Program graduates per year, or the attendance to the course on Women in the Legislative Process; and sometimes exceeding expectations, as in the case of female young leaders graduating each year from the Program’s regular course; or the education multiplier effect promoted by young leaders at the grassroots level of their organizations, which has been reported to be quite considerable. 3.2 Of special note concerning the Program final outputs are indeed the overachievements in terms of women graduating from the regular courses and the short term multiplier educational effect mentioned above. Measurements consistently show that female young leaders received the Program training courses in numbers much greater than expected, and way above the 33% usually considered among the political parties of the Dominican Republic as an accepted benchmark for minimum women’s participation in positions of leadership. Women participation has certainly become a very visible trademark of the FYL program 2nd phase. Also worth noticing is the important effect of the decision -taken for this 2nd phase- of changing the 1st phase’s Technical Assistance Program (P.A.T.) into a training course specifically designed to prepare the best graduates from the regular courses, so that they could later facilitate workshops on the same subjects. This change proved pivotal in unleashing the important number of workshops and other training activities done by graduates with members of the participant organizations grassroots after the Program regular courses finished for each cohort. 3.3 So, beyond the fact that a number of young leaders received a better fine-tuned, denser and more coherent regular course in the 2nd phase of the Program, the high proportion of women among these graduates and their quite visible proactivity in divulging and extending to others the knowledge, values and skills they acquired, greatly enhances the quality of this output. Other outputs worth mentioning, for their implications for increased knowledge of the political reality and the continued expansion of the Program’s learning experience beyond execution, are the products associated with the diagnosis of the contextual situation in the Dominican Republic, and with promoting the interfacing of young political and social leaders beyond the boundaries of their respective organizations. The updating of the Diagnóstico del Sistema de Partidos and the realization of the National Young Leaders Congress, with the subsequent consolidation of the Young Leaders Network achieved in the context of the Program are steps in this direction. 48 3.4 Therefore, despite some observed fluctuations in the numbers, in general the Program appears to have submitted its intermediate and final products in the quantity generally expected, with appreciable quality, and -controlling for the fact that there was a delay at the start for circumstances specified in the present report- within the time stipulated at inception. Beyond the opinion of graduates, who are universally appreciative of the training received, other social stakeholders of the Program, such as senior political leaders and other institutional representatives, also generally value its outputs positively and praise the dedication of the executing Consortium to the Program’s correct and timely operation. Consequently, it is safe to conclude that the Program submitted its deliverables in quantity, quality and opportunity sufficient enough to be able to induce the outcomes and to contribute to the impacts that are expected from it, as stipulated in its original development hypothesis. B. Short-term changes induced 3.5 The individual changes the present research set out to investigated referred not only to the kind of cognitive changes as can established by a learning test administered at the end of an education process, to measure the individual’s gain in knowledge. We also attempted to evaluate the behaviors of the individual graduates that were expected in actual life as a consequence of the learning proccess. In other words, the working hypothesis of the present reasearch was based on the idea that the only way to determine whether the knowledge gained truly changed the graduates attitudes -“changed the chip in the minds of young leaders” as put by a former AID staffer- is by evaluating the external behavior of the relevant subjects. 1. Acquisition of knowledge 3.6 The knowledge gain of participants in the Program has been substantially documented in section IIC, as well as proven to be statistically significant by the several independent tests applied in the present research on the data reported by the executing consortium. Comparison with knowledge gain of individuals of the control group -i.e. the result of possible separate, independent political training￾was not feasible in the present study; yet the research did control for the fact that individuals of the control groups might have received during the same period some such separate, independent training. Results of statistical tests on the two populations showed a very clear and statistically significant difference in the incidence of training between the two groups; and a negligible weight of such equivalent training in individuals of the Control Groups. Therefore, differences in ensuing behaviors between the two groups are more aptly correlated with the presence of political training in the Treatment Groups and the lack thereof in the Control Groups, than with mere happenstance or other uncontrolled factors. 3.7 Analysis of trends in knowledge gain has also shown strong general equalizing effects. Having started from disparate grades at entry at least in one of the measurements used, the different student cohorts in all regions of the Program have tended to improve and to converge toward a similar score at the courses’ exit point. This equalizing effect is even more consistent and pronounced in the case 49 of gender. Female participants in the courses have consistently started with lower grades than male participants, and have always improved during the courses to the point of equalizing, and even surpassing, the scores of male graduates: a very salient and important Program outcome. 3.8 The hard evidence provided by statistics and likelihood tests for knowledge gain based on scores has been complemented by the opinion of some senior political leaders interviewed. In their views, the learning of young leaders under their supervision went beyond the acquiring of theoretical concepts and practical skills, to actually “living politics” in a different way than the normal experience those young leaders have within the currently dominant political culture. This kind of learning is generally associated with the experience the students had of interfacing and networking with other political and social leaders, during the Program: an interfacing and networking based on ideas and programmatic proposals, as well as conducive to the development of primary relations, instead of the usual inter-party confrontation and rancor. Therefore, as in the previous phase of the Program, tolerance and democratic coexistence with ideological and political foes have become part of the learning experience, going beyond the pure cognitive realm into actual sensitivity and emotional intelligence training: another significant outcome of the Program. 2. Observed behavioral trends 3.9 The study has recognized that designers included in the Program expected results chain behavioral outcomes that can be interfered more heavily than others by factors originating outside the Program’s control or influence domain. The achievement of outcomes measured by indicators such as: the promotion of young leaders to positions of higher responsibility, or the participation of young leaders as candidates in election processes depend on decisions made by the current senior leadership of the organizations involved to a much greater extent than the achievement of outcomes measured by indicators such as: the Implementation of training courses for the organizations’ grassroots; the submission of proposals for organizational reform; the increased young leaders’ involvement in municipal participation or inter-party dialogue initiatives; and the increased young leaders’ transparency & accountability. The latter we have called type A indicators and the former type B indicators. The study has found important differences between the two types of behavioral results. 3.10 As a general rule and measured by the percentage of leaders showing expected behaviors, much greater differences (changes) from the base-line values to the exit and closing values are found in the Treatment groups, as compared to those of the Control groups in what concerns type A indicators than those found concerning type B indicators. Behavioral changes have also found to be always positive (upward trend) and fairly pronounced in the Treatment groups on what concerns type A indicators; while the trend in Control groups is much flatter or outright downward in many cases, concerning the same type A indicators. On the other hand, for type B indicators the trend has been found to be either positive (upward) or negative (downward) for both groups at the same time. Finally, and most importantly, the differences between control and treatment groups have been generally found to be no statistically significant in what concern type B indicators, regardless of the 50 direction of the trend; suggesting the workings of factors other than the Program in these particular results. Conversely, differences between treatment and control groups in several of the type A indicators have been found to be statistically significant; suggesting high correlation with the training given by the Program. 3.11 Indeed, the research confirms positive behavioral changes in the treatment groups well above those of the control group in terms of the percentage of individuals increasing their activities of training directed to the organizations’ grassroots, submission of reform proposals, inter-party dialogue, and accountability, both internal and external to their organizations. “Upward” behavioral changes concerning the dialogue beyond the organization’s boundaries deserve special notice, particularly when compared with the clear downward trend of the control group in the same behavior during the same period. The latter could be expected in times of high political tension, as was the period of Program execution when two major political elections occurred. That’s precisely why the fact that the treatment group had trended in the opposite direction with high levels of statistical significance during that period is a noteworthy Program outcome. 3.12 Also the research has yielded statistically significant differences in values and in trends between the treatment and control groups, concerning transparency and accountability behaviors; specifically the habits of disclosure toward: (i) superiors within the organization; (ii) subordinates within the organization; and (iii) general public outside the organization. The upward trend observed in all these habits is noteworthy as well when compared with the corresponding downward trend in the control groups; especially in what concerns the disclosure toward superiors: a behavior supposed to be mandatory in all hierarchical organizations. But even more remarkable is such trend in the disclosure toward the outside public, which is not only not-mandatory, but also not a “normal” behavior either in the case of political parties. 3.13 Finally, concerning type B indicators, the observed trend is that more and more young leaders participate in internal election both in control and treatment groups, with very small and not statistically significant differences between the two. However, the trend is the opposite in what concerns young leaders being promoted to higher positions, both in control and treatment groups, also with small and not statistically significant differences between the two. Since statistical tests suggest a low correlation with the workings of the Program, these results should be taken as a clear consequence of factors operating outside the influence domain of the intervention. C. Medium-term changes induced 3.14 The field surveys done after the closing of the Program in 2012 shed light on the medium-term persistence of behavioral changes observed during the Program execution, allowing some confirmation of changes, or providing important qualifications to those trend results. Also, despite the fact that ex-post comparisons between control and treatment groups were not statistically possible, because of difficulties with the control groups, there are strongly significant results observed inside the treatment groups that provide additional confirmation of results and new 51 interpretation angles to the quantitative data observed, as well as complementary qualitative perspectives on medium term changes. 3.15 A first important conclusion is that despite the downward trend in the promotion of young leaders in general -treatment and control groups- the opposite has consistently occurred in the case of female graduates who, as a trend, have been promoted in greater proportions than their male counterparts; trend in a type B indicator that has been essentially maintained as confirmed by the ex-post survey. A full 64% of all young leaders’ promotions have been to provincial level positions or below (54% to municipal level or below), and only 20% to national level positions. The surveys also confirm that the female graduates’ proactivity as measured by type A indicators, such as reform initiatives taken, participation in dialogues with other organizations, and involvement with the young-leaders’ network have remained high, compared to those of male graduates, after the Program was closed; with tests yielding high statistical significance, i.e. high correlation of results with the workings of the Program. 3.16 Performance in terms of reform proposals presents a mixed bag. A clear preponderance of proposals (42%) focused on issues of internal democracy and youth participation; yet, only 40% of proponents report that their initiatives were approved. Still, the ex post surveys confirm that the advantage of female graduates concerning such type B indicator such as the approval of proposals have remained high after the Program’s closing, compared to those of male graduates. The surveys also confirm that only 40% of proponents felt their reform initiatives were successful. 29% of proponents mentioned lack of authorities’ support as reason for failure; and 28% mention authorities’ support as reason for success. Also, 26% of those who felt their proposals failed, and 14% of those who felt they succeded, mentioned that results are still pending; suggesting that, in their view, the changes are still in the process of materializing. In what concerns educational proposals, 40% of them referred to direct dissemination of knowledge and/or the creation/strengthening of Political Education schools; and, again, 40% of proponents felt their proposals had succeeded. 30% of those who said they did not suceed mentioned specifically the lack of leadership support in their organizations; and of those who said they had succeeded only 12% cared to mention that authorities lent their support, while another 12% mentioned grassroots mobilization as a reason for success. 3.17 Finally, the ex post surveys show that only a little more than half of young leaders who participated in the Program have gone on to run for elected positions (51% of those at the provincial level or below), but a substantial majority of those running (65%) have actually won those elections; a most important type B indicator of Program outcomes. D. Contributions to long-term transformations 3.18 The data discussed in section II.E of the present report appears generally inconclusive about impacts of the Program as hereby defined, or suggestive that they are not yet visible, especially in what concerns the structural and institutional behavior of the political organizations involved. Some early evolution is observable in terms of upward movement of graduates, specially female, to local and 52 higher leadership positions, and there is clear atempts by an appreciable number of young leaders to induce changes, both structural and behavioral, in their respective organizations. However, there is no compelling evidence that such attempts have induced any, but the most minimal, transformational processes inside the organizations. Even in what concerns earlier manifestation of proximal or intermediate impacts, the evidence at hand is rather feeble. It appears that young leaders educated in the modern democracy paradigm must still gain further access to power positions in order to be more effective at their change initiatives. 3.19 But the absence of visibly permanent institutional changes does not mean that such changes might not be in the process. The new behaviors induced by the Program, getting entrenched in the young leaders normal practice enough as to even contradict prevailing attitudes and face of strong currents in the opposite direction; the demostration effect of such Program graduates’s behavior, generating some contagions inside their organizations; and the clear gender equalization effect of the Program, strengthening the young female leaders’ positions and, presumably, their weight in future decision, as they are promoted and/or elected to higher leadership posts, can be viewed as precursors signs of Program development impacts brewing under the surface. 53 IV. Recommendations 4.1 The 2nd phase of the Program on Political Education and Management, object of the present study, adopted a number of adjustments that had been recommended by the evaluation study done back in 2008 when the 1st phase of the Program was completed. Now it is easy to see how, at least in part as consequence of such adjustments, the 2nd phase of the Program resulted in an intervention with a sharpened development focus and a more nuanced strategy to achieve the expected results. The comments in this section are intended to bring some detailed perspective on the effects of the changes adopted; and suggest to political parties, the private sector, civil society organizations and the USAID mission, ways to build on achievements and keep improving performance in future political modernization efforts of this type, based on lessons from the experience and conclusion of the foregoing analysis. A. On the extent and reach of the Program 4.2 It is generally acknowledged that educational efforts do not normally yield societal transformations over the short run, and that it would usually take a longer period for structural transformations to come about; the more ambitious the expected changes are, and the deeper the resistance to such changes is rooted in the environment. The passive but effective workings of the resistance to change, and the influence of a number of other factors falling outside the control domain of any transformational effort, but impinging negatively upon its results chain, not only contribute to delay changes, but may even reverse them in the absence of a continuous counterbalancing effort. In the opinion of the authors, the type of transformations pursued by the present Program entails a long￾term endeavor of social engineering and the expected changes probably necessitate generational replacements, institutional re-designing and underlying shifts in the body social. 4.3 The recommendations on evaluation methodology made in the study of the Program’s 1st phase and adopted by designers for the Program’s 2nd phase are, to a great extent, what now makes possible for us in the present study to discuss the evidence for the Program’s effectiveness on a more scientific and objectively verifiable basis. Such evidence and its conclusions, however, are significant only for the 3-4 years of the Program’s 2nd phase: a period admittedly much shorter than the one required by a transformational process of the type discussed, to bear its final fruits. The reason for this is that the mentioned necessary methodological adjustments in the Program’s evaluation framework had to be adopted at mid-course, which made impossible to evaluate the 1st and the 2nd phases on the same epistemic basis, thus forgoing the chance to rigorously ascertain changes that may have been emerging through a total intervention period of eight years. Nevertheless, the study yields enough precursor evidences, which are statistically significant, as well as strong circumstantial and qualitative hints, to suggest that at least some of the final structural transformations intended by the Program (1st and 2nd phases) may be already in the offing, albeit not openly visible yet. 4.4 The attitudinal and behavioral changes promoted in the young leaders through the educational effort should arguably remain the main vector to induce the higher-end institutional behavioral changes in 54 the targeted political organizations and in their leadership, as envisioned by the Program in its first two phases. However a continuing intervention will also probably need a better focus on separately targeting for change the factors of resistance operating in society at large, as well as on better handling the ineffectiveness risks born in factors outside the control domain of the intervention. 4.5 Perhaps one of the main structural factors resisting change in the political system is the fact that, although frustration with politicians is growing in the Dominican Republic, the general culture actually sustains the way politicians currently do business. It is a general consensus among program stakeholders that the voice of clientele-pandering and political favors resounds loudly at the social grassroots level, because this is still the language that ordinary people understand most in everyday life; and so, the relationship between political parties and society in the Dominican Republic remains one of manipulation on the part of the political parties and of dovetailing to favor-peddling on the part of society. Also, to the extent that this feedback mechanism is self-reinforcing, one can understand that the system of graft and political kickbacks, instead of weakening, may actually be in the process of getting stronger and further entrenched by its own social success. Barring some momentous convulsion in the system that may shake such deep rooted expectations in the political clientele -as, for instance, a major economic debacle undermining the populace’s faith in the current political class- it would normally be difficult for any emerging leadership to succeed in promoting a non-clientele-pandering political behavior, when such a feedback system of expectations is operating and continually being self-reinforced in the opposite direction. 4.6 Of course, for as long as major counterbalancing governance mechanisms are not consolidated in the society at large -such as, for instance a justice system that truly serves as a leverage for social control and political accountability- the current system is unlikely to go away easily; and there is perhaps very little that interventions such as the present program can do about those macro-level mechanisms without unduly changing its nature and scope. However some addressing of the social resistance factors may still be possible within the curfew of the Program, through specifically targeting civil society attitudes as well as cracks at the base of the political edifice that are already emerging, especially in what concerns the pandering system. For instance, despite the arguments above and the fact that on the surface things appear very much business as usual, in real fact society and political parties are growing apart, not closer. Arguably people keep giving the benefit of the doubts to democracy, but not necessarily to the behavior of current political parties. The general support is still displayed to leaders who solve personal problems through concrete handouts, especially at the local and grass-roots levels; yet the “price” attached to such handouts, both personally and socially, when it becomes increasingly clear that the power of this system is linked, for instance, to the power of activities in the underworld economy and other socially unsavory endeavors, make at least a portion of society to recoil from such practices. 4.7 Also, on the positive side, factors moving political behavior in a non-clientele-pandering direction are also already at work as a consequence of the Program, and should be strengthened. Senior political leaders interviewed generally agree that the primary relations developed among participants of the Program have contribute to enhanced communication between young leaders of different political 55 parties, and between those and social leaders. This is also seen as a small but significant contribution to positively changing the political culture in the country. Specifically, in sharing and networking with social leaders and individuals of different minds, young leaders not only cognitively learn the curriculum contents on modern democracy, but also learn to “live” politics in a different way: one in which interfacing and negotiations can be based on principles and rational arguments, and not on possible kickbacks; and one in which issues can be discussed on the basis of mutual respect, and on a more serious, programmatic manner. 4.8 So, concerning the extent and reach of the Program, we make the following suggestions based on what has been learned from the present research: Recommendation # 1: Keep the program working with the design improvements. For the reasons discussed in the previous paragraph, it stands to reason to maintain this development intervention; not only in order to ensure long term impacts, but also to avoid the wastage represented by the possibility that hard-fought-for changes, which are still budding, may be reversed at the end by the forces resisting modernization, for want of a more prescient and perseverant educational effort. Recommendation # 2: Better target societal factors retarding political modernization. Include or strengthen in the courses curricula, those elements relating to community mobilization in favor of a more enlightened behavior on the part of political leaders. Consider also better targeting individuals belonging to the society at large in the courses and seminars, along with the young political leaders mainly targeted now. Natural community leaders at the grassroots level, receiving the same political education as the young leaders in political parties, might positively complement the effort, enabling the Program to deploy the same ferment vectors toward a modern democratic paradigm, both inside political parties and directly in the mist of communities as well. Also increase the effort to identifying possible threats to the effectiveness of the modernization drive, originating outside the interventions’ curfew, making more explicit the underlying assumptions associated with those external factors, and stimulating the adoption of risk management strategies that may mitigate the negative effects of the identified threats. Recommendation # 3: Emphasize training on digital networks at the beginning of the course, for an early eliciting of the habit of internet outreach and networking activities. The good networking and liaison results observed in the Program so far between young leaders of political parties and of Civil Society organizations, through which they have learned to value and appreciate each other, augurs success for the possibility of strengthening a more program-based, development-oriented relationship between parties and society as a whole; especially to the extent that this networking is done in the context of spreading non-clientele-pandering political values, attitudes and behaviors. The earlier the networking skills are promoted in the course, the better effect is should have in the actual conversation between young political and social leaders. 56 Recommendation # 4: Emphasize more the sharing of practical activities within courses, as part of the curricular design. The primary relations developed among participants in the Program, and generally praised by all stakeholders, appear to develop more or less as a “natural” outcome of the Program, associated with the simple fact that students do partake and share the same space and experience for several months. Yet, this outcome may be more specifically targeted in the course curriculum proper. Additional practical activities requiring team-work may be included by design in the curriculum to enhance a shared learning experience an promote the “living” of politics as respectful participatory exercise of engaging social constituents and political adversaries, not “clients” and “enemies”. B. On the recruitment and selection process 4.9 The establishment of control and treatment groups, recommended in the study of the Program 1st phase, and adopted in the selection of participants in the 2nd phase is what allows us to make a much more robust argument for attributing the development results observed to the working the Program, and not to mere chance or other unknown factors. As understood by the present researchers, the process combined criteria for eligibility with a random selection of participants, by first selecting the general eligible population, based on academic aptitudes for entry, and then making the random selection of individuals in the Control and Treatment group. The selection, however, was done anew each and every year of the Program, with the unforeseen result that those individuals belonging to the general eligible population, but not selected to take the course each time around, eventually developed a perception of being repeatedly “excluded” on purpose, with the associated sentiments of frustration and discontent. 4.10 As discussed in the text, by the time of the ex post survey, which was designed to confirm the differences between resulting behaviors of the Control and Treatment groups found by the surveys performed during execution, these sentiments had come to spark such levels of animosity in the individuals of the Control groups as to provoke the actual refusal of many of them to even answer the survey. This, in turn, severely restricted the size of the sample of control individuals that could actually be surveyed, affecting the statistical power of survey results, in terms of comparison with the Treatment group, as well as the very comparability of results from those control individuals who at the end did participate, with those of the Treatment group. Although control groups usually are the most difficult component to manage in a quasi-experimental design, especially with porous, overlapping populations subject to high expectations, there are alternative ways to handle the selection process, still respecting the necessary random or combined selection criteria, but at the same time mitigating the adverse consequences of the expectations game. 4.11 The courses were also equally divided in terms of number and participants in the six regions of the country where the Program was executed, ostensibly because of egalitarian considerations. Given the demographics of the Dominican Republic, however, the population is very unevenly divided 57 between these regions and, therefore, the criterion of equal number of events and participants for each region is not really egalitarian. A principle of proportionality in selection of individuals by regions -as the one adopted, for instance, to establish the “quota” of participants of each political party- would have better satisfy the criterion of “equal access” to the courses. Apart from the resulting lack of equality, there is also circumstantial evidence of selection distortions due the application of this “equal” access criterion. For instance: in some cases, the selection criteria may have been relaxed in order to fill a local or regional quota for which there was not enough eligible candidates; or the “leaking” of individuals may have taken place from one region where they would not be selected, to another in which their chances considerably improved; or a similar “leaking” of individuals may have taken place from political parties to CSOs, for similar reasons. 4.12 So, concerning the recruitment and selection process, we make the following suggestions based on what has been learned from the present research: Recommendation # 5: Avoid repeating each year the selection of control & treatment groups There is probably no practical way of avoiding the sense of frustration in any individuals who has not been chosen to participate in a program for which the demand is great and on which the expectations are high; no matter how truly random the selection process may have been. However, repeating the selection exercise every year may have contributed unnecessarily to exacerbate that sentiment on the people not chosen each time around; especially if the general population is, for whatever reason, already pre-disposed or suspicious of the selection process (an attitude not unusual in the “realpolitick” scenario of the Dominican Republic) The fact that each time around participants in the selection process were also required to fill up a very extensive survey form -a condition required by the “moving base-line” stipulation established by the USAID consultants before the start of the Program- may have also been an aggravating factor for those not selected at the end. A one-time only selection exercise, of the participants in the different cohorts, done at the very start of the program may neutralize or at lessen this cumulative negative reaction from the part of the members of the control groups, whose only definition is simply and precisely: not having been chosen to participate in the courses, but whose participation in the all-important evaluation activities later is very important, but necessarily voluntary. On the contrary, in the event that individuals chosen at the start to participate were not available later at the moment of the course start, a random selection of individuals from the control group could be made to replace them, with no frustration implied. 36 36 It must also be notice that moving base-lines are not strictly necessary to establish attribution of results to development interventions, nor it is a questionnaire excessively extensive to collect the necessary information in surveys or, for that matter, to establish moving base-lines. 58 Recommendation # 6: Apply a criterion of demographic proportionality in programing the number of courses and participants per region Demographic proportionality not only is a truer criterion for equality in access to the Program, but also may avoid unduly incentives to distort the correct representation in each region. It is also not incompatible with any other measure to proactively promote more participation of disadvantaged regions; such as, for instance: adding, once the proportional representation in chosen, additional representatives from those “minority” regions. C. On the program effectiveness focus 4.10 Results show time and again that after graduation young leaders frequently find themselves in an environment unfriendly or otherwise not conducive to the realization of reform ideas within their organizations. This implies a great risk for graduates to fall back into inactivity or to develop cynic attitudes toward the idea of change unless some sort of support is provided and enticement to keep up the struggle. Networking, developing trans-party leagues and joining community groups -practices for which courses provide a lot of sensitivity training and stimulation- represent a useful way of maintaining the collective spirits up in the face of adversity. Yet, it must be recognize that proposing ideas that may not have much practical impact or be unduly difficult to apply in practice, may add unnecessarily to the sense of frustration in young leaders, as well as to the sense of general ineffectiveness of the effort. The Program may find ways to increase the potential success of proposal from the very drawing table, during tutorships. The Program may also add additional divulging activities with the organizations’ leadership to aid in the general promotion of initiatives. Recommendation # 7: Emphasize the issues of potential incidence and feasibility as design criteria for proposals developed during the tutorships Strictly academic criteria, methodological aspects and formal presentation of papers may have acquired preponderance over the issues of actual impact and viability, as requisites for presentation and approval of reform initiatives and proposal in the Program’s tutorship. The whole tutorship process also appears to have been lacking in uniformity and consistent criteria for what is considered appropriate in such proposals. Requiring, for instance, a specific implementation strategy, and a content index denser in operational details, may contribute to proposals conceived not only as academic exercises, but with emphasis on real effects and practical feasibility, within the respective organizations Recommendation # 8: Find ways of winning the hearts and minds of established leaders to favor modernization Taking advantage of the clear footing and prestige acquired by the Program within the political parties, a more intense effort to divulge results and proposals may be undertaken, not only with the leaders in charge of education, doctrine or political schools in the parties –who are generally 59 convinced of the importance of modernization- but with leaders positioned closer to the real centers of political decision making in those parties. Recommendation # 9: Offer ways to improve the graduates’ ability to flesh out and discuss their initiatives with colleagues and superiors. Ways should be found to provide Graduates with orientations as to how to present or proceed with their proposals vis-à-vis the superiority in their organization, following a specific implementation plan. The Program may teach student to design specific incentives for “selling” initiatives on their technical quality or benefits for the party, beyond pure theoretical or doctrinal correctness. D. On program metrics and evaluability Recommendation # 10: Instruct the avoidance of excessively prolific survey questionnaires Heeding recommendations from the first evaluation study, the Program performance indicators were streamlined and the indicators were much better defined for the 2nd phase, resulting in a more precise and solid final evaluation of results. But, in the meantime, the number of variables included in the surveys done during the Program execution exploded several orders of magnitude beyond the few indicators required by the Agency; making data collection and handling unnecessarily difficult and producing other adverse consequences. The important question in evaluation is the quality of the information gathered and of indicators measured, not their quantity. Very extensive questionnaires with inquiries much in excess of the ones required to measure a relevant small number of well-defined indicators, were not only unnecessary but also risk-inducing inadequate reactions from the subject surveyed, whose voluntary, bona-fide and truthful participation is critical for the adequacy of surveys. Recommendation # 11: Instruct the avoidance of undue changes in base-line definitions Evidence has been found of changes in the way questions are posed to the individuals surveyed in different base-line surveys, resulting in ambiguity of what may have been actually measured in each survey. Moving base-lines mean that an indicator may vary in is quantitative starting value from period to period, but never that the indicator may change its narrative or definition, on pain of losing the comparability of values from period to period. To maintain consistency, and comparability, it is important that the way the questions are articulated be identical in each survey. F. On the effort’s long term sustainability 4.18 It is clear to almost everyone involved that no long term sustainability of the political education effort is feasible unless the political organizations themselves own up to such effort, either through 60 their own internal institutional mechanisms (especially in what relates to the most ideological aspects of the education) or through funding the education of their leaders by outside independent institutions of higher studies (especially in the more technical, transversal kind of political education, applicable to all political parties). In this context, the matching grant strategy, i.e.: that by which donor agencies contribute resources to programs only on the basis of local contributions towards the same goal, is a reasonable approach in the present case, and one that has long been a standard practice to promote sustainability of development efforts. When combined with receding amounts of external aid, this approach is supposed to further promote self-reliance of local institutions, as they would presumably see the advantage of contributing increasing amounts of resources to successful development programs, in order to maintain or even increase their benefits. 4.19 The latter means, therefore, that, for the matching grant approach to work, the programs themselves not only have to be successful, but also very successful in the eyes of the target institutions, which should highly appreciate their benefits; a problematic prospect in the cases of the Dominican political parties, some of whose most important leaders must see the success of this Program as the end of their way of doing business. Also, doubts have long been expressed (see, for instance the 2008 evaluation study) about the odds for success of a strategy of inducing the sustainability of the Program by diminishing external funding for it, in the hope that the political parties themselves will pick up the tab for future work. Skeptics point out that, if history with the Political Schools provides any indication, it is that when left to the parties’ own devices, this sort of education effort would fade away and disappear. 4.19 The present researchers have no indication that a decision has been made to secure sustainability of the present effort through eliciting local funding, or otherwise. However, should a diminishing matching grant strategy be adopted, we hereby discuss some ideas as to how to maximize its prospects for success. First, there is no apparent absence of recognition by the current political leadership of the benefits of political education, and both the statistical and circumstantial evidences available that there is indeed a budding process toward more generalized behavioral changes in the political parties induced by the Program are also recognized and appreciated by leaders inside the parties, as a desirable step going forward. The problem is that leaders who appreciate such advances for what they truly represent generally are those who are closely related to the parties’ own educational efforts, but who are not normally positioned close to the real decision making centers in the political parties. Strengthening the hand of these leaders would be required for matching grants to succeed. Second, the fact that there has been no matching grant approach to the funding of the present Program has deprived the effort of a way of testing in practice whether such approach may or may not further elicit the interest of leaders closer to the core decision centers. Third, although convincing senior leaders to funnel privately collected party funds towards education as local counterpart in matching grants would be very difficult, the situation may be different with public funds governed by law, as it is the case, for instance, of funds provided to the Political Parties by the Government, whose specific allocation, at least in part, to political education may be more feasible. Also, if resources provided by other donor agencies can be recognized as local counterparts in 61 matching grant schemes, this might as well provide a more secured way of earmarking resources for political education. Recommendation # 12: Try decreasing but substantial matching grants, requesting counterpart public funds allocated by law to education. Offering funds to be matched by public funds,that can be forcefully earmarked for use on militants’ education, may be a way of strengthening the hand of leaders inside the parties who are truly interested in education as a long term goal, vis-à-vis those leaders that may have different priorities for the use of available funds. Recommendation # 13: Explore possibilities of accepting other external resources as part of matching grant deals to expand the multiplication power of locally originated fund. Provided that locally originated counterpart funds are present, the acceptance of additional external funds from other possible donors as counterpart funds, within a political education Program with clearly agreed-on development objectives, may further leverage both the hand of political leaders truly interested in education and the sustainable impact of the Program itself. UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT DOMINICAN REPUBLIC PROGRAM FOR POLITICAL EDUCATION AND MANAGEMENT, 2 ND PHASE DEVELOPMENT EFFECTIVENESS EVALUATION STUDY STATISTICAL APPENDIX HUGO J. PIRELA, PH. D EMMANUEL SILVESTRE, PH.D. SANTO DOMINGO, D.R. NOVEMBER 2012 Statistical Appendix This Appendix includes the rationale and the raw tables from three different clusters of analysis: 1 st. The analysis of the databases of the surveys supplied by the Consortium to the population of the participants in the program for both groups, Treatment and Control, along the different years of the 2nd phase of the program: 2009, 2010 and 2011; 2 nd. The analysis of the Knowledge gain of the students of the Treatment group of the program based on the databases supplied by the Consortium, for the students of the three years of the program; 3 rd. The analysis of the Evaluation survey made by the evaluators on a sample of the participants from the Treatment and Control groups of the whole 2nd phase of the program. Statistical Tests Non-parametric analysis: For the analysis of the categorical dependent variables we used the statistic Chi-square from the Crosstab procedure of the SPSS, version 18. For additional non￾parametric analysis of the chi-squared we used the calculator found on Preacher, K. J. (2001, April). Calculation for the chi-square test: An interactive calculation tool for chi-square tests of goodness of fit and independence [Computer software]. Available from http://quantpsy.org Parametric analysis: For the analysis of the numeric data we used the following procedures: t test to compare means for independent samples and for paired samples from the SPSS procedures. ANOVA tests to compare more than two groups of means from the SPSS procedures. General Lineal Model to compare more than two groups with repeated measures with one or more factors from SPSS procedures. Effect size and Power: GPower software from Universitat Kiel, Germany, and How to Calculate Effect Sizes from Published Research: A Simplified Spreadsheet By Will Thalheimer and Samantha Cook. Section 1: N of Baselines and Surveys The evaluators planned first to use four Independent Variables in these analyses: Type of group (Treatment-Control), Sex (Male-Female), Type of organization (Political parties-OSC), and 6 Regions (Metropolitan-North-Northwest-South-East-Northeast) From the tables in of this Appendix we determined, through the Chi-square analysis, that the frequencies of the Type of organization and the Regions were not enough to sustain the proper statistical analysis of the data. So, we had to eliminate the Type of organization as an Independent Variable, as well as to add up the regions East and Northeast. The Chi-square analysis criteria to determine if the observed frequencies were enough to interpret correctly the analysis are the presence of less than 20% of the expected frequencies less than 5. So the definite Independent variables used in our analysis were three: Type of group (Treatment-Control), Sex (Male-Female), and 5 Regions (Metropolitan, North, Northwest, South, East-Northeast). Section 2: Sample Selection To select the sample for the Evaluation survey we used the SPSS procedure Complex Sample. First, we determined the N by Independent Variable (Table 1). Second, we calculate the percentages for those N (Table 2). Third, we calculate the sample quotas for those percentages (Table 3). The rest of the tables of this section contain the sample plan, summary and selection of the samples for each cohort and the total. Section 3: Comparison Between Groups Before running the statistical tests on the different databases we made several comparisons between the treatment and control group on different control variables as knowledge of and membership to networks, years in their organizations, years in actual position, monthly income, age, and type of organization, among others. The objective of these comparisons was to determine the comparability of both groups and the results are reported in the main paper. Section 4: Activities The data on the databases supplied by the Consortium were organized by cohorts. It is incorrect to add up the different baselines of the three years to compare them to the added surveys of the different years and yet to compare those added baselines and surveys with the closure survey, where all the participants of the three cohorts were mixed up. This procedure would not take account of the experimental mortality and would not permit us to determine the effect of the program on each cohort independently. Thus, we had to match the participants of each cohort who were present in both, the baseline survey of each year and the end survey of that year. Also we had to identify the participants on the closure survey that were present in both surveys of their respective cohort. This procedure permitted us to compare the effect of the time elapsed from the initial base line to the closure survey over the same subjects. In the main paper we reported the analyses that presented significant differences between the groups, sex and regions. In this Section 4 we report the tables obtained through the analysis made on different activities found in the databases as informs to the community, publication of bulletins, received political/social training, etc. The tables are organized by cohorts, by type of group, by sex, and by region. Section 5: Indicators Using the same design we analyzed the main indicators of the program and we report in this Section 5 the tables with the results on the Interparty participation, participation in elections, Increases in leadership level, Modernization proposals, Teaching of political courses, and the Accountability Index proportioned by the Consortium. These tables are also organized by cohort, by type of group, by sex, and by region. Section 6: Knowledge gain of the students of the Treatment group In the subsection Exams Distributions we report the tables and figures resulting from the analysis of the normality of the distributions of the test scores from the Entry Exam, the Final Exam and the Total score for the cohorts of 2009, 2010 and 2011. In the rest of this Section 6 we report the tables of the analysis of the knowledge gain of the students through Indexes 1 and 2. Index 1 is the comparison between the scores on the Entry Exam and the scores on the Final Exam. Index 2 is the comparison between the scores on the Entry Exam and the scores on the Total scale. For both Indexes it is incorrect to compare the global mean on the first measure (Entry Exam) with the global mean on the second measure (Final Exam or Total). The correct statistical test in both cases compares the gain of the same student from the first measure to the second one. To determine the knowledge gain of the students we run a General Lineal Model ANOVA for repeated measures, including the Within Subject Contrasts for the repeated measures, as well as the Between Subject Effects for the Sex and Region factors. In this Section 6 we can find the tables with the results of the said analysis, as well as analysis showing a higher correlation between the Entry Exam and the Final Exam. Section 7: Evaluation Survey In this section we report the tables resulting from the quantitative analysis of the closed question of the evaluation Survey. The sums of Promotions, Reform Proposals, Reform Approvals, Promotion of Trainings, Approval of Training Proposals, and Won Elections, were analyzed with a two way ANOVA using Type of Group and Sex as independent variables. The categorical data on Participation in elections, Participation in discussions, inform the community, and the like, were analyzed with the Crosstabulation procedure using also Type of Group and Sex as independent variables. In the annexed Excel file we provide a complete List of the Files in this appendix. Section 1: N of Baselines and Surveys N Baseline 2009 Region * Sex * Organization * Group Crosstabulation Count Group Organization Sex Total Male Female Treatment Political Party Region Metropolitan 23 18 41 North 21 16 37 Northwest 18 21 39 Northeast 18 19 37 South 21 12 33 East 22 16 38 Total 123 102 225 Organization region Metropolitan 3 5 8 North 3 6 9 Northwest 2 1 3 Northeast 4 2 6 South 4 7 11 East 2 3 5 Total 18 24 42 Control Political Party Region Metropolitan 68 15 83 North 8 3 11 Northwest 7 9 16 Northeast 1 0 1 South 15 9 24 East 31 7 38 Total 130 43 173 Organization region Metropolitan 16 5 21 North 4 5 9 South 2 1 3 East 2 3 5 Total 24 14 38 Chi-Square Tests Group Organization Value df Asymp. Sig. (2-sided) Treatment Political Party Pearson Chi-Square 3.011a 5 .698 Likelihood Ratio 3.021 5 .697 Linear-by-Linear Association .177 1 .674 N of Valid Cases 225 Organization Pearson Chi-Square 2.716b 5 .744 Likelihood Ratio 2.714 5 .744 Linear-by-Linear Association .051 1 .821 N of Valid Cases 42 Control Political Party Pearson Chi-Square 13.750c 5 .017 Likelihood Ratio 12.701 5 .026 Linear-by-Linear Association .613 1 .434 N of Valid Cases 173 Organization Pearson Chi-Square 4.056d 3 .256 Likelihood Ratio 4.049 3 .256 Linear-by-Linear Association 1.648 1 .199 N of Valid Cases 38 a. 0 cells ( 0.0% ) have expected count less than 5. b. 10 cells ( 83.3% ) have expected count less than 5. c. 4 cells ( 33.3% ) have expected count less than 5. d. 5 cells ( 62.5% ) have expected count less than 5. N Survey 2009 REGION * SEXO * Tipo de organización * Tipo de grupo Crosstabulation Count Group Organization Sex Total Male Female Treatment Political Party Region Metropolitan 9 14 23 North 8 5 13 Northwest 11 7 18 Northeast 10 7 17 South 9 9 18 East 8 6 14 Total 55 48 103 Organization region Metropolitan 0 1 1 North 4 3 7 Northwest 1 1 2 Northeast 0 2 2 South 1 1 2 East 2 3 5 Total 8 11 19 Control Organization region Metropolitan 39 7 46 North 5 3 8 Northwest 2 5 7 South 8 4 12 East 17 4 21 Total 71 23 94 OSC REGION Metropolitana 10 5 15 Norte 3 2 5 Noroeste 1 0 1 South 3 1 4 East 2 2 4 Total 19 10 29 Chi-Square Tests Group Organization Value df Asymp. Sig. (2-sided) Treatment Political Party Pearson Chi-Square 3.022a 5 .697 Likelihood Ratio 3.033 5 .695 Linear-by-Linear Association .594 1 .441 N of Valid Cases 103 Organization Pearson Chi-Square 2.943b 5 .709 Likelihood Ratio 4.028 5 .545 Linear-by-Linear Association .074 1 .786 N of Valid Cases 19 Control Political Party Pearson Chi-Square 12.062c 4 .017 Likelihood Ratio 10.684 4 .030 Linear-by-Linear Association .789 1 .374 N of Valid Cases 94 Organization Pearson Chi-Square 1.188d 4 .880 Likelihood Ratio 1.493 4 .828 Linear-by-Linear Association .055 1 .814 N of Valid Cases 29 a. 0 cells ( 0.0% ) have expected count less than 5. b. 12 cells ( 100.0% ) have expected count less than 5. c. 3 cells ( 30.0% ) have expected count less than 5. d. 8 cells ( 80.0% ) have expected count less than 5. N Baseline 2010 Male Female Metropolitan 29 15 44 North 27 17 44 Northwest 29 16 45 South 24 16 40 East 19 19 38 Northeast 21 11 32 149 94 243 Metropolitan 3 3 6 North 3 2 5 Northwest 1 4 5 South 3 2 5 East 2 2 4 Northeast 4 5 9 16 18 34 Metropolitan 46 26 72 North 32 8 40 Northwest 17 8 25 South 8 6 14 East 3 7 10 106 55 161 Metropolitan 12 1 13 North 2 2 4 South 6 9 15 East 2 1 3 Northeast 1 3 4 23 16 39 Control Political Party REGION Total OSC REGION Total Treatment Political Party REGION Total OSC REGION Total REGION * Sex * Type of organization * Type of group Crosstabulation Count Type of group Type of organization Sex Total Type of group Value df Asymp. Sig. (2-sided) Pearson Chi-Square 2.908a 5 .714 Likelihood Ratio 2.867 5 .720 Linear-by-Linear Association .511 1 .475 N of Valid Cases 243 Pearson Chi-Square 2.201b 5 .821 Likelihood Ratio 2.324 5 .803 Linear-by-Linear Association .023 1 .880 N of Valid Cases 34 Pearson Chi-Square 9.922c 4 .042 Likelihood Ratio 9.858 4 .043 Linear-by-Linear Association 2.305 1 .129 N of Valid Cases 161 Pearson Chi-Square 10.317d 4 .035 Likelihood Ratio 11.698 4 .020 Linear-by-Linear Association 7.396 1 .007 N of Valid Cases 39 a. 0 cells ( 0.0% ) have expected count less than 5. b. 12 cells ( 100.0% ) have expected count less than 5. c. 2 cells ( 20.0% ) have expected count less than 5. d. 6 cells ( 60.0% ) have expected count less than 5. Chi-Square Tests Type of organization Treatment Political Party OSC Control Political Party OSC N Survey 2010 Male Female Metropolitan 14 5 19 North 14 7 21 Northwest 14 7 21 South 11 8 19 East 6 10 16 Northeast 11 6 17 70 43 113 Metropolitan 1 1 2 North 0 1 1 South 2 0 2 East 2 1 3 Northeast 3 0 3 8 3 11 Metropolitan 28 16 44 North 23 4 27 Northwest 11 5 16 South 4 2 6 East 1 5 6 67 32 99 Metropolitan 9 0 9 North 1 0 1 South 4 8 12 East 2 0 2 Northeast 1 1 2 17 9 26 Control Political Party Region Total OSC Region Total Treatment Political Party Region Total OSC Region Total Region * Sex * Type of organization * Type of group Crosstabulation Count Type of group Type of organization Sex Total Type of group Value df Asymp. Sig. (2-sided) Pearson Chi-Square 5.751a 5 .331 Likelihood Ratio 5.661 5 .341 Linear-by-Linear Association 2.121 1 .145 N of Valid Cases 113 Pearson Chi-Square 5.118b 4 .275 Likelihood Ratio 6.299 4 .178 Linear-by-Linear Association 2.267 1 .132 N of Valid Cases 11 Pearson Chi-Square 11.260c 4 .024 Likelihood Ratio 11.344 4 .023 Linear-by-Linear Association 1.774 1 .183 N of Valid Cases 99 Pearson Chi-Square 12.009d 4 .017 Likelihood Ratio 15.493 4 .004 Linear-by-Linear Association 5.647 1 .017 N of Valid Cases 26 a. 0 cells ( 0.0% ) have expected count less than 5. b. 10 cells ( 100.0% ) have expected count less than 5. c. 4 cells ( 40.0% ) have expected count less than 5. d. 8 cells ( 80.0% ) have expected count less than 5. Chi-Square Tests Type of organization Treatment Political Party OSC Control Political Party OSC N Baseline 2011 Male Female Metropolitan 24 22 46 North 27 18 45 Northwest 19 26 45 South 21 21 42 East 21 25 46 Northeast 19 23 42 131 135 266 Metropolitan 3 1 4 North 4 1 5 Northwest 3 3 6 South 2 3 5 East 2 2 4 Northeast 3 2 5 17 12 29 Metropolitan 29 12 41 North 10 2 12 Northwest 17 12 29 South 17 18 35 East 9 15 24 82 59 141 Metropolitan 14 7 21 North 9 11 20 Northwest 3 3 6 South 4 5 9 East 2 9 11 Northeast 0 8 8 32 43 75 Control Political Party Region Total OSC Region Total Treatment Political Party Region Total OSC Region Total Region * Sex * Type of organization * Type of group Crosstabulation Count Type of group Type of organization Sex Total Type of group Value df Asymp. Sig. (2-sided) Pearson Chi-Square 3.645a 5 .602 Likelihood Ratio 3.662 5 .599 Linear-by-Linear Association 1.246 1 .264 N of Valid Cases 266 Pearson Chi-Square 2.410b 5 .790 Likelihood Ratio 2.510 5 .775 Linear-by-Linear Association .723 1 .395 N of Valid Cases 29 Pearson Chi-Square 11.322c 4 .023 Likelihood Ratio 11.731 4 .019 Linear-by-Linear Association 9.432 1 .002 N of Valid Cases 141 Pearson Chi-Square 13.782d 5 .017 Likelihood Ratio 16.980 5 .005 Linear-by-Linear Association 11.953 1 .001 N of Valid Cases 75 a. 0 cells ( 0.0% ) have expected count less than 5. b. 12 cells ( 100.0% ) have expected count less than 5. c. 0 cells ( 0.0% ) have expected count less than 5. d. 6 cells ( 50.0% ) have expected count less than 5. Chi-Square Tests Type of organization Treatment Political Party OSC Control Political Party OSC N Closure Survey Male Female Metropolitan 34 21 55 North 22 12 34 Northwest 20 19 39 South 28 25 53 East 30 30 60 Northeast 26 23 49 160 130 290 Metropolitan 3 2 5 North 4 2 6 Northwest 1 2 3 South 3 3 6 East 4 2 6 Northeast 6 0 6 21 11 32 Metropolitan 49 28 77 North 15 6 21 Northwest 8 12 20 South 18 15 33 East 19 10 29 109 71 180 Metropolitan 23 8 31 North 7 6 13 Northwest 2 1 3 South 7 13 20 East 3 10 13 Northeast 0 3 3 42 41 83 Control Political Party Region Total OSC Region Total Treatment Political Party Region Total OSC Region Total Region * Sex * Type of organization * Type of group Crosstabulation Count Type of group Type of organization Sex Total Type of group Value df Asymp. Sig. (2-sided) Pearson Chi-Square 3.325a 5 .650 Likelihood Ratio 3.357 5 .645 Linear-by-Linear Association 2.036 1 .154 N of Valid Cases 290 Pearson Chi-Square 5.255b 5 .386 Likelihood Ratio 7.040 5 .218 Linear-by-Linear Association 1.520 1 .218 N of Valid Cases 32 Pearson Chi-Square 5.681c 4 .224 Likelihood Ratio 5.621 4 .229 Linear-by-Linear Association .357 1 .550 N of Valid Cases 180 Pearson Chi-Square 16.228d 5 .006 Likelihood Ratio 17.940 5 .003 Linear-by-Linear Association 15.264 1 .000 N of Valid Cases 83 a. 0 cells ( 0.0% ) have expected count less than 5. b. 12 cells ( 100.0% ) have expected count less than 5. c. 0 cells ( 0.0% ) have expected count less than 5. d. 4 cells ( 33.3% ) have expected count less than 5. Chi-Square Tests Type of organization Treatment Political Party OSC Control Political Party OSC N Closure Survey without Organization Male Female Metropolitan 37 23 60 North 26 14 40 Northwest 21 21 42 South 31 28 59 East 34 32 66 Northeast 32 23 55 181 141 322 Metropolitan 72 36 108 North 22 12 34 Northwest 10 13 23 South 25 28 53 East 22 20 42 Northeast 0 3 3 151 112 263 Control Region Total Region * Sex * Type of group Crosstabulation Count Type of group Sex Total Treatment Region Total Value df Asymp. Sig. (2- sided) Pearson Chi-Square 3.640a 5 .602 Likelihood Ratio 3.664 5 .599 Linear-by-Linear Association .953 1 .329 N of Valid Cases 322 Pearson Chi-Square 13.102b 5 .022 Likelihood Ratio 14.230 5 .014 Linear-by-Linear Association 8.420 1 .004 N of Valid Cases 263 a. 0 cells ( 0.0% ) have expected count less than 5. b. 2 cells ( 16.7% ) have expected count less than 5. Control Chi-Square Tests Type of group Treatment N Baseline 2009 6 Regions Male Female Count 11 11 22 % within Region 50.0% 50.0% 100.0% % within Sex 16.2% 23.4% 19.1% Count 13 6 19 % within Region 68.4% 31.6% 100.0% % within Sex 19.1% 12.8% 16.5% Count 12 7 19 % within Region 63.2% 36.8% 100.0% % within Sex 17.6% 14.9% 16.5% Count 11 8 19 % within Region 57.9% 42.1% 100.0% % within Sex 16.2% 17.0% 16.5% Count 10 9 19 % within Region 52.6% 47.4% 100.0% % within Sex 14.7% 19.1% 16.5% Count 11 6 17 % within Region 64.7% 35.3% 100.0% % within Sex 16.2% 12.8% 14.8% Count 68 47 115 % within Region 59.1% 40.9% 100.0% % within Sex 100.0% 100.0% 100.0% Count 45 11 56 % within Region 80.4% 19.6% 100.0% % within Sex 54.9% 40.7% 51.4% Count 7 5 12 % within Region 58.3% 41.7% 100.0% % within Sex 8.5% 18.5% 11.0% Count 3 4 7 % within Region 42.9% 57.1% 100.0% % within Sex 3.7% 14.8% 6.4% Count 10 3 13 % within Region 76.9% 23.1% 100.0% % within Sex 12.2% 11.1% 11.9% Count 17 4 21 % within Region 81.0% 19.0% 100.0% % within Sex 20.7% 14.8% 19.3% Count 82 27 109 % within Region 75.2% 24.8% 100.0% % within Sex 100.0% 100.0% 100.0% Total East Northeast Total Control Region Metropolitan North Northwest South East Treatment Region Metropolitan North Northwest South Region * Sex * Type of group Crosstabulation Type of group Sex Total Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 2.128a 5 .831 Likelihood Ratio 2.137 5 .830 Linear-by￾Linear Association .070 1 .792 N of Valid Cases 115 Pearson Chi￾Square 6.954b 4 .138 Likelihood Ratio 6.194 4 .185 Linear-by￾Linear Association .022 1 .882 N of Valid Cases 109 a. 0 cell ( 0.0% ) have expected count less than 5 c. 3 cell ( 30.0% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control N Baseline 2009 5 Regions Male Female Count 11 11 22 % within Región X 50.0% 50.0% 100.0% % within Sex 16.2% 23.4% 19.1% Count 13 6 19 % within Región X 68.4% 31.6% 100.0% % within Sex 19.1% 12.8% 16.5% Count 12 7 19 % within Región X 63.2% 36.8% 100.0% % within Sex 17.6% 14.9% 16.5% Count 11 8 19 % within Región X 57.9% 42.1% 100.0% % within Sex 16.2% 17.0% 16.5% Count 21 15 36 % within Región X 58.3% 41.7% 100.0% % within Sex 30.9% 31.9% 31.3% Count 68 47 115 % within Región X 59.1% 40.9% 100.0% % within Sex 100.0% 100.0% 100.0% Count 45 11 56 % within Región X 80.4% 19.6% 100.0% % within Sex 54.9% 40.7% 51.4% Count 7 5 12 % within Región X 58.3% 41.7% 100.0% % within Sex 8.5% 18.5% 11.0% Count 3 4 7 % within Región X 42.9% 57.1% 100.0% % within Sex 3.7% 14.8% 6.4% Count 10 3 13 % within Región X 76.9% 23.1% 100.0% % within Sex 12.2% 11.1% 11.9% Count 17 4 21 % within Región X 81.0% 19.0% 100.0% % within Sex 20.7% 14.8% 19.3% Count 82 27 109 % within Región X 75.2% 24.8% 100.0% % within Sex 100.0% 100.0% 100.0% Total Control Región X Metropolitan North Northwest South East￾Northeast Total Type of group Sex Total Treatment Región X Metropolitan North Northwest South East￾Northeast Región X * Sex * Type of group Crosstabulation Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 1.587a 4 .811 Likelihood Ratio 1.596 4 .809 Linear-by￾Linear Association .032 1 .857 N of Valid Cases 115 Pearson Chi￾Square 6.954b 4 .138 Likelihood Ratio 6.194 4 .185 Linear-by￾Linear Association .022 1 .882 N of Valid Cases 109 a. 0 cell ( 0.0% ) have expected count less than 5 c. 3 cell ( 30.0% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control N Baseline 2010, 6 Regions Male Female Count 11 8 19 % within Region 57.9% 42.1% 100.0% % within Sex 17.5% 13.3% 15.4% Count 12 6 18 % within Region 66.7% 33.3% 100.0% % within Sex 19.0% 10.0% 14.6% Count 10 12 22 % within Region 45.5% 54.5% 100.0% % within Sex 15.9% 20.0% 17.9% Count 9 11 20 % within Region 45.0% 55.0% 100.0% % within Sex 14.3% 18.3% 16.3% Count 12 13 25 % within Region 48.0% 52.0% 100.0% % within Sex 19.0% 21.7% 20.3% Count 9 10 19 % within Region 47.4% 52.6% 100.0% % within Sex 14.3% 16.7% 15.4% Count 63 60 123 % within Region 51.2% 48.8% 100.0% % within Sex 100.0% 100.0% 100.0% Count 30 16 46 % within Region 65.2% 34.8% 100.0% % within Sex 41.1% 25.8% 34.1% Count 9 7 16 % within Region 56.3% 43.8% 100.0% % within Sex 12.3% 11.3% 11.9% Count 10 7 17 % within Region 58.8% 41.2% 100.0% % within Sex 13.7% 11.3% 12.6% Count 15 13 28 % within Region 53.6% 46.4% 100.0% % within Sex 20.5% 21.0% 20.7% Count 9 17 26 % within Region 34.6% 65.4% 100.0% % within Sex 12.3% 27.4% 19.3% Count 0 2 2 % within Region .0% 100.0% 100.0% % within Sex .0% 3.2% 1.5% Count 73 62 135 % within Region 54.1% 45.9% 100.0% % within Sex 100.0% 100.0% 100.0% Control Region Metropolitan North Northwest South East Northeast Total Treatment Region Metropolitan North Northwest South East Northeast Total Region * Sex * Type of group Crosstabulation Type of group Sex Total Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 2.877a 5 .719 Likelihood Ratio 2.916 5 .713 Linear-by￾Linear Association 1.240 1 .265 N of Valid Cases 123 Pearson Chi￾Square 8.807b 5 .117 Likelihood Ratio 9.632 5 .086 Linear-by￾Linear Association 6.703 1 .010 N of Valid Cases 135 a. 0 cell ( 0.0% ) have expected count less than 5 c. 2 cell ( 16.7% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control N Baseline 2010, 5 Regions Male Female Count 11 8 19 % within Región X 57.9% 42.1% 100.0% % within Sex 17.5% 13.3% 15.4% Count 12 6 18 % within Región X 66.7% 33.3% 100.0% % within Sex 19.0% 10.0% 14.6% Count 10 12 22 % within Región X 45.5% 54.5% 100.0% % within Sex 15.9% 20.0% 17.9% Count 9 11 20 % within Región X 45.0% 55.0% 100.0% % within Sex 14.3% 18.3% 16.3% Count 21 23 44 % within Región X 47.7% 52.3% 100.0% % within Sex 33.3% 38.3% 35.8% Count 63 60 123 % within Región X 51.2% 48.8% 100.0% % within Sex 100.0% 100.0% 100.0% Count 30 16 46 % within Región X 65.2% 34.8% 100.0% % within Sex 41.1% 25.8% 34.1% Count 9 7 16 % within Región X 56.3% 43.8% 100.0% % within Sex 12.3% 11.3% 11.9% Count 10 7 17 % within Región X 58.8% 41.2% 100.0% % within Sex 13.7% 11.3% 12.6% Count 15 13 28 % within Región X 53.6% 46.4% 100.0% % within Sex 20.5% 21.0% 20.7% Count 9 19 28 % within Región X 32.1% 67.9% 100.0% % within Sex 12.3% 30.6% 20.7% Count 73 62 135 % within Región X 54.1% 45.9% 100.0% % within Sex 100.0% 100.0% 100.0% Control Región X Metropolitan North Northwest South East￾Northeast Total Treatment Región X Metropolitan North Northwest South East￾Northeast Total Región X * Sex * Type of group Crosstabulation Type of group Sex Total Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 2.875a 4 .579 Likelihood Ratio 2.915 4 .572 Linear-by￾Linear Association 1.375 1 .241 N of Valid Cases 123 Pearson Chi￾Square 7.911b 4 .095 Likelihood Ratio 8.009 4 .091 Linear-by￾Linear Association 6.310 1 .012 N of Valid Cases 135 a. 0 cell ( 0.0% ) have expected count less than 5 c. 0 cell ( 0.0% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control N Baseline 2011, 6 Regions Male Female Count 11 8 19 % within Region 57.9% 42.1% 100.0% % within Sex 17.5% 13.3% 15.4% Count 12 6 18 % within Region 66.7% 33.3% 100.0% % within Sex 19.0% 10.0% 14.6% Count 10 12 22 % within Region 45.5% 54.5% 100.0% % within Sex 15.9% 20.0% 17.9% Count 9 11 20 % within Region 45.0% 55.0% 100.0% % within Sex 14.3% 18.3% 16.3% Count 12 13 25 % within Region 48.0% 52.0% 100.0% % within Sex 19.0% 21.7% 20.3% Count 9 10 19 % within Region 47.4% 52.6% 100.0% % within Sex 14.3% 16.7% 15.4% Count 63 60 123 % within Region 51.2% 48.8% 100.0% % within Sex 100.0% 100.0% 100.0% Count 30 16 46 % within Region 65.2% 34.8% 100.0% % within Sex 41.1% 25.8% 34.1% Count 9 7 16 % within Region 56.3% 43.8% 100.0% % within Sex 12.3% 11.3% 11.9% Count 10 7 17 % within Region 58.8% 41.2% 100.0% % within Sex 13.7% 11.3% 12.6% Count 15 13 28 % within Region 53.6% 46.4% 100.0% % within Sex 20.5% 21.0% 20.7% Count 9 17 26 % within Region 34.6% 65.4% 100.0% % within Sex 12.3% 27.4% 19.3% Count 0 2 2 % within Region .0% 100.0% 100.0% % within Sex .0% 3.2% 1.5% Count 73 62 135 % within Region 54.1% 45.9% 100.0% % within Sex 100.0% 100.0% 100.0% Control Region Metropolitan North Northwest South East Northeast Total Treatment Region Metropolitan North Northwest South East Northeast Total Region * Sex * Type of group Crosstabulation Type of group Sex Total Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 2.877a 5 .719 Likelihood Ratio 2.916 5 .713 Linear-by￾Linear Association 1.240 1 .265 N of Valid Cases 123 Pearson Chi￾Square 8.807b 5 .117 Likelihood Ratio 9.632 5 .086 Linear-by￾Linear Association 6.703 1 .010 N of Valid Cases 135 a. 0 cell ( 0.0% ) have expected count less than 5 c. 2 cell ( 16.7% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control N Baseline 2011, 5 Regions Male Female Count 11 8 19 % within Region X 57.9% 42.1% 100.0% % within Sex 17.5% 13.3% 15.4% Count 12 6 18 % within Region X 66.7% 33.3% 100.0% % within Sex 19.0% 10.0% 14.6% Count 10 12 22 % within Region X 45.5% 54.5% 100.0% % within Sex 15.9% 20.0% 17.9% Count 9 11 20 % within Region X 45.0% 55.0% 100.0% % within Sex 14.3% 18.3% 16.3% Count 21 23 44 % within Region X 47.7% 52.3% 100.0% % within Sex 33.3% 38.3% 35.8% Count 63 60 123 % within Region X 51.2% 48.8% 100.0% % within Sex 100.0% 100.0% 100.0% Count 30 16 46 % within Region X 65.2% 34.8% 100.0% % within Sex 41.1% 25.8% 34.1% Count 9 7 16 % within Region X 56.3% 43.8% 100.0% % within Sex 12.3% 11.3% 11.9% Count 10 7 17 % within Region X 58.8% 41.2% 100.0% % within Sex 13.7% 11.3% 12.6% Count 15 13 28 % within Region X 53.6% 46.4% 100.0% % within Sex 20.5% 21.0% 20.7% Count 9 19 28 % within Region X 32.1% 67.9% 100.0% % within Sex 12.3% 30.6% 20.7% Count 73 62 135 % within Region X 54.1% 45.9% 100.0% % within Sex 100.0% 100.0% 100.0% Control Region X Metropolitan North Northwest South East￾Northwest Total Treatment Region X Metropolitan North Northwest South East￾Northwest Total Region X * Sex * Type of group Crosstabulation Type of group Sex Total Value df Asymp. Sig. (2- sided) Pearson Chi￾Square 2.875a 4 .579 Likelihood Ratio 2.915 4 .572 Linear-by￾Linear Association 1.375 1 .241 N of Valid Cases 123 Pearson Chi￾Square 7.911b 4 .095 Likelihood Ratio 8.009 4 .091 Linear-by￾Linear Association 6.310 1 .012 N of Valid Cases 135 a. 0 cell ( 0.0% ) have expected count less than 5 c. 0 cell ( 0.0% ) have expected count less than 5 Chi-Square Tests Type of group Treatment Control 2. Sample Selection Quotas TABLE 1: N Base 2009 Base 2010 Base 2011 Type of group Sex Total Sex Total Sex Total Male Female Male Female Male Female Treatment Region Metropolitan 12 11 23 15 6 21 8 8 16 North 14 6 20 14 9 23 8 4 12 Northwest 12 7 19 14 6 20 10 11 21 South 11 8 19 13 8 21 9 11 20 East-Northeast 21 15 36 22 17 39 17 22 39 Total 70 47 117 78 46 124 52 56 108 Control Region Metropolitan 46 11 57 37 16 53 26 10 36 North 7 5 12 24 3 27 7 4 11 Northwest 3 4 7 11 5 16 10 6 16 South 11 5 16 8 10 18 12 10 22 East-Northeast 18 4 22 4 6 10 9 18 27 Total 85 29 114 84 40 124 64 48 112 TABLE 2: Percentages Base 2009 Base 2010 Base 2011 Type of group Sex Total Sex Total Sex Total Male Female Male Female Male Female Treatment Region Metropolitan 1.7% 1.6% 3.3% 2.1% 0.9% 3.0% 1.1% 1.1% 2.3% North 2.0% 0.9% 2.9% 2.0% 1.3% 3.3% 1.1% 0.6% 1.7% Northwest 1.7% 1.0% 2.7% 2.0% 0.9% 2.9% 1.4% 1.6% 3.0% South 1.6% 1.1% 2.7% 1.9% 1.1% 3.0% 1.3% 1.6% 2.9% East-Northeast 3.0% 2.1% 5.2% 3.1% 2.4% 5.6% 2.4% 3.1% 5.6% Total 10.0% 6.7% 16.7% 11.2% 6.6% 17.7% 7.4% 8.0% 15.5% Control Region Metropolitan 6.6% 1.6% 8.2% 5.3% 2.3% 7.6% 3.7% 1.4% 5.2% North 1.0% 0.7% 1.7% 3.4% 0.4% 3.9% 1.0% 0.6% 1.6% Northwest 0.4% 0.6% 1.0% 1.6% 0.7% 2.3% 1.4% 0.9% 2.3% South 1.6% 0.7% 2.3% 1.1% 1.4% 2.6% 1.7% 1.4% 3.1% East-Northeast 2.6% 0.6% 3.1% 0.6% 0.9% 1.4% 1.3% 2.6% 3.9% Total 12.2% 4.1% 16.3% 12.0% 5.7% 17.7% 9.2% 6.9% 16.0% TABLA 3: Sample Quotas Base 2009 Base 2010 Base 2011 Type of group Sex Total Sex Total Sex Total Male Female Male Female Male Female Treatment Region Metropolitan 4 4 8 5 2 7 3 3 5 North 5 2 7 5 3 8 3 1 4 Northwest 4 2 6 5 2 7 3 4 7 South 4 3 6 4 3 7 3 4 7 East-Northeast 7 5 12 7 6 13 6 7 13 Total 23 15 38 26 15 41 17 18 36 Control Region Metropolitan 15 4 19 12 5 17 9 3 12 North 2 2 4 8 1 9 2 1 4 Northwest 1 1 2 4 2 5 3 2 5 South 4 2 5 3 3 6 4 3 7 East-Northeast 6 1 7 1 2 3 3 6 9 Total 28 10 38 28 13 41 21 16 37 Complex Samples: Plan Summary Stage 1 Stage 2 Design Variables Stratification 1 Tipo de grupo Región X Cluster 1 sexo Sample Information Selection Method Sample SIMPLE_WOR Sample SIMPLE_WOR Number of Units Sampled 115 Obtained from matrix specification Variables Created or Modified Stagewise Inclusion (Selection) Probability InclusionProbability_1_ InclusionProbability_2_ Stagewise Cumulative Sample Weight SampleWeightCumulative_1_ SampleWeightCumulative_2_ Stagewise Population Size PopulationSize_1_ PopulationSize_2_ Stagewise Sample Size SampleSize_1_ SampleSize_2_ Stagewise Sampling Rate SamplingRate_1_ SamplingRate_2_ Stagewise Sample Weight SampleWeight_1_ SampleWeight_2_ Analysis Information Estimator Assumption Equal probability sampling without replacement Equal probability sampling without replacement Inclusion Probability Obtained from variable InclusionProbability_1_ Obtained from variable InclusionProbability_2_ Complex Samples: Selection 2009 Requested Actual Requested Actual Metropolitan 4 4 33.3% 33.3% North 5 5 35.7% 35.7% Northeast 4 4 33.3% 33.3% South 4 4 36.4% 36.4% East-Northeast 7 7 33.3% 33.3% Metropolitan 4 4 36.4% 36.4% North 2 2 33.3% 33.3% Northeast 2 2 28.6% 28.6% South 3 3 37.5% 37.5% East-Northeast 5 5 33.3% 33.3% Metropolitan 15 15 32.6% 32.6% North 2 2 28.6% 28.6% Northeast 1 1 33.3% 33.3% South 4 4 36.4% 36.4% East-Northeast 6 6 33.3% 33.3% Metropolitan 4 4 36.4% 36.4% North 2 2 40.0% 40.0% Northeast 1 1 25.0% 25.0% South 2 2 40.0% 40.0% East-Northeast 1 1 25.0% 25.0% Summary for Stage 2 Type of group sex Region X Number of Units Sampled Proportion of Units Sampled Treatment Male Female Control Male Female 2010 Requested Actual Requested Actual Metropolitan 5 5 31.3% 31.3% North 6 6 33.3% 33.3% Northeast 4 4 36.4% 36.4% South 5 5 33.3% 33.3% East￾Northeast 6 6 33.3% 33.3% Metropolitan 2 2 33.3% 33.3% North 3 3 33.3% 33.3% Northeast 2 2 33.3% 33.3% South 5 5 35.7% 35.7% East￾Northeast 4 4 36.4% 36.4% Metropolitan 12 12 33.3% 33.3% North 6 6 33.3% 33.3% Northeast 6 6 35.3% 35.3% South 1 1 25.0% 25.0% East￾Northeast 3 3 33.3% 33.3% Metropolitan 5 5 31.3% 31.3% North 2 2 40.0% 40.0% Northeast 1 1 50.0% 50.0% South 2 2 33.3% 33.3% East￾Northeast 4 4 36.4% 36.4% Summary for Stage 2 Type of group sex REGION X Number of Units Sampled Proportion of Units Sampled Treatment Male Female Control Male Female 2011 Requested Actual Requested Actual Metropolitan 3 3 37.5% 37.5% North 3 3 37.5% 37.5% Northeast 3 3 30.0% 30.0% South 3 3 33.3% 33.3% East￾Northeast 6 6 35.3% 35.3% Metropolitan 3 3 37.5% 37.5% North 1 1 25.0% 25.0% Northeast 4 4 36.4% 36.4% South 4 4 36.4% 36.4% East￾Northeast 7 7 31.8% 31.8% Metropolitan 9 9 34.6% 34.6% North 2 2 28.6% 28.6% Northeast 3 3 30.0% 30.0% South 4 4 33.3% 33.3% East￾Northeast 3 3 33.3% 33.3% Metropolitan 3 3 30.0% 30.0% North 1 1 25.0% 25.0% Northeast 2 2 33.3% 33.3% South 3 3 30.0% 30.0% East￾Northeast 6 6 33.3% 33.3% Summary for Stage 2 Type of group sex Region X Number of Units Sampled Proportion of Units Sampled Treatment Male Female Control Male Female Total Requested Actual Requested Actual Metropolitan 12 12 34.0% 34.0% North 14 14 35.5% 35.5% Northeast 11 11 33.2% 33.2% South 12 12 34.3% 34.3% East￾Northeast 19 19 34.0% 34.0% Metropolitan 9 9 35.7% 35.7% North 6 6 30.6% 30.6% Northeast 8 8 32.8% 32.8% South 12 12 36.5% 36.5% East￾Northeast 16 16 33.8% 33.8% Metropolitan 36 36 33.5% 33.5% North 10 10 30.2% 30.2% Northeast 10 10 32.9% 32.9% South 9 9 31.6% 31.6% East￾Northeast 12 12 33.3% 33.3% Metropolitan 12 12 32.5% 32.5% North 5 5 35.0% 35.0% Northeast 4 4 36.1% 36.1% South 7 7 34.4% 34.4% East￾Northeast 11 11 31.6% 31.6% Summary for Stage 2 Type of group sex Region X Number of Units Sampled Proportion of Units Sampled Treatment Male Female Control Male Female 3. Comparisons Between Groups Type of group Treatment Control Treatment Control Treatment Control Yes Count 67 69 74 52 71 34 % within IV 49.3% 50.7% 58.7% 41.3% 67.6% 32.4% % within DV 58.3% 63.3% 64.3% 47.7% 84.5% 63.0% No Count 46 36 41 57 13 18 % within IV 56.1% 43.9% 41.8% 58.2% 41.9% 58.1% % within DV 40.0% 33.0% 35.7% 52.3% 15.5% 33.3% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Knowledge of the existence of political/social networks Base 2009 Survey 2009 Closure 2009 Treatment Control Treatment Control Treatment Control Yes Count 63 67 102 77 94 45 % within IV 48.5% 51.5% 57.0% 43.0% 67.6% 32.4% % within DV 51.6% 53.6% 82.9% 61.6% 81.7% 57.7% No Count 59 58 21 48 19 32 % within IV 50.4% 49.6% 30.4% 69.6% 37.3% 62.7% % within DV 48.4% 46.4% 17.1% 38.4% 16.5% 41.0% No response Count 0 0 0 0 2 1 % within IV .0% .0% .0% .0% 66.7% 33.3% % within DV .0% .0% .0% .0% 1.7% 1.3% Total Count 122 125 123 125 115 78 % within IV 49.4% 50.6% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Knowledge of the existence of political/social networks Base 2010 Survey 2010 Closure 2010 Yes No Yes No Treatment Count 57 66 96 27 % within IV 46.3% 53.7% 77.4% 21.8% % within DV 42.9% 52.8% 56.5% 33.3% Control Count 76 59 74 54 % within IV 56.3% 43.7% 55.6% 40.6% % within DV 57.1% 47.2% 43.5% 66.7% Total Count 133 125 170 81 % within IV 51.6% 48.4% 66.1% 31.5% % within DV 100.0% 100.0% 100.0% 100.0% Knowledge of the existence of political/social networks Base 2011 Closure 2011 Treatment Control Treatment Control Treatment Control Yes Count 65 57 49 33 54 16 % within IV 53.3% 46.7% 59.8% 40.2% 77.1% 22.9% % within DV 56.5% 52.3% 59.8% 58.9% 77.1% 47.1% No Count 8 11 33 23 16 18 % within IV 42.1% 57.9% 58.9% 41.1% 47.1% 52.9% % within DV 7.0% 10.1% 40.2% 41.1% 22.9% 52.9% N/R Count 42 41 0 0 0 0 % within IV 50.6% 49.4% .0% .0% .0% .0% % within DV 36.5% 37.6% .0% .0% .0% .0% Total Count 115 109 82 56 70 34 % within IV 51.3% 48.7% 59.4% 40.6% 67.3% 32.7% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Member of political/social network Base 2009 Survey 2009 Closure 2009 Treatment Control Treatment Control Treatment Control Yes Count 37 39 82 46 71 27 % within IV 48.7% 51.3% 64.1% 35.9% 72.4% 27.6% % within DV 52.9% 53.4% 80.4% 59.7% 75.5% 60.0% No Count 33 34 0 2 23 18 % within IV 49.3% 50.7% .0% 100.0% 56.1% 43.9% % within DV 47.1% 46.6% .0% 2.6% 24.5% 40.0% N/R Count 0 0 20 29 0 0 % within IV .0% .0% 40.8% 59.2% .0% .0% % within DV .0% .0% 19.6% 37.7% .0% .0% Total Count 70 73 102 77 94 45 % within IV 49.0% 51.0% 57.0% 43.0% 67.6% 32.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Member of political/social networks Base 2010 Survey 2010 Closure 2010 Yes No Yes No Treatment Count 36 21 60 36 % within IV 63.2% 36.8% 62.5% 37.5% % within DV 40.9% 45.7% 56.1% 55.4% Control Count 52 25 47 29 % within IV 67.5% 32.5% 61.0% 37.7% % within DV 59.1% 54.3% 43.9% 44.6% Total Count 88 46 107 65 % within IV 65.7% 34.3% 61.8% 37.6% % within DV 100.0% 100.0% 100.0% 100.0% Member of political/social networks Base 2011 Closure 2011 Time on Network Group Base 2009 Survey 2009 Closure 2009 Treatment 3.2 2.5 2.2 Control 3.6 4.0 1.5 Time on Network Type of group Base 2010 Survey 2010 Closure 2010 Treatment 3.2 2.5 2.2 Control 3.6 4.0 1.5 Time on Network Type of group Base 2011 Closure 2011 Treatment 4.5 1.9 Control 4.3 1.4 Years in organizanization Group Base 2009 Survey 2009 Closure 2009 Treatment 8.5 9.6 9.5 Control 6.2 8.0 7.6 Years in organization Type of group Base 2010 Survey 2010 Closure 2010 Treatment 8.5 8.6 9.1 Control 6.5 8.4 8.6 Years in organization Type of group Base 2011 Closure 2011 Treatment 8.0 9.1 Control 7.2 7.8 Discussion activities on party Group Base 2009 Survey 2009 Closure 2009 Treatment 1.9 8.7 16.3 Control 2.6 11.6 3.5 Discussion activities on party Type of group Base 2010 Survey 2010 Closure 2010 Treatment 11.7 10.9 4.8 Control 10.6 7.2 14.4 Discussion activities on party Type of group Base 2011 Closure 2011 Treatment 7.6 5.0 Control 3.5 8.1 Years in actual position Group Base 2009 Survey 2009 Closure 2009 Treatment 3.6 4.0 4.9 Control 3.4 4.2 4.6 Years in actual position Type of group Base 2010 Survey 2010 Closure 2010 Treatment 4.0 4.1 4.8 Control 3.7 4.3 5.4 Years in actual position Type of group Base 2011 Closure 2011 Treatment 4.4 4.7 Control 4.1 4.9 Monthly Income Group Base 2009 Survey 2009 Closure 2009 Treatment 22,938.72$ $ 24,810.16 32,862.67$ Control 28,076.02$ $ 28,007.77 30,163.47$ Monthly Income Type of group Base 2010 Survey 2010 Closure 2010 Treatment 19,286.73$ $ 21,193.99 24,182.19$ Control 20,188.06$ $ 24,267.91 22,948.74$ Monthly income Type of group Base 2011 Closure 2011 Treatment 20,880.55$ 20,730.20$ Control 18,803.16$ 21,173.00$ Treatment Control Treatment Control Treatment Control Public Count 44 45 48 45 39 21 % within IV 49.4% 50.6% 51.6% 48.4% 65.0% 35.0% % within DV 45.8% 46.4% 46.2% 47.9% 48.1% 42.9% Private Count 29 31 35 34 29 18 % within IV 48.3% 51.7% 50.7% 49.3% 61.7% 38.3% % within DV 30.2% 32.0% 33.7% 36.2% 35.8% 36.7% Social organization Count 5 4 0 1 0 2 % within IV 55.6% 44.4% .0% 100.0% .0% 100.0% % within DV 5.2% 4.1% .0% 1.1% .0% 4.1% Self Count 16 13 21 14 6 6 % within IV 55.2% 44.8% 60.0% 40.0% 50.0% 50.0% % within DV 16.7% 13.4% 20.2% 14.9% 7.4% 12.2% No response Count 2 4 0 0 7 2 % within IV 33.3% 66.7% .0% .0% 77.8% 22.2% % within DV 2.1% 4.1% .0% .0% 8.6% 4.1% Total Count 96 97 104 94 81 49 % within IV 49.7% 50.3% 52.5% 47.5% 62.3% 37.7% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% In which industry do you work? Base 2009 Survey 2009 Closure 2009 Treatment Control Treatment Control Treatment Control Public Count 47 51 48 48 58 33 % within IV 48.0% 52.0% 50.0% 50.0% 63.7% 36.3% % within DV 42.3% 49.0% 43.6% 45.3% 50.9% 44.0% Private Count 36 30 43 37 33 22 % within IV 54.5% 45.5% 53.8% 46.3% 60.0% 40.0% % within DV 32.4% 28.8% 39.1% 34.9% 28.9% 29.3% International organization Count 0 1 1 2 0 1 % within IV .0% 100.0% 33.3% 66.7% .0% 100.0% % within DV .0% 1.0% .9% 1.9% .0% 1.3% Social organization Count 2 3 2 5 1 2 % within IV 40.0% 60.0% 28.6% 71.4% 33.3% 66.7% % within DV 1.8% 2.9% 1.8% 4.7% .9% 2.7% Self Count 26 19 16 14 10 11 % within IV 57.8% 42.2% 53.3% 46.7% 47.6% 52.4% % within DV 23.4% 18.3% 14.5% 13.2% 8.8% 14.7% N/R Count 0 0 0 0 12 6 % within IV .0% .0% .0% .0% 66.7% 33.3% % within DV .0% .0% .0% .0% 10.5% 8.0% Total Count 111 104 110 106 114 75 % within IV 51.6% 48.4% 50.9% 49.1% 60.3% 39.7% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% In which industry do you work? Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Public Count 59 69 60 73 % within IV 46.1% 53.9% 45.1% 54.9% % within DV 56.7% 59.5% 51.3% 56.6% Private Count 27 26 34 33 % within IV 50.9% 49.1% 50.7% 49.3% % within DV 26.0% 22.4% 29.1% 25.6% International organization Count 0 1 0 2 % within IV .0% 100.0% .0% 100.0% % within DV .0% .9% .0% 1.6% Social organization Count 2 11 2 6 % within IV 15.4% 84.6% 25.0% 75.0% % within DV 1.9% 9.5% 1.7% 4.7% Self Count 16 9 10 7 % within IV 64.0% 36.0% 58.8% 41.2% % within DV 15.4% 7.8% 8.5% 5.4% Total Count 104 116 11 8 % within IV 47.3% 52.7% 57.9% 42.1% % within DV 100.0% 100.0% 9.4% 6.2% Base 2011 Closure 2011 In which industry do you work? Treatment Control Treatment Control Treatment Control Work Count 53 58 64 60 55 34 % within IV 47.7% 52.3% 51.6% 48.4% 61.8% 38.2% % within DV 46.1% 53.2% 55.7% 55.6% 65.5% 63.0% Study Count 14 10 6 7 4 3 % within IV 58.3% 41.7% 46.2% 53.8% 57.1% 42.9% % within DV 12.2% 9.2% 5.2% 6.5% 4.8% 5.6% Work & study Count 43 39 38 34 22 12 % within IV 52.4% 47.6% 52.8% 47.2% 64.7% 35.3% % within DV 37.4% 35.8% 33.0% 31.5% 26.2% 22.2% Do not work Count 3 1 4 6 3 5 % within IV 75.0% 25.0% 40.0% 60.0% 37.5% 62.5% % within DV 2.6% .9% 3.5% 5.6% 3.6% 9.3% Niether work nor study Count 2 1 3 1 0 0 % within IV 66.7% 33.3% 75.0% 25.0% .0% .0% % within DV 1.7% .9% 2.6% .9% .0% .0% Total Count 115 109 115 108 84 54 % within IV 51.3% 48.7% 51.6% 48.4% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Occupation or employment status Base 2009 Survey 2009 Closure 2009 Treatment Control Treatment Control Treatment Control Work Count 67 54 58 64 63 41 % within IV 55.4% 44.6% 47.5% 52.5% 60.6% 39.4% % within DV 54.5% 43.2% 47.2% 51.2% 54.8% 52.6% Study Count 11 14 7 5 9 6 % within IV 44.0% 56.0% 58.3% 41.7% 60.0% 40.0% % within DV 8.9% 11.2% 5.7% 4.0% 7.8% 7.7% Work & Study Count 42 50 52 42 42 28 % within IV 45.7% 54.3% 55.3% 44.7% 60.0% 40.0% % within DV 34.1% 40.0% 42.3% 33.6% 36.5% 35.9% Do not work Count 3 6 5 11 1 3 % within IV 33.3% 66.7% 31.3% 68.8% 25.0% 75.0% % within DV 2.4% 4.8% 4.1% 8.8% .9% 3.8% Neither work nor study Count 0 1 1 3 0 0 % within IV .0% 100.0% 25.0% 75.0% .0% .0% % within DV .0% .8% .8% 2.4% .0% .0% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Occupation or employment status Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Work Count 60 56 66 65 % within IV 51.7% 48.3% 50.4% 49.6% % within DV 48.8% 41.5% 53.2% 48.5% Study Count 9 10 10 6 % within IV 47.4% 52.6% 62.5% 37.5% % within DV 7.3% 7.4% 8.1% 4.5% Work & Study Count 44 60 41 57 % within IV 42.3% 57.7% 41.8% 58.2% % within DV 35.8% 44.4% 33.1% 42.5% Do not work Count 8 8 5 3 % within IV 50.0% 50.0% 62.5% 37.5% % within DV 6.5% 5.9% 4.0% 2.2% Neither work nor study Count 2 1 2 3 % within IV 66.7% 33.3% 40.0% 60.0% % within DV 1.6% .7% 1.6% 2.2% Total Count 123 135 124 134 % within IV 47.7% 52.3% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% Base 2011 Occupation or employment status Closure 2011 Age Group Base 2009 Survey 2009 Closure 2009 Treatment 31.46 31.67 33.61 Control 30.64 31.45 34.06 Age Type of group Base 2010 Survey 2010 Closure 2010 Treatment 32.33 32.34 32.47 Control 30.98 31.38 31.27 Age Type of group Base 2011 Closure 2011 Treatment 30.9 30.6 Control 31.1 31.2 Political party OCS Political party OCS Political party OCS Type of group Treatment Count 101 14 96 19 74 10 % within IV 87.8% 12.2% 83.5% 16.5% 88.1% 11.9% % within DV 54.9% 35.0% 54.5% 39.6% 63.8% 45.5% Control Count 83 26 80 29 42 12 % within IV 76.1% 23.9% 73.4% 26.6% 77.8% 22.2% % within DV 45.1% 65.0% 45.5% 60.4% 36.2% 54.5% Total Count 184 40 176 48 116 22 % within IV 82.1% 17.9% 78.6% 21.4% 84.1% 15.9% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Base 2009 Survey 2009 Closure 2009 Type of Organization Political Party OCS Political Party OCS Political Party OCS Type of group Treatment Count 111 12 112 11 103 12 % within IV 90.2% 9.8% 91.1% 8.9% 89.6% 10.4% % within DV 53.1% 30.8% 53.1% 29.7% 65.6% 33.3% Control Count 98 27 99 26 54 24 % within IV 78.4% 21.6% 79.2% 20.8% 69.2% 30.8% % within DV 46.9% 69.2% 46.9% 70.3% 34.4% 66.7% Total Count 209 39 211 37 157 36 % within IV 84.3% 15.7% 85.1% 14.9% 81.3% 18.7% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Type of organization Base 2010 Survey 2010 Closure 2010 Political party OCS Political party OCS Type of group Treatment Count 116 7 116 8 % within IV 94.3% 5.7% 93.5% 6.5% % within DV 57.7% 12.3% 58.0% 13.8% Control Count 85 50 84 50 % within IV 63.0% 37.0% 62.7% 37.3% % within DV 42.3% 87.7% 42.0% 86.2% Total Count 201 57 200 58 % within IV 77.9% 22.1% 77.5% 22.5% % within DV 100.0% 100.0% 100.0% 100.0% Base 2011 Cierre 2011 Type of Organization 4. Activities Treatment Control Treatment Control Treatment Control Yes Count 70 57 65 54 49 26 % within IV 55.1% 44.9% 54.6% 45.4% 65.3% 34.7% % within DV 60.9% 52.3% 56.5% 49.5% 58.3% 48.1% No Count 45 52 50 55 34 28 % within IV 46.4% 53.6% 47.6% 52.4% 54.8% 45.2% % within DV 39.1% 47.7% 43.5% 50.5% 40.5% 51.9% N/R Count 0 0 0 0 1 0 % within IV .0% .0% .0% .0% 100.0% .0% % within DV .0% .0% .0% .0% 1.2% .0% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Inform the community on party actions Base 2009 Survey 2009 Closure 2009 Treatment Control Treatment Control Treatment Control Yes Count 65 65 73 55 74 44 % within IV 50.0% 50.0% 57.0% 43.0% 62.7% 37.3% % within DV 52.8% 52.0% 59.3% 44.0% 64.3% 56.4% No Count 58 60 50 70 40 33 % within IV 49.2% 50.8% 41.7% 58.3% 54.8% 45.2% % within DV 47.2% 48.0% 40.7% 56.0% 34.8% 42.3% N/R Count 0 0 0 0 1 1 % within IV .0% .0% .0% .0% 50.0% 50.0% % within DV .0% .0% .0% .0% .9% 1.3% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Inform the community on party actions Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Yes Count 66 71 77 88 % within IV 48.2% 51.8% 46.7% 53.3% % within DV 53.7% 53.0% 62.1% 65.7% No Count 57 63 46 42 % within IV 47.5% 52.5% 52.3% 47.7% % within DV 46.3% 47.0% 37.1% 31.3% N/R Count 0 0 1 4 % within IV .0% .0% 20.0% 80.0% % within DV .0% .0% .8% 3.0% Total Count 123 134 124 134 % within IV 47.9% 52.1% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% Inform the community on party actions Base 2011 Closure 2011 Type of group Inform the community on party actions BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Sex Male Count 43 33 27 % within IV 63.2% 54.1% 55.1% % within DV 61.4% 50.8% 55.1% Female Count 27 32 22 % within IV 57.4% 59.3% 62.9% % within DV 38.6% 49.2% 44.9% Total Count 70 65 49 % within IV 60.9% 56.5% 58.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 44 38 19 % within IV 53.7% 47.5% 48.7% % within DV 77.2% 70.4% 73.1% Female Count 13 16 7 % within IV 48.1% 55.2% 46.7% % within DV 22.8% 29.6% 26.9% Total Count 57 54 26 % within IV 52.3% 49.5% 48.1% % within DV 100.0% 100.0% 100.0% Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 33 44 42 % within IV 42.9% 57.1% 62.7% % within DV 56.9% 60.3% 56.8% Female Count 25 29 32 % within IV 54.3% 63.0% 66.7% % within DV 43.1% 39.7% 43.2% Total Count 58 73 74 % within IV 47.2% 59.3% 64.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 38 39 33 % within IV 45.2% 46.4% 71.7% % within DV 63.3% 70.9% 75.0% Female Count 22 16 11 % within IV 53.7% 39.0% 34.4% % within DV 36.7% 29.1% 25.0% Total Count 60 55 44 % within IV 48.0% 44.0% 56.4% % within DV 100.0% 100.0% 100.0% Inform the community on party actions Type of group Base 2011 Closure 2011 Treatment Sex Male Count 36 40 % within IV 57.1% 61.5% % within DV 54.5% 51.9% Female Count 30 37 % within IV 50.0% 62.7% % within DV 45.5% 48.1% Total Count 66 77 % within IV 53.7% 62.1% % within DV 100.0% 100.0% Control Sex Male Count 40 48 % within IV 55.6% 70.6% % within DV 56.3% 54.5% Female Count 31 40 % within IV 50.0% 60.6% % within DV 43.7% 45.5% Total Count 71 88 % within IV 53.0% 65.7% % within DV 100.0% 100.0% Inform the community on party actions Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Región X Metropolitan Count 15 12 8 % within IV 68.2% 54.5% 44.4% % within DV 21.4% 18.5% 16.3% North Count 16 9 2 % within IV 84.2% 47.4% 25.0% % within DV 22.9% 13.8% 4.1% Northwest Count 8 10 7 % within IV 42.1% 52.6% 70.0% % within DV 11.4% 15.4% 14.3% South Count 11 11 10 % within IV 57.9% 57.9% 71.4% % within DV 15.7% 16.9% 20.4% East-Northeast Count 20 23 22 % within IV 55.6% 63.9% 64.7% % within DV 28.6% 35.4% 44.9% Total Count 70 65 49 % within IV 60.9% 56.5% 58.3% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 27 30 16 % within IV 48.2% 53.6% 53.3% % within DV 47.4% 55.6% 61.5% North Count 10 4 3 % within IV 83.3% 33.3% 60.0% % within DV 17.5% 7.4% 11.5% Northwest Count 2 3 1 % within IV 28.6% 42.9% 14.3% % within DV 3.5% 5.6% 3.8% South Count 9 4 1 % within IV 69.2% 30.8% 14.3% % within DV 15.8% 7.4% 3.8% East-Northeast Count 9 13 6 % within IV 42.9% 61.9% 50.0% % within DV 15.8% 24.1% 23.1% Total Count 57 54 26 % within IV 52.3% 49.5% 48.1% % within DV 100.0% 100.0% 100.0% Inform the community on party actions Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 12 9 13 % within IV 60.0% 45.0% 61.9% % within DV 20.7% 12.3% 17.6% North Count 6 14 10 % within IV 27.3% 63.6% 66.7% % within DV 10.3% 19.2% 13.5% Northwest Count 11 12 9 % within IV 52.4% 57.1% 64.3% % within DV 19.0% 16.4% 12.2% South Count 9 16 16 % within IV 42.9% 76.2% 72.7% % within DV 15.5% 21.9% 21.6% East-Northeast Count 20 22 26 % within IV 51.3% 56.4% 60.5% % within DV 34.5% 30.1% 35.1% Total Count 58 73 74 % within IV 47.2% 59.3% 64.3% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 25 24 23 % within IV 47.2% 45.3% 62.2% % within DV 41.7% 43.6% 52.3% North Count 14 14 8 % within IV 50.0% 50.0% 61.5% % within DV 23.3% 25.5% 18.2% Northwest Count 9 6 5 % within IV 56.3% 37.5% 83.3% % within DV 15.0% 10.9% 11.4% South Count 5 8 5 % within IV 27.8% 44.4% 29.4% % within DV 8.3% 14.5% 11.4% East-Northeast Count 7 3 3 % within IV 70.0% 30.0% 60.0% % within DV 11.7% 5.5% 6.8% Total Count 60 55 44 % within IV 48.0% 44.0% 56.4% % within DV 100.0% 100.0% 100.0% Inform the community on party actions Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 11 9 % within IV 57.9% 47.4% % within DV 16.7% 11.7% North Count 11 12 % within IV 61.1% 66.7% % within DV 16.7% 15.6% Northwest Count 10 14 % within IV 45.5% 63.6% % within DV 15.2% 18.2% South Count 13 17 % within IV 65.0% 85.0% % within DV 19.7% 22.1% East-Northeast Count 21 25 % within IV 47.7% 55.6% % within DV 31.8% 32.5% Total Count 66 77 % within IV 53.7% 62.1% % within DV 100.0% 100.0% Control Región X Metropolitan Count 27 30 % within IV 58.7% 66.7% % within DV 38.0% 34.1% North Count 8 10 % within IV 50.0% 62.5% % within DV 11.3% 11.4% Northwest Count 10 12 % within IV 62.5% 70.6% % within DV 14.1% 13.6% South Count 14 21 % within IV 50.0% 72.4% % within DV 19.7% 23.9% East-Northeast Count 12 15 % within IV 42.9% 55.6% % within DV 16.9% 17.0% Total Count 71 88 % within IV 53.0% 65.7% % within DV 100.0% 100.0% Inform the community on party actions Treatment Control Treatment Control Treatment Control Yes Count 46 43 45 32 30 13 % within IV 51.7% 48.3% 58.4% 41.6% 69.8% 30.2% % within DV 40.0% 39.4% 39.1% 29.4% 35.7% 24.1% No Count 69 66 70 77 54 41 % within IV 51.1% 48.9% 47.6% 52.4% 56.8% 43.2% % within DV 60.0% 60.6% 60.9% 70.6% 64.3% 75.9% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Meetings with citizens on Base 2009 Survey 2009 Closure 2009 accountability Treatment Control Treatment Control Treatment Control Yes Count 45 36 48 36 49 19 % within IV 55.6% 44.4% 57.1% 42.9% 72.1% 27.9% % within DV 36.6% 28.8% 39.0% 28.8% 42.6% 24.4% No Count 78 89 75 89 66 59 % within IV 46.7% 53.3% 45.7% 54.3% 52.8% 47.2% % within DV 63.4% 71.2% 61.0% 71.2% 57.4% 75.6% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Meetings with citizens on accountability Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Yes Count 29 49 38 36 % within IV 37.2% 62.8% 51.4% 48.6% % within DV 23.6% 36.3% 30.6% 26.9% No Count 94 86 86 98 % within IV 52.2% 47.8% 46.7% 53.3% % within DV 76.4% 63.7% 69.4% 73.1% Total Count 123 135 124 134 % within IV 47.7% 52.3% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% Meetings with citezens on accountability Base 2011 Closure 2011 Type of group Meetings with citizens on accountability BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Sex Male Count 30 27 18 % within IV 44.1% 44.3% 36.7% % within DV 65.2% 60.0% 60.0% Female Count 16 18 12 % within IV 34.0% 33.3% 34.3% % within DV 34.8% 40.0% 40.0% Total Count 46 45 30 % within IV 40.0% 39.1% 35.7% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 31 20 11 % within IV 37.8% 25.0% 28.2% % within DV 72.1% 62.5% 84.6% Female Count 12 12 2 % within IV 44.4% 41.4% 13.3% % within DV 27.9% 37.5% 15.4% Total Count 43 32 13 % within IV 39.4% 29.4% 24.1% % within DV 100.0% 100.0% 100.0% Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 23 31 32 % within IV 29.9% 40.3% 47.8% % within DV 51.1% 64.6% 65.3% Female Count 22 17 17 % within IV 47.8% 37.0% 35.4% % within DV 48.9% 35.4% 34.7% Total Count 45 48 49 % within IV 36.6% 39.0% 42.6% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 21 26 11 % within IV 25.0% 31.0% 23.9% % within DV 58.3% 72.2% 57.9% Female Count 15 10 8 % within IV 36.6% 24.4% 25.0% % within DV 41.7% 27.8% 42.1% Total Count 36 36 19 % within IV 28.8% 28.8% 24.4% % within DV 100.0% 100.0% 100.0% Meetings with citizens on accountability Type of group Base 2011 Closure 2011 Treatment Sex Male Count 16 21 % within IV 25.4% 32.3% % within DV 55.2% 55.3% Female Count 13 17 % within IV 21.7% 28.8% % within DV 44.8% 44.7% Total Count 29 38 % within IV 23.6% 30.6% % within DV 100.0% 100.0% Control Sex Male Count 31 24 % within IV 42.5% 35.3% % within DV 63.3% 66.7% Female Count 18 12 % within IV 29.0% 18.2% % within DV 36.7% 33.3% Total Count 49 36 % within IV 36.3% 26.9% % within DV 100.0% 100.0% Meetings with citezens on accountability Type of group Base 2009 Encuesta 2009 Cierre 2009 Treatment Región X Metropolitan Count 9 7 7 % within IV 40.9% 31.8% 38.9% % within DV 19.6% 15.6% 23.3% North Count 9 8 2 % within IV 47.4% 42.1% 25.0% % within DV 19.6% 17.8% 6.7% Northwest Count 8 4 5 % within IV 42.1% 21.1% 50.0% % within DV 17.4% 8.9% 16.7% South Count 8 7 7 % within IV 42.1% 36.8% 50.0% % within DV 17.4% 15.6% 23.3% East-Northeast Count 12 19 9 % within IV 33.3% 52.8% 26.5% % within DV 26.1% 42.2% 30.0% Total Count 46 45 30 % within IV 40.0% 39.1% 35.7% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 20 14 4 % within IV 35.7% 25.0% 13.3% % within DV 46.5% 43.8% 30.8% North Count 3 3 1 % within IV 25.0% 25.0% 20.0% % within DV 7.0% 9.4% 7.7% Northwest Count 4 1 0 % within IV 57.1% 14.3% 0.0% % within DV 9.3% 3.1% 0.0% South Count 7 6 2 % within IV 53.8% 46.2% 28.6% % within DV 16.3% 18.8% 15.4% East-Northeast Count 9 8 6 % within IV 42.9% 38.1% 50.0% % within DV 20.9% 25.0% 46.2% Total Count 43 32 13 % within IV 39.4% 29.4% 24.1% % within DV 100.0% 100.0% 100.0% Meetings with citizens on accountability Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 8 10 7 % within IV 40.0% 50.0% 33.3% % within DV 17.8% 20.8% 14.3% North Count 6 8 10 % within IV 27.3% 36.4% 66.7% % within DV 13.3% 16.7% 20.4% Northwest Count 9 2 3 % within IV 42.9% 9.5% 21.4% % within DV 20.0% 4.2% 6.1% South Count 7 12 13 % within IV 33.3% 57.1% 59.1% % within DV 15.6% 25.0% 26.5% East-Northeast Count 15 16 16 % within IV 38.5% 41.0% 37.2% % within DV 33.3% 33.3% 32.7% Total Count 45 48 49 % within IV 36.6% 39.0% 42.6% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 16 17 8 % within IV 30.2% 32.1% 21.6% % within DV 44.4% 47.2% 42.1% North Count 7 10 2 % within IV 25.0% 35.7% 15.4% % within DV 19.4% 27.8% 10.5% Northwest Count 7 1 3 % within IV 43.8% 6.3% 50.0% % within DV 19.4% 2.8% 15.8% South Count 3 5 3 % within IV 16.7% 27.8% 17.6% % within DV 8.3% 13.9% 15.8% East-Northeast Count 3 3 3 % within IV 30.0% 30.0% 60.0% % within DV 8.3% 8.3% 15.8% Total Count 36 36 19 % within IV 28.8% 28.8% 24.4% % within DV 100.0% 100.0% 100.0% Meetings with citizens on accountability Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 4 3 % within IV 21.1% 15.8% % within DV 13.8% 7.9% North Count 5 10 % within IV 27.8% 55.6% % within DV 17.2% 26.3% Northwest Count 6 4 % within IV 27.3% 18.2% % within DV 20.7% 10.5% South Count 4 5 % within IV 20.0% 25.0% % within DV 13.8% 13.2% East-Northeast Count 10 16 % within IV 22.7% 35.6% % within DV 34.5% 42.1% Total Count 29 38 % within IV 23.6% 30.6% % within DV 100.0% 100.0% Control Región X Metropolitan Count 19 15 % within IV 41.3% 33.3% % within DV 38.8% 41.7% North Count 9 3 % within IV 56.3% 18.8% % within DV 18.4% 8.3% Northwest Count 5 1 % within IV 29.4% 5.9% % within DV 10.2% 2.8% South Count 9 11 % within IV 32.1% 37.9% % within DV 18.4% 30.6% East-Northeast Count 7 6 % within IV 25.0% 22.2% % within DV 14.3% 16.7% Total Count 49 36 % within IV 36.3% 26.9% % within DV 100.0% 100.0% Meetings with citizens on accountability Treatment Control Treatment Control Treatment Control Yes Count 8 8 11 8 8 2 % within IV 50.0% 50.0% 57.9% 42.1% 80.0% 20.0% % within DV 7.0% 7.3% 9.6% 7.3% 9.5% 3.7% No Count 107 101 104 101 76 52 % within IV 51.4% 48.6% 50.7% 49.3% 59.4% 40.6% % within DV 93.0% 92.7% 90.4% 92.7% 90.5% 96.3% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% File sworn statements of assets Base 2009 Survey 2009 Closure 2009 when exerting public functions Treatment Control Treatment Control Treatment Control Yes Count 6 9 14 10 13 2 % within IV 40.0% 60.0% 58.3% 41.7% 86.7% 13.3% % within DV 4.9% 7.2% 11.4% 8.0% 11.3% 2.6% No Count 117 116 109 115 102 76 % within IV 50.2% 49.8% 48.7% 51.3% 57.3% 42.7% % within DV 95.1% 92.8% 88.6% 92.0% 88.7% 97.4% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Closure 2010 File sworn statements of assets when exerting public functions Base 2010 Survey 2010 Treatment Control Treatment Control Yes Count 7 9 13 9 % within IV 43.8% 56.3% 59.1% 40.9% % within DV 5.7% 6.7% 10.5% 6.7% No Count 116 126 111 125 % within IV 47.9% 52.1% 47.0% 53.0% % within DV 94.3% 93.3% 89.5% 93.3% Total Count 123 135 124 134 % within IV 47.7% 52.3% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% File sworn statements of assets when exerting public functions Base 2011 Closure 2011 Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Sex Male Count 5 7 5 % within IV 7.4% 11.5% 10.2% % within DV 62.5% 63.6% 62.5% Female Count 3 4 3 % within IV 6.4% 7.4% 8.6% % within DV 37.5% 36.4% 37.5% Total Count 8 11 8 % within IV 7.0% 9.6% 9.5% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 7 4 2 % within IV 8.5% 5.0% 5.1% % within DV 87.5% 50.0% 100.0% Female Count 1 4 0 % within IV 3.7% 13.8% .0% % within DV 12.5% 50.0% .0% Total Count 8 8 2 % within IV 7.3% 7.3% 3.7% % within DV 100.0% 100.0% 100.0% File sworn statements of assets when exerting public functions Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 1 10 9 % within IV 1.3% 13.0% 13.4% % within DV 16.7% 71.4% 69.2% Female Count 5 4 4 % within IV 10.9% 8.7% 8.3% % within DV 83.3% 28.6% 30.8% Total Count 6 14 13 % within IV 4.9% 11.4% 11.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 5 7 2 % within IV 6.0% 8.3% 4.3% % within DV 55.6% 70.0% 100.0% Female Count 4 3 0 % within IV 9.8% 7.3% .0% % within DV 44.4% 30.0% .0% Total Count 9 10 2 % within IV 7.2% 8.0% 2.6% % within DV 100.0% 100.0% 100.0% File sworn statements of assets when exerting public functions Type of group Base 2011 Closure 2011 Treatment Sex Male Count 6 8 % within IV 9.5% 12.3% % within DV 85.7% 61.5% Female Count 1 5 % within IV 1.7% 8.5% % within DV 14.3% 38.5% Total Count 7 13 % within IV 5.7% 10.5% % within DV 100.0% 100.0% Control Sex Male Count 7 5 % within IV 9.6% 7.4% % within DV 77.8% 55.6% Female Count 2 4 % within IV 3.2% 6.1% % within DV 22.2% 44.4% Total Count 9 9 % within IV 6.7% 6.7% % within DV 100.0% 100.0% File sworn statements of assets when exerting public functions Type of group Base 2009 Survey 2009 Closure 2009 Treatment Región X Metropolitan Count 1 2 1 % within IV 4.5% 9.1% 5.6% % within DV 12.5% 18.2% 12.5% North Count 2 3 2 % within IV 10.5% 15.8% 25.0% % within DV 25.0% 27.3% 25.0% Northwest Count 0 1 1 % within IV 0.0% 5.3% 10.0% % within DV 0.0% 9.1% 12.5% South Count 2 2 1 % within IV 10.5% 10.5% 7.1% % within DV 25.0% 18.2% 12.5% East-Northeast Count 3 3 3 % within IV 8.3% 8.3% 8.8% % within DV 37.5% 27.3% 37.5% Total Count 8 11 8 % within IV 7.0% 9.6% 9.5% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 2 4 1 % within IV 3.6% 7.1% 3.3% % within DV 25.0% 50.0% 50.0% North Count 1 2 0 % within IV 8.3% 16.7% 0.0% % within DV 12.5% 25.0% 0.0% Northwest Count 1 0 0 % within IV 14.3% 0.0% 0.0% % within DV 12.5% 0.0% 0.0% South Count 2 1 0 % within IV 15.4% 7.7% 0.0% % within DV 25.0% 12.5% 0.0% East-Northeast Count 2 1 1 % within IV 9.5% 4.8% 8.3% % within DV 25.0% 12.5% 50.0% Total Count 8 8 2 % within IV 7.3% 7.3% 3.7% % within DV 100.0% 100.0% 100.0% File sworn statements of assets when exerting public functions Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 2 1 1 % within IV 10.0% 5.0% 4.8% % within DV 33.3% 7.1% 7.7% North Count 1 1 2 % within IV 4.5% 4.5% 13.3% % within DV 16.7% 7.1% 15.4% Northwest Count 1 2 1 % within IV 4.8% 9.5% 7.1% % within DV 16.7% 14.3% 7.7% South Count 1 4 4 % within IV 4.8% 19.0% 18.2% % within DV 16.7% 28.6% 30.8% East-Northeast Count 1 6 5 % within IV 2.6% 15.4% 11.6% % within DV 16.7% 42.9% 38.5% Total Count 6 14 13 % within IV 4.9% 11.4% 11.3% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 1 4 1 % within IV 1.9% 7.5% 2.7% % within DV 11.1% 40.0% 50.0% North Count 3 3 1 % within IV 10.7% 10.7% 7.7% % within DV 33.3% 30.0% 50.0% Northwest Count 3 0 0 % within IV 18.8% 0.0% 0.0% % within DV 33.3% 0.0% 0.0% South Count 1 2 0 % within IV 5.6% 11.1% 0.0% % within DV 11.1% 20.0% 0.0% East-Northeast Count 1 1 0 % within IV 10.0% 10.0% 0.0% % within DV 11.1% 10.0% 0.0% Total Count 9 10 2 % within IV 7.2% 8.0% 2.6% % within DV 100.0% 100.0% 100.0% File sworn statements of assets when exerting public functions Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 1 0 % within IV 5.3% 0.0% % within DV 14.3% 0.0% North Count 1 0 % within IV 5.6% 0.0% % within DV 14.3% 0.0% Northwest Count 2 4 % within IV 9.1% 18.2% % within DV 28.6% 30.8% South Count 1 1 % within IV 5.0% 5.0% % within DV 14.3% 7.7% East-Northeast Count 2 8 % within IV 4.5% 17.8% % within DV 28.6% 61.5% Total Count 7 13 % within IV 5.7% 10.5% % within DV 100.0% 100.0% Control Región X Metropolitan Count 4 3 % within IV 8.7% 6.7% % within DV 44.4% 33.3% North Count 0 1 % within IV 0.0% 6.3% % within DV 0.0% 11.1% Northwest Count 1 2 % within IV 5.9% 11.8% % within DV 11.1% 22.2% South Count 2 1 % within IV 7.1% 3.4% % within DV 22.2% 11.1% East-Northeast Count 2 2 % within IV 7.1% 7.4% % within DV 22.2% 22.2% Total Count 9 9 % within IV 6.7% 6.7% % within DV 100.0% 100.0% File sworn statements of assets when exerting public functions Treatment Control Treatment Control Treatment Control Yes Count 57 44 44 49 40 22 % within IV 56.4% 43.6% 47.3% 52.7% 64.5% 35.5% % within DV 49.6% 40.4% 38.3% 45.0% 47.6% 40.7% No Count 58 65 71 60 44 32 % within IV 47.2% 52.8% 54.2% 45.8% 57.9% 42.1% % within DV 50.4% 59.6% 61.7% 55.0% 52.4% 59.3% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Stimulate senior leaders to make public Base 2009 Survey 2009 Closure 2009 the administration of their funds Treatment Control Treatment Control Treatment Control Yes Count 41 39 57 39 53 26 % within IV 51.3% 48.8% 59.4% 40.6% 67.1% 32.9% % within DV 33.3% 31.2% 46.3% 31.2% 46.1% 33.3% No Count 82 86 66 86 62 52 % within IV 48.8% 51.2% 43.4% 56.6% 54.4% 45.6% % within DV 66.7% 68.8% 53.7% 68.8% 53.9% 66.7% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Yes Count 37 44 53 49 % within IV 45.7% 54.3% 52.0% 48.0% % within DV 30.1% 32.6% 42.7% 36.6% No Count 86 90 71 85 % within IV 48.9% 51.1% 45.5% 54.5% % within DV 69.9% 66.7% 57.3% 63.4% Total Count 123 135 124 134 % within IV 47.7% 52.3% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Base 2011 Closure 2011 Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 41 24 26 % within IV 60.3% 39.3% 53.1% % within DV 71.9% 54.5% 65.0% Female Count 16 20 14 % within IV 34.0% 37.0% 40.0% % within DV 28.1% 45.5% 35.0% Total Count 57 44 40 % within IV 49.6% 38.3% 47.6% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 36 31 16 % within IV 43.9% 38.8% 41.0% % within DV 81.8% 63.3% 72.7% Female Count 8 18 6 % within IV 29.6% 62.1% 40.0% % within DV 18.2% 36.7% 27.3% Total Count 44 49 22 % within IV 40.4% 45.0% 40.7% % within DV 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 26 37 33 % within IV 33.8% 48.1% 49.3% % within DV 63.4% 64.9% 62.3% Female Count 15 20 20 % within IV 32.6% 43.5% 41.7% % within DV 36.6% 35.1% 37.7% Total Count 41 57 53 % within IV 33.3% 46.3% 46.1% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 26 27 18 % within IV 31.0% 32.1% 39.1% % within DV 66.7% 69.2% 69.2% Female Count 13 12 8 % within IV 31.7% 29.3% 25.0% % within DV 33.3% 30.8% 30.8% Total Count 39 39 26 % within IV 31.2% 31.2% 33.3% % within DV 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Type of group Base 2011 Closure 2011 Treatment Sex Male Count 19 29 % within IV 30.2% 44.6% % within DV 51.4% 54.7% Female Count 18 24 % within IV 30.0% 40.7% % within DV 48.6% 45.3% Total Count 37 53 % within IV 30.1% 42.7% % within DV 100.0% 100.0% Control Sex Male Count 27 30 % within IV 37.0% 44.1% % within DV 61.4% 61.2% Female Count 17 19 % within IV 27.4% 28.8% % within DV 38.6% 38.8% Total Count 44 49 % within IV 32.6% 36.6% % within DV 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Type of group Base 2009 Survey 2009 Closure 2009 Treatment Región X Metropolitan Count 10 5 7 % within IV 45.5% 22.7% 38.9% % within DV 17.5% 11.4% 17.5% North Count 11 5 4 % within IV 57.9% 26.3% 50.0% % within DV 19.3% 11.4% 10.0% Northwest Count 10 9 4 % within IV 52.6% 47.4% 40.0% % within DV 17.5% 20.5% 10.0% South Count 11 8 8 % within IV 57.9% 42.1% 57.1% % within DV 19.3% 18.2% 20.0% East-Northeast Count 15 17 17 % within IV 41.7% 47.2% 50.0% % within DV 26.3% 38.6% 42.5% Total Count 57 44 40 % within IV 49.6% 38.3% 47.6% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 22 28 11 % within IV 39.3% 50.0% 36.7% % within DV 50.0% 57.1% 50.0% North Count 4 6 3 % within IV 33.3% 50.0% 60.0% % within DV 9.1% 12.2% 13.6% Northwest Count 4 2 0 % within IV 57.1% 28.6% 0.0% % within DV 9.1% 4.1% 0.0% South Count 5 5 0 % within IV 38.5% 38.5% 0.0% % within DV 11.4% 10.2% 0.0% East-Northeast Count 9 8 8 % within IV 42.9% 38.1% 66.7% % within DV 20.5% 16.3% 36.4% Total Count 44 49 22 % within IV 40.4% 45.0% 40.7% % within DV 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 12 5 8 % within IV 60.0% 25.0% 38.1% % within DV 29.3% 8.8% 15.1% North Count 7 13 7 % within IV 31.8% 59.1% 46.7% % within DV 17.1% 22.8% 13.2% Northwest Count 4 6 3 % within IV 19.0% 28.6% 21.4% % within DV 9.8% 10.5% 5.7% South Count 8 12 15 % within IV 38.1% 57.1% 68.2% % within DV 19.5% 21.1% 28.3% East-Northeast Count 10 21 20 % within IV 25.6% 53.8% 46.5% % within DV 24.4% 36.8% 37.7% Total Count 41 57 53 % within IV 33.3% 46.3% 46.1% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 21 18 9 % within IV 39.6% 34.0% 24.3% % within DV 53.8% 46.2% 34.6% North Count 8 10 10 % within IV 28.6% 35.7% 76.9% % within DV 20.5% 25.6% 38.5% Northwest Count 4 4 3 % within IV 25.0% 25.0% 50.0% % within DV 10.3% 10.3% 11.5% South Count 4 6 2 % within IV 22.2% 33.3% 11.8% % within DV 10.3% 15.4% 7.7% East-Northeast Count 2 1 2 % within IV 20.0% 10.0% 40.0% % within DV 5.1% 2.6% 7.7% Total Count 39 39 26 % within IV 31.2% 31.2% 33.3% % within DV 100.0% 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 8 5 % within IV 42.1% 26.3% % within DV 21.6% 9.4% North Count 5 8 % within IV 27.8% 44.4% % within DV 13.5% 15.1% Northwest Count 6 10 % within IV 27.3% 45.5% % within DV 16.2% 18.9% South Count 8 8 % within IV 40.0% 40.0% % within DV 21.6% 15.1% East-Northeast Count 10 22 % within IV 22.7% 48.9% % within DV 27.0% 41.5% Total Count 37 53 % within IV 30.1% 42.7% % within DV 100.0% 100.0% Control Región X Metropolitan Count 20 21 % within IV 43.5% 46.7% % within DV 45.5% 42.9% North Count 4 6 % within IV 25.0% 37.5% % within DV 9.1% 12.2% Northwest Count 5 3 % within IV 29.4% 17.6% % within DV 11.4% 6.1% South Count 7 8 % within IV 25.0% 27.6% % within DV 15.9% 16.3% East-Northeast Count 8 11 % within IV 28.6% 40.7% % within DV 18.2% 22.4% Total Count 44 49 % within IV 32.6% 36.6% % within DV 100.0% 100.0% Stimulate senior leaders to make public the administration of their funds Treatment Control Treatment Control Treatment Control Yes Count 38 26 35 24 31 10 % within IV 59.4% 40.6% 59.3% 40.7% 75.6% 24.4% % within DV 33.0% 23.9% 30.4% 22.0% 36.9% 18.5% No Count 77 83 80 85 53 44 % within IV 48.1% 51.9% 48.5% 51.5% 54.6% 45.4% % within DV 67.0% 76.1% 69.6% 78.0% 63.1% 81.5% Total Count 115 109 115 109 84 54 % within IV 51.3% 48.7% 51.3% 48.7% 60.9% 39.1% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Offer up public declarations on Base 2009 Survey 2009 Closure 2009 internal matters of your organization Treatment Control Treatment Control Treatment Control Yes Count 26 32 41 37 37 16 % within IV 44.8% 55.2% 52.6% 47.4% 69.8% 30.2% % within DV 21.1% 25.6% 33.3% 29.6% 32.2% 20.5% No Count 97 93 82 88 78 62 % within IV 51.1% 48.9% 48.2% 51.8% 55.7% 44.3% % within DV 78.9% 74.4% 66.7% 70.4% 67.8% 79.5% Total Count 123 125 123 125 115 78 % within IV 49.6% 50.4% 49.6% 50.4% 59.6% 40.4% % within DV 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Base 2010 Survey 2010 Closure 2010 Treatment Control Treatment Control Yes Count 29 33 39 41 % within IV 46.8% 53.2% 48.8% 51.3% % within DV 23.6% 24.4% 31.5% 30.6% No Count 94 101 85 93 % within IV 48.2% 51.8% 47.8% 52.2% % within DV 76.4% 74.8% 68.5% 69.4% Total Count 123 135 124 134 % within IV 47.7% 52.3% 48.1% 51.9% % within DV 100.0% 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Base 2011 Closure 2011 Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 30 18 27 % within IV 44.1% 29.5% 55.1% % within DV 78.9% 51.4% 50.9% Female Count 8 17 26 % within IV 17.0% 31.5% 74.3% % within DV 21.1% 48.6% 49.1% Total Count 38 35 53 % within IV 33.0% 30.4% 63.1% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 21 17 30 % within IV 25.6% 21.3% 76.9% % within DV 80.8% 70.8% 68.2% Female Count 5 7 14 % within IV 18.5% 24.1% 93.3% % within DV 19.2% 29.2% 31.8% Total Count 26 24 44 % within IV 23.9% 22.0% 81.5% % within DV 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 13 29 24 % within IV 16.9% 37.7% 35.8% % within DV 50.0% 70.7% 64.9% Female Count 13 12 13 % within IV 28.3% 26.1% 27.1% % within DV 50.0% 29.3% 35.1% Total Count 26 41 37 % within IV 21.1% 33.3% 32.2% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 21 31 9 % within IV 25.0% 36.9% 19.6% % within DV 65.6% 83.8% 56.3% Female Count 11 6 7 % within IV 26.8% 14.6% 21.9% % within DV 34.4% 16.2% 43.8% Total Count 32 37 16 % within IV 25.6% 29.6% 20.5% % within DV 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group Base 2011 Closure 2011 Treatment Sex Male Count 17 24 % within IV 27.0% 36.9% % within DV 58.6% 61.5% Female Count 12 15 % within IV 20.0% 25.4% % within DV 41.4% 38.5% Total Count 29 39 % within IV 23.6% 31.5% % within DV 100.0% 100.0% Control Sex Male Count 21 26 % within IV 28.8% 38.2% % within DV 63.6% 63.4% Female Count 12 15 % within IV 19.4% 22.7% % within DV 36.4% 36.6% Total Count 33 41 % within IV 24.4% 30.6% % within DV 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group Base 2009 Survey 2009 Closure 2009 Treatment Región X Metropolitan Count 8 4 4 % within IV 36.4% 18.2% 22.2% % within DV 21.1% 11.4% 12.9% North Count 6 5 3 % within IV 31.6% 26.3% 37.5% % within DV 15.8% 14.3% 9.7% Northwest Count 4 5 4 % within IV 21.1% 26.3% 40.0% % within DV 10.5% 14.3% 12.9% South Count 7 6 8 % within IV 36.8% 31.6% 57.1% % within DV 18.4% 17.1% 25.8% East-Northeast Count 13 15 12 % within IV 36.1% 41.7% 35.3% % within DV 34.2% 42.9% 38.7% Total Count 38 35 31 % within IV 33.0% 30.4% 36.9% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 10 14 4 % within IV 17.9% 25.0% 13.3% % within DV 38.5% 58.3% 40.0% North Count 3 2 1 % within IV 25.0% 16.7% 20.0% % within DV 11.5% 8.3% 10.0% Northwest Count 1 1 0 % within IV 14.3% 14.3% 0.0% % within DV 3.8% 4.2% 0.0% South Count 4 1 0 % within IV 30.8% 7.7% 0.0% % within DV 15.4% 4.2% 0.0% East-Northeast Count 8 6 5 % within IV 38.1% 28.6% 41.7% % within DV 30.8% 25.0% 50.0% Total Count 26 24 10 % within IV 23.9% 22.0% 18.5% % within DV 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X MetropolitanCount 7 4 2 % within IV 35.0% 20.0% 9.5% % within DV 26.9% 9.8% 5.4% North Count 4 10 4 % within IV 18.2% 45.5% 26.7% % within DV 15.4% 24.4% 10.8% Northwest Count 1 6 5 % within IV 4.8% 28.6% 35.7% % within DV 3.8% 14.6% 13.5% South Count 6 12 12 % within IV 28.6% 57.1% 54.5% % within DV 23.1% 29.3% 32.4% East-Northeast Count 8 9 14 % within IV 20.5% 23.1% 32.6% % within DV 30.8% 22.0% 37.8% Total Count 26 41 37 % within IV 21.1% 33.3% 32.2% % within DV 100.0% 100.0% 100.0% Control Región X MetropolitanCount 15 13 4 % within IV 28.3% 24.5% 10.8% % within DV 46.9% 35.1% 25.0% North Count 6 12 3 % within IV 21.4% 42.9% 23.1% % within DV 18.8% 32.4% 18.8% Northwest Count 4 2 1 % within IV 25.0% 12.5% 16.7% % within DV 12.5% 5.4% 6.3% South Count 3 6 4 % within IV 16.7% 33.3% 23.5% % within DV 9.4% 16.2% 25.0% East-Northeast Count 4 4 4 % within IV 40.0% 40.0% 80.0% % within DV 12.5% 10.8% 25.0% Total Count 32 37 16 % within IV 25.6% 29.6% 20.5% % within DV 100.0% 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 5 6 % within IV 26.3% 31.6% % within DV 17.2% 15.4% North Count 4 5 % within IV 22.2% 27.8% % within DV 13.8% 12.8% Northwest Count 4 7 % within IV 18.2% 31.8% % within DV 13.8% 17.9% South Count 8 5 % within IV 40.0% 25.0% % within DV 27.6% 12.8% East-Northeast Count 8 16 % within IV 18.2% 35.6% % within DV 27.6% 41.0% Total Count 29 39 % within IV 23.6% 31.5% % within DV 100.0% 100.0% Control Región X Metropolitan Count 11 18 % within IV 23.9% 40.0% % within DV 33.3% 43.9% North Count 6 7 % within IV 37.5% 43.8% % within DV 18.2% 17.1% Northwest Count 4 5 % within IV 23.5% 29.4% % within DV 12.1% 12.2% South Count 5 6 % within IV 17.9% 20.7% % within DV 15.2% 14.6% East-Northeast Count 7 5 % within IV 25.0% 18.5% % within DV 21.2% 12.2% Total Count 33 41 % within IV 24.4% 30.6% % within DV 100.0% 100.0% Offer up public declarations on internal matters of your organization Type of group Base 2009 Survey 2009 Closure 2009 Treatment Count 63 108 76 % within IV 54.8% 100.0% 90.5% % within DV 100.0% 96.4% 100.0% Control Count 65 51 36 % within IV 59.6% 100.0% 66.7% % within DV 100.0% 48.6% 100.0% Have you received any political/social training course Type of group Base 2010 Survey 2010 Closure 2010 Type of group Treatment Count 61 103 90 % within IV 50.4% 83.7% 92.8% % within DV 100.0% 100.0% 100.0% Control Count 69 62 32 % within IV 55.6% 49.6% 50.0% % within DV 100.0% 100.0% 100.0% Have you received any political/social training course Type of group Base 2011 Closure 2011 Treatment Count 53 112 % within IV 41.1% 86.8% % within DV 100.0% 100.0% Control Count 73 75 % within IV 54.1% 54.7% % within DV 100.0% 100.0% Have you received any political/social training course Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 34 56 43 % within IV 50.0% 51.9% 87.8% % within DV 54.0% 50.0% 56.6% Female Count 29 52 33 % within IV 61.7% 48.1% 94.3% % within DV 46.0% 46.4% 43.4% Total Count 63 108 76 % within IV 54.8% 100.0% 90.5% % within DV 100.0% 96.4% 100.0% Control Sex Male Count 47 34 27 % within IV 57.3% 66.7% 69.2% % within DV 72.3% 32.4% 75.0% Female Count 18 17 9 % within IV 66.7% 33.3% 60.0% % within DV 27.7% 16.2% 25.0% Total Count 65 51 36 % within IV 59.6% 100.0% 66.7% % within DV 100.0% 48.6% 100.0% Have you received any political/social training course Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 40 64 60 % within IV 52.6% 82.1% 89.6% % within DV 66.7% 62.7% 57.1% Female Count 20 38 45 % within IV 46.5% 86.4% 93.8% % within DV 33.3% 37.3% 42.9% Total Count 60 102 105 % within IV 50.4% 83.6% 91.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 48 40 23 % within IV 57.1% 47.6% 50.0% % within DV 69.6% 64.5% 54.8% Female Count 21 22 19 % within IV 51.2% 53.7% 59.4% % within DV 30.4% 35.5% 45.2% Total Count 69 62 42 % within IV 55.2% 49.6% 53.8% % within DV 100.0% 100.0% 100.0% Have you received any political/social training course Type of group Base 2011 Closure 2011 Treatment Sex Male Count 25 53 % within IV 39.7% 81.5% % within DV 52.1% 49.1% Female Count 23 55 % within IV 38.3% 93.2% % within DV 47.9% 50.9% Total Count 48 108 % within IV 39.0% 87.1% % within DV 100.0% 100.0% Control Sex Male Count 36 37 % within IV 49.3% 54.4% % within DV 49.3% 52.1% Female Count 37 34 % within IV 59.7% 51.5% % within DV 50.7% 47.9% Total Count 73 71 % within IV 54.1% 53.0% % within DV 100.0% 100.0% Have you received any political/social training course Type of group Base 2009 Survey 2009 Closure 2009 Treatment Región X Metropolitan Count 13 22 18 % within IV 59.1% 100.0% 100.0% % within DV 20.6% 20.4% 23.7% North Count 12 19 8 % within IV 63.2% 100.0% 100.0% % within DV 19.0% 17.6% 10.5% Northwest Count 7 17 8 % within IV 36.8% 94.4% 80.0% % within DV 11.1% 15.7% 10.5% South Count 9 17 11 % within IV 47.4% 89.5% 78.6% % within DV 14.3% 15.7% 14.5% East-Northeast Count 22 33 31 % within IV 61.1% 97.1% 91.2% % within DV 34.9% 30.6% 40.8% Total Count 63 108 76 % within IV 54.8% 96.4% 90.5% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 37 27 19 % within IV 66.1% 49.1% 63.3% % within DV 56.9% 52.9% 52.8% North Count 7 5 4 % within IV 58.3% 45.5% 80.0% % within DV 10.8% 9.8% 11.1% Northwest Count 4 1 0 % within IV 57.1% 16.7% 0.0% % within DV 6.2% 2.0% 0.0% South Count 6 7 2 % within IV 46.2% 58.3% 28.6% % within DV 9.2% 13.7% 5.6% East-Northeast Count 11 11 11 % within IV 52.4% 52.4% 91.7% % within DV 16.9% 21.6% 30.6% Total Count 65 51 36 % within IV 59.6% 48.6% 66.7% % within DV 100.0% 100.0% 100.0% Have you received any political/social training course Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 14 20 21 % within IV 70.0% 100.0% 100.0% % within DV 22.6% 19.6% 20.0% North Count 9 19 13 % within IV 40.9% 86.4% 86.7% % within DV 14.5% 18.6% 12.4% Northwest Count 7 15 10 % within IV 33.3% 71.4% 71.4% % within DV 11.3% 14.7% 9.5% South Count 9 18 22 % within IV 45.0% 85.7% 100.0% % within DV 14.5% 17.6% 21.0% East-Northeast Count 23 30 39 % within IV 62.2% 76.9% 90.7% % within DV 37.1% 29.4% 37.1% Total Count 62 102 105 % within IV 51.7% 82.9% 91.3% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 35 35 23 % within IV 66.0% 66.0% 62.2% % within DV 50.7% 56.5% 54.8% North Count 15 6 3 % within IV 53.6% 21.4% 23.1% % within DV 21.7% 9.7% 7.1% Northwest Count 4 8 1 % within IV 25.0% 50.0% 16.7% % within DV 5.8% 12.9% 2.4% South Count 11 6 10 % within IV 61.1% 33.3% 58.8% % within DV 15.9% 9.7% 23.8% East-Northeast Count 4 7 5 % within IV 40.0% 70.0% 100.0% % within DV 5.8% 11.3% 11.9% Total Count 69 62 42 % within IV 55.2% 49.6% 53.8% % within DV 100.0% 100.0% 100.0% Have you received any political/social training course Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 5 16 % within IV 26.3% 84.2% % within DV 10.4% 14.8% North Count 7 16 % within IV 38.9% 88.9% % within DV 14.6% 14.8% Northwest Count 8 16 % within IV 36.4% 72.7% % within DV 16.7% 14.8% South Count 6 16 % within IV 30.0% 80.0% % within DV 12.5% 14.8% East-Northeast Count 22 44 % within IV 50.0% 97.8% % within DV 45.8% 40.7% Total Count 48 108 % within IV 39.0% 87.1% % within DV 100.0% 100.0% Control Región X Metropolitan Count 28 27 % within IV 60.9% 60.0% % within DV 38.4% 38.0% North Count 12 10 % within IV 75.0% 62.5% % within DV 16.4% 14.1% Northwest Count 9 13 % within IV 52.9% 76.5% % within DV 12.3% 18.3% South Count 8 6 % within IV 28.6% 20.7% % within DV 11.0% 8.5% East-Northeast Count 16 15 % within IV 57.1% 55.6% % within DV 21.9% 21.1% Total Count 73 71 % within IV 54.1% 53.0% % within DV 100.0% 100.0% Have you received any political/social training course Base 2009 Survey 2009 Closure 2009 Type of group Treatment Count 22 20 19 % within IV 19.1% 17.4% 22.6% % within DV 52.4% 52.6% 65.5% Control Count 20 18 10 % within IV 18.3% 16.5% 18.5% % within DV 47.6% 47.4% 34.5% Total Count 42 38 29 % within IV 18.8% 17.0% 21.0% % within DV 100.0% 100.0% 100.0% Publishes performance bulletins of the institution Base 2011 Closure 2011 Type of group Treatment Count 18 29 % within IV 14.6% 23.4% % within DV 33.3% 48.3% Control Count 36 31 % within IV 26.7% 23.1% % within DV 66.7% 51.7% Total Count 54 60 % within IV 20.9% 23.3% % within DV 100.0% 100.0% Publishes performance bulletins of the institution Base 2010 Survey 2010 Closure 2010 Type of group Treatment Count 27 42 33 % within IV 22.0% 34.1% 28.7% % within DV 58.7% 61.8% 80.5% Control Count 19 26 8 % within IV 15.2% 20.8% 10.3% % within DV 41.3% 38.2% 19.5% Total Count 46 68 41 % within IV 18.5% 27.4% 21.2% % within DV 100.0% 100.0% 100.0% Publishes performance bulletins of the institution Sex Base 2009 Survey 2009 Closure 2009 Male Count 30 23 21 % within IV 20.0% 16.3% 23.9% % within DV 71.4% 60.5% 72.4% Female Count 12 15 8 % within IV 16.2% 18.1% 16.0% % within DV 28.6% 39.5% 27.6% Publishes performance bulletins of the institution Sex Base 2010 Survey 2010 Closure 2010 Male Count 32 40 28 % within IV 12.4% 24.8% 24.8% % within DV 59.3% 58.8% 68.3% Female Count 22 28 13 % within IV 8.5% 32.2% 16.3% % within DV 40.7% 41.2% 31.7% Publishes performance bulletins of the institution Sex Base 2011 Closure 2011 Male Count 32 35 % within IV 23.5% 26.3% % within DV 59.3% 58.3% Female Count 22 25 % within IV 18.0% 20.0% % within DV 40.7% 41.7% Total Count 54 60 % within IV 20.9% 23.3% % within DV 100.0% 100.0% Publishes performance bulletins of the institution BASE 2009 SURVEY 2009 CLOSURE 2009 Región X Metropolitan Count 11 11 7 % within IV 14.1% 14.1% 14.6% % within DV 26.2% 28.9% 24.1% North Count 9 5 2 % within IV 29.0% 16.1% 15.4% % within DV 21.4% 13.2% 6.9% Northwest Count 4 5 3 % within IV 15.4% 19.2% 30.0% % within DV 9.5% 13.2% 10.3% South Count 7 2 4 % within IV 21.9% 6.3% 19.0% % within DV 16.7% 5.3% 13.8% East-Northeast Count 11 15 13 % within IV 19.3% 26.3% 28.3% % within DV 26.2% 39.5% 44.8% Total Count 42 38 29 % within IV 18.8% 17.0% 21.0% % within DV 100.0% 100.0% 100.0% Publishes performance bulletins of the institution BASE 2010 SURVEY 2010 CLOSURE 2010 Región X Metropolitan Count 17 15 9 % within IV 6.6% 20.5% 15.5% % within DV 31.5% 22.1% 22.0% North Count 9 15 7 % within IV 3.5% 30.0% 25.0% % within DV 16.7% 22.1% 17.1% Northwest Count 6 6 2 % within IV 2.3% 16.2% 10.0% % within DV 11.1% 8.8% 4.9% South Count 11 12 9 % within IV 4.3% 30.8% 23.1% % within DV 20.4% 17.6% 22.0% East-Northeast Count 11 20 14 % within IV 4.3% 40.8% 29.2% % within DV 20.4% 29.4% 34.1% Total Count 54 68 41 % within IV 20.9% 27.4% 21.2% % within DV 100.0% 100.0% 100.0% Publishes performance bulletins of the institution BASE 2011 CLOSURE 2011 Región X Metropolitan Count 17 16 % within IV 26.2% 25.0% % within DV 31.5% 26.7% North Count 9 9 % within IV 26.5% 26.5% % within DV 16.7% 15.0% Northwest Count 6 9 % within IV 15.4% 23.1% % within DV 11.1% 15.0% South Count 11 9 % within IV 22.9% 18.4% % within DV 20.4% 15.0% East-Northeast Count 11 17 % within IV 15.3% 23.6% % within DV 20.4% 28.3% Total Count 54 60 % within IV 20.9% 23.3% % within DV 100.0% 100.0% Publishes performance bulletins of the institution Base 2009 Survey 2009 Closure 2009 Type of group Treatment Count 81 82 73 % within IV 70.4% 71.3% 86.9% % within DV 50.6% 52.6% 70.9% Control Count 79 74 30 % within IV 72.5% 67.9% 55.6% % within DV 49.4% 47.4% 29.1% Total Count 160 156 103 % within IV 71.4% 69.6% 74.6% % within DV 100.0% 100.0% 100.0% Submit reports to immediate superiors Base 2010 Survey 2010 Closure 2010 Type of group Treatment Count 78 91 98 % within IV 63.4% 74.0% 85.2% % within DV 51.3% 51.7% 65.3% Control Count 74 85 52 % within IV 59.2% 68.0% 66.7% % within DV 48.7% 48.3% 34.7% Total Count 152 176 150 % within IV 61.3% 71.0% 77.7% % within DV 100.0% 100.0% 100.0% Submit reports to immediate superiors Base 2011 Closure 2011 Type of group Treatment Count 75 96 % within IV 61.0% 77.4% % within DV 45.2% 49.2% Control Count 91 99 % within IV 67.9% 73.9% % within DV 54.8% 50.8% Total Count 166 195 % within IV 64.6% 75.6% % within DV 100.0% 100.0% Submit reports to immediate superiors Base 2009 Survey 2009 Closure 2009 Sex Male Count 115 101 64 % within IV 76.7% 71.6% 72.7% % within DV 71.9% 64.7% 62.1% Female Count 45 55 39 % within IV 60.8% 66.3% 78.0% % within DV 28.1% 35.3% 37.9% Total Count 160 156 103 % within IV 71.4% 69.6% 74.6% % within DV 100.0% 100.0% 100.0% Submits reports to immediate superiors Base 2010 Survey 2010 Closure 2010 Sex Male Count 98 116 95 % within IV 38.1% 72.0% 84.1% % within DV 59.0% 65.9% 63.3% Female Count 68 60 55 % within IV 26.5% 69.0% 68.8% % within DV 41.0% 34.1% 36.7% Total Count 166 176 150 % within IV 64.6% 71.0% 77.7% % within DV 100.0% 100.0% 100.0% Submits reports to immediate superiors Base 2011 Closure 2011 Sex Male Count 98 108 % within IV 72.6% 81.2% % within DV 59.0% 55.4% Female Count 68 87 % within IV 55.7% 69.6% % within DV 41.0% 44.6% Total Count 166 195 % within IV 64.6% 75.6% % within DV 100.0% 100.0% Submit reports to immediate superiors BASE 2009 SURVEY 2009 CLOSURE 2009 Región X Metropolitan Count 56 51 35 % within IV 71.8% 65.4% 72.9% % within DV 35.0% 32.7% 34.0% North Count 23 21 9 % within IV 74.2% 67.7% 69.2% % within DV 14.4% 13.5% 8.7% Northwest Count 16 18 9 % within IV 61.5% 69.2% 90.0% % within DV 10.0% 11.5% 8.7% South Count 24 20 15 % within IV 75.0% 62.5% 71.4% % within DV 15.0% 12.8% 14.6% East-Northeast Count 41 46 35 % within IV 71.9% 80.7% 76.1% % within DV 25.6% 29.5% 34.0% Total Count 160 156 103 % within IV 71.4% 69.6% 74.6% % within DV 100.0% 100.0% 100.0% Submits reports to immediate superiors BASE 2010 SURVEY 2010 CLOSURE 2010 Región X Metropolitan Count 49 54 44 % within IV 19.1% 74.0% 75.9% % within DV 29.5% 30.7% 29.3% North Count 19 38 24 % within IV 7.4% 76.0% 85.7% % within DV 11.4% 21.6% 16.0% Northwest Count 20 21 15 % within IV 7.8% 56.8% 75.0% % within DV 12.0% 11.9% 10.0% South Count 32 25 27 % within IV 12.5% 64.1% 69.2% % within DV 19.3% 14.2% 18.0% East-Northeast Count 46 38 40 % within IV 17.9% 77.6% 83.3% % within DV 27.7% 21.6% 26.7% Total Count 166 176 150 % within IV 64.6% 71.0% 77.7% % within DV 100.0% 100.0% 100.0% Submits reports to immediate superiors BASE 2011 CLOSURE 2011 Región X Metropolitan Count 49 50 % within IV 75.4% 78.1% % within DV 29.5% 25.6% North Count 19 22 % within IV 55.9% 64.7% % within DV 11.4% 11.3% Northwest Count 20 28 % within IV 52.6% 71.8% % within DV 12.0% 14.4% South Count 32 36 % within IV 66.7% 73.5% % within DV 19.3% 18.5% East-Northeast Count 46 59 % within IV 63.9% 81.9% % within DV 27.7% 30.3% Total Count 166 195 % within IV 64.6% 75.6% % within DV 100.0% 100.0% Submit reports to immediate superiors Type of group Base 2009 Survey 2009 Closure 2009 Treatment Count 70 69 58 % within IV 60.9% 60.0% 69.0% % within DV 100.0% 100.0% 100.0% Control Count 60 62 18 % within IV 55.0% 56.9% 33.3% % within DV 100.0% 100.0% 100.0% Submits reports to members under his/her leadership Type of group Base 2010 Survey 2010 Closure 2010 Treatment Count 64 88 78 % within IV 52.0% 71.5% 67.8% % within DV 50.0% 54.7% 61.4% Control Count 64 73 49 % within IV 51.2% 58.4% 62.8% % within DV 50.0% 45.3% 38.6% Submits reports to members under his/her leadership Type of group Base 2011 Closure 2011 Treatment Count 60 78 % within IV 48.8% 62.9% % within DV 100.0% 100.0% Control Count 74 78 % within IV 54.8% 58.2% % within DV 100.0% 100.0% Submits reports to members under his/her leadership Base 2009 Survey 2009 Closure 2009 Sex Male Count 83 79 48 % within IV 55.3% 56.0% 54.5% % within DV 63.8% 60.3% 63.2% Female Count 47 52 28 % within IV 63.5% 62.7% 56.0% % within DV 36.2% 39.7% 36.8% Total Count 130 131 76 % within IV 58.0% 58.5% 55.1% % within DV 100.0% 100.0% 100.0% Submits reports to members under his/her leadership Base 2010 Survey 2010 Closure 2010 Sex Male Count 80 108 86 % within IV 31.0% 67.1% 76.1% % within DV 59.7% 67.1% 67.7% Female Count 54 53 41 % within IV 20.9% 60.9% 51.3% % within DV 40.3% 32.9% 32.3% Total Count 134 161 127 % within IV 51.9% 64.9% 65.8% % within DV 100.0% 100.0% 100.0% Submits reports to members under his/her leadership Base 2011 Closure 2011 Sex Male Count 80 90 % within IV 58.8% 67.7% % within DV 59.7% 57.7% Female Count 54 66 % within IV 44.3% 52.8% % within DV 40.3% 42.3% Total Count 134 156 % within IV 51.9% 60.5% % within DV 100.0% 100.0% Submits reports to members under his/her leadership BASE 2009 SURVEY 2009 CLOSURE 2009 Región X Metropolitan Count 41 43 22 % within IV 52.6% 55.1% 45.8% % within DV 31.5% 32.8% 28.9% North Count 27 19 10 % within IV 87.1% 61.3% 76.9% % within DV 20.8% 14.5% 13.2% Northwest Count 12 15 4 % within IV 46.2% 57.7% 40.0% % within DV 9.2% 11.5% 5.3% South Count 15 16 10 % within IV 46.9% 50.0% 47.6% % within DV 11.5% 12.2% 13.2% East-Northeast Count 35 38 30 % within IV 61.4% 66.7% 65.2% % within DV 26.9% 29.0% 39.5% Total Count 130 131 76 % within IV 58.0% 58.5% 55.1% % within DV 100.0% 100.0% 100.0% Submits reports to members under his/her leadership Base 2010 Survey 2010 Closure 2010 Región X Metropolitan Count 40 47 33 % within IV 15.5% 64.4% 56.9% % within DV 29.9% 29.2% 26.0% North Count 13 36 18 % within IV 5.0% 72.0% 64.3% % within DV 9.7% 22.4% 14.2% Northwest Count 20 18 13 % within IV 7.8% 48.6% 65.0% % within DV 14.9% 11.2% 10.2% South Count 27 25 30 % within IV 10.5% 64.1% 76.9% % within DV 20.1% 15.5% 23.6% East-Northeast Count 34 35 33 % within IV 13.2% 71.4% 68.8% % within DV 25.4% 21.7% 26.0% Total Count 134 161 127 % within IV 51.9% 64.9% 65.8% % within DV 100.0% 100.0% 100.0% Submits reports to members under his/her leadership BASE 2011 CLOSURE 2011 Región X Metropolitan Count 40 47 % within IV 61.5% 73.4% % within DV 29.9% 30.1% North Count 13 20 % within IV 38.2% 58.8% % within DV 9.7% 12.8% Northwest Count 20 18 % within IV 51.3% 46.2% % within DV 14.9% 11.5% South Count 27 23 % within IV 56.3% 46.9% % within DV 20.1% 14.7% East-Northeast Count 34 48 % within IV 47.2% 66.7% % within DV 25.4% 30.8% Total Count 134 156 % within IV 51.9% 60.5% % within DV 100.0% 100.0% Submits reports to members under his/her leadership 5. Indicators Indicator 10: Interparty participation Indicator 10: Interparty participation BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Count 52 49 54 % within IV 45.2% 42.6% 64.3% % within DV 100.0% 100.0% 100.0% Control Count 49 33 16 % within IV 45.0% 30.3% 29.6% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Base 2010 Survey 2010 Closure 2010 Treatment Count 37 37 61 % within IV 29.8% 30.1% 62.9% % within DV 100.0% 100.0% 100.0% Control Count 39 39 18 % within IV 31.5% 31.2% 28.1% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Base 2011 Closure 2011 Treatment Count 40 64 % within IV 31.0% 49.6% % within DV 100.0% 100.0% Control Count 51 47 % within IV 37.8% 34.3% % within DV 100.0% 100.0% Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 36 25 25 % within IV 52.9% 41.0% 51.0% % within DV 69.2% 51.0% 46.3% Female Count 16 24 29 % within IV 34.0% 44.4% 82.9% % within DV 30.8% 49.0% 53.7% Total Count 52 49 54 % within IV 45.2% 42.6% 64.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 38 26 12 % within IV 46.3% 32.5% 30.8% % within DV 77.6% 78.8% 75.0% Female Count 11 7 4 % within IV 40.7% 24.1% 26.7% % within DV 22.4% 21.2% 25.0% Total Count 49 33 16 % within IV 45.0% 30.3% 29.6% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 21 52 45 % within IV 27.3% 67.5% 67.2% % within DV 56.8% 63.4% 63.4% Female Count 16 30 26 % within IV 34.8% 65.2% 54.2% % within DV 43.2% 36.6% 36.6% Total Count 37 82 71 % within IV 30.1% 66.7% 61.7% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 31 32 19 % within IV 36.9% 38.1% 41.3% % within DV 79.5% 69.6% 70.4% Female Count 8 14 8 % within IV 19.5% 34.1% 25.0% % within DV 20.5% 30.4% 29.6% Total Count 39 46 27 % within IV 31.2% 36.8% 34.6% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Type of group Base 2011 Closure 2011 Treatment Sex Male Count 14 30 % within IV 22.2% 46.2% % within DV 38.9% 50.0% Female Count 22 30 % within IV 36.7% 50.8% % within DV 61.1% 50.0% Total Count 36 60 % within IV 29.3% 48.4% % within DV 100.0% 100.0% Control Sex Male Count 28 29 % within IV 38.4% 42.6% % within DV 53.8% 61.7% Female Count 24 18 % within IV 38.7% 27.3% % within DV 46.2% 38.3% Total Count 52 47 % within IV 38.5% 35.1% % within DV 100.0% 100.0% Indicator 10: Interparty participation Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Región X MetropolitanCount 8 16 4 % within IV 36.4% 72.7% 22.2% % within DV 12.7% 24.2% 13.3% North Count 9 9 3 % within IV 47.4% 47.4% 37.5% % within DV 14.3% 13.6% 10.0% Northwest Count 12 10 4 % within IV 63.2% 52.6% 40.0% % within DV 19.0% 15.2% 13.3% South Count 8 14 5 % within IV 42.1% 73.7% 35.7% % within DV 12.7% 21.2% 16.7% East-Northeast Count 26 17 14 % within IV 72.2% 47.2% 41.2% % within DV 41.3% 25.8% 46.7% Total Count 63 66 30 % within IV 54.8% 57.4% 35.7% % within DV 100.0% 100.0% 100.0% Control Región X MetropolitanCount 25 42 19 % within IV 44.6% 75.0% 63.3% % within DV 41.7% 55.3% 50.0% North Count 7 9 4 % within IV 58.3% 75.0% 80.0% % within DV 11.7% 11.8% 10.5% Northwest Count 5 5 0 % within IV 71.4% 71.4% 0.0% % within DV 8.3% 6.6% 0.0% South Count 8 10 6 % within IV 61.5% 76.9% 85.7% % within DV 13.3% 13.2% 15.8% East-Northeast Count 15 10 9 % within IV 71.4% 47.6% 75.0% % within DV 25.0% 13.2% 23.7% Total Count 60 76 38 % within IV 55.0% 69.7% 70.4% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 9 14 17 % within IV 45.0% 70.0% 81.0% % within DV 24.3% 17.1% 23.9% North Count 3 13 6 % within IV 13.6% 59.1% 40.0% % within DV 8.1% 15.9% 8.5% Northwest Count 3 11 8 % within IV 14.3% 52.4% 57.1% % within DV 8.1% 13.4% 11.3% South Count 5 18 13 % within IV 23.8% 85.7% 59.1% % within DV 13.5% 22.0% 18.3% East-Northeast Count 17 26 27 % within IV 43.6% 66.7% 62.8% % within DV 45.9% 31.7% 38.0% Total Count 37 82 71 % within IV 30.1% 66.7% 61.7% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 19 22 13 % within IV 35.8% 41.5% 35.1% % within DV 48.7% 47.8% 48.1% North Count 6 8 4 % within IV 21.4% 28.6% 30.8% % within DV 15.4% 17.4% 14.8% Northwest Count 4 5 1 % within IV 25.0% 31.3% 16.7% % within DV 10.3% 10.9% 3.7% South Count 3 5 7 % within IV 16.7% 27.8% 41.2% % within DV 7.7% 10.9% 25.9% East-Northeast Count 7 6 2 % within IV 70.0% 60.0% 40.0% % within DV 17.9% 13.0% 7.4% Total Count 39 46 27 % within IV 31.2% 36.8% 34.6% % within DV 100.0% 100.0% 100.0% Indicator 10: Interparty participation Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 6 9 % within IV 31.6% 47.4% % within DV 16.7% 15.0% North Count 7 10 % within IV 38.9% 55.6% % within DV 19.4% 16.7% Northwest Count 7 5 % within IV 31.8% 22.7% % within DV 19.4% 8.3% South Count 7 12 % within IV 35.0% 60.0% % within DV 19.4% 20.0% East-Northeast Count 9 24 % within IV 20.5% 53.3% % within DV 25.0% 40.0% Total Count 36 60 % within IV 29.3% 48.4% % within DV 100.0% 100.0% Control Región X Metropolitan Count 19 20 % within IV 41.3% 44.4% % within DV 36.5% 42.6% North Count 15 10 % within IV 93.8% 62.5% % within DV 28.8% 21.3% Northwest Count 4 3 % within IV 23.5% 17.6% % within DV 7.7% 6.4% South Count 7 6 % within IV 25.0% 20.7% % within DV 13.5% 12.8% East-Northeast Count 7 8 % within IV 25.0% 29.6% % within DV 13.5% 17.0% Total Count 52 47 % within IV 38.5% 35.1% % within DV 100.0% 100.0% Indicator 10: Interparty participation Indicator 11: Participation in elections Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Count 68 98 84 % within IV 59.1% 85.2% 100.0% % within DV 100.0% 100.0% 100.0% Control Count 62 84 51 % within IV 56.9% 77.1% 94.4% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group Base 2010 Survey 2010 Closure 2010 Treatment Count 102 101 97 % within IV 82.3% 82.1% 100.0% % within DV 100.0% 100.0% 100.0% Control Count 95 96 61 % within IV 76.6% 76.8% 95.3% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group Base 2011 Closure 2011 Treatment Count 94 129 % within IV 72.9% 100.0% % within DV 100.0% 100.0% Control Count 87 132 % within IV 64.4% 96.4% % within DV 100.0% 100.0% Indicator 11: Participation in elections Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 48 54 49 % within IV 70.6% 88.5% 100.0% % within DV 70.6% 55.1% 58.3% Female Count 20 44 35 % within IV 42.6% 81.5% 100.0% % within DV 29.4% 44.9% 41.7% Total Count 68 98 84 % within IV 59.1% 85.2% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 54 63 38 % within IV 65.9% 78.8% 97.4% % within DV 87.1% 75.0% 74.5% Female Count 8 21 13 % within IV 29.6% 72.4% 86.7% % within DV 12.9% 25.0% 25.5% Total Count 62 84 51 % within IV 56.9% 77.1% 94.4% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group BASE 2010 ENCUESTA 2010 CIERRE 2010 Treatment Sex Male Count 62 67 67 % within IV 80.5% 87.0% 100.0% % within DV 62.0% 62.6% 58.3% Female Count 38 40 48 % within IV 82.6% 87.0% 100.0% % within DV 38.0% 37.4% 41.7% Total Count 100 107 115 % within IV 81.3% 87.0% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 61 58 46 % within IV 72.6% 69.0% 100.0% % within DV 63.5% 65.9% 59.7% Female Count 35 30 31 % within IV 85.4% 73.2% 96.9% % within DV 36.5% 34.1% 40.3% Total Count 96 88 77 % within IV 76.8% 70.4% 98.7% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group Base 2011 Closure 2011 Treatment Sex Male Count 45 65 % within IV 71.4% 100.0% % within DV 51.1% 52.4% Female Count 43 59 % within IV 71.7% 100.0% % within DV 48.9% 47.6% Total Count 88 124 % within IV 71.5% 100.0% % within DV 100.0% 100.0% Control Sex Male Count 50 68 % within IV 68.5% 100.0% % within DV 58.1% 51.5% Female Count 36 64 % within IV 58.1% 97.0% % within DV 41.9% 48.5% Total Count 86 132 % within IV 63.7% 98.5% % within DV 100.0% 100.0% Indicator 11: Participation in elections Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Región X Metropolitan Count 10 19 18 % within IV 45.5% 86.4% 100.0% % within DV 14.7% 19.4% 21.4% North Count 11 13 8 % within IV 57.9% 68.4% 100.0% % within DV 16.2% 13.3% 9.5% Northwest Count 6 18 10 % within IV 31.6% 94.7% 100.0% % within DV 8.8% 18.4% 11.9% South Count 15 17 14 % within IV 78.9% 89.5% 100.0% % within DV 22.1% 17.3% 16.7% East-Northeast Count 26 31 34 % within IV 72.2% 86.1% 100.0% % within DV 38.2% 31.6% 40.5% Total Count 68 98 84 % within IV 59.1% 85.2% 100.0% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 33 40 28 % within IV 58.9% 71.4% 93.3% % within DV 53.2% 47.6% 54.9% North Count 4 7 5 % within IV 33.3% 58.3% 100.0% % within DV 6.5% 8.3% 9.8% Northwest Count 3 5 0 % within IV 42.9% 71.4% 0.0% % within DV 4.8% 6.0% 0.0% South Count 8 11 6 % within IV 61.5% 84.6% 85.7% % within DV 12.9% 13.1% 11.8% East-Northeast Count 14 21 12 % within IV 66.7% 100.0% 100.0% % within DV 22.6% 25.0% 23.5% Total Count 62 84 51 % within IV 56.9% 77.1% 94.4% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 16 17 21 % within IV 80.0% 85.0% 100.0% % within DV 16.0% 15.9% 18.3% North Count 19 20 15 % within IV 86.4% 90.9% 100.0% % within DV 19.0% 18.7% 13.0% Northwest Count 15 17 14 % within IV 71.4% 81.0% 100.0% % within DV 15.0% 15.9% 12.2% South Count 20 17 22 % within IV 95.2% 81.0% 100.0% % within DV 20.0% 15.9% 19.1% East-Northeast Count 30 36 43 % within IV 76.9% 92.3% 100.0% % within DV 30.0% 33.6% 37.4% Total Count 100 107 115 % within IV 81.3% 87.0% 100.0% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 41 34 36 % within IV 77.4% 64.2% 97.3% % within DV 42.7% 38.6% 46.8% North Count 19 22 13 % within IV 67.9% 78.6% 100.0% % within DV 19.8% 25.0% 16.9% Northwest Count 11 8 6 % within IV 68.8% 50.0% 100.0% % within DV 11.5% 9.1% 7.8% South Count 15 16 17 % within IV 83.3% 88.9% 100.0% % within DV 15.6% 18.2% 22.1% East-Northeast Count 10 8 5 % within IV 100.0% 80.0% 100.0% % within DV 10.4% 9.1% 6.5% Total Count 96 88 77 % within IV 76.8% 70.4% 98.7% % within DV 100.0% 100.0% 100.0% Indicator 11: Participation in elections Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 13 19 % within IV 68.4% 100.0% % within DV 14.8% 15.3% North Count 8 18 % within IV 44.4% 100.0% % within DV 9.1% 14.5% Northwest Count 18 22 % within IV 81.8% 100.0% % within DV 20.5% 17.7% South Count 19 20 % within IV 95.0% 100.0% % within DV 21.6% 16.1% East-Northeast Count 30 45 % within IV 68.2% 100.0% % within DV 34.1% 36.3% Total Count 88 124 % within IV 71.5% 100.0% % within DV 100.0% 100.0% Control Región X Metropolitan Count 30 45 % within IV 65.2% 100.0% % within DV 34.9% 34.1% North Count 11 15 % within IV 68.8% 93.8% % within DV 12.8% 11.4% Northwest Count 8 17 % within IV 47.1% 100.0% % within DV 9.3% 12.9% South Count 22 29 % within IV 78.6% 100.0% % within DV 25.6% 22.0% East-Northeast Count 15 26 % within IV 53.6% 96.3% % within DV 17.4% 19.7% Total Count 86 132 % within IV 63.7% 98.5% % within DV 100.0% 100.0% Indicator 11: Participation in elections Indicator 12: Increases leadership level Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Count 54 40 28 % within IV 47.0% 34.8% 33.3% % within DV 100.0% 100.0% 100.0% Control Count 73 46 19 % within IV 67.0% 42.2% 35.2% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group Base 2010 Survey 2010 Closure 2010 Treatment Count 60 59 34 % within IV 48.4% 48.0% 35.1% % within DV 100.0% 100.0% 100.0% Control Count 54 54 17 % within IV 43.5% 43.2% 26.6% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group Base 2011 Closure 2011 Treatment Count 65 43 % within IV 50.4% 33.3% % within DV 100.0% 100.0% Control Count 64 52 % within IV 47.4% 38.0% % within DV 100.0% 100.0% Indicator 12: Increases leadership level Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Sex Male Count 25 18 12 % within IV 36.8% 29.5% 24.5% % within DV 46.3% 45.0% 42.9% Female Count 29 22 16 % within IV 61.7% 40.7% 45.7% % within DV 53.7% 55.0% 57.1% Total Count 54 40 28 % within IV 47.0% 34.8% 33.3% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 58 39 12 % within IV 70.7% 48.8% 30.8% % within DV 79.5% 84.8% 63.2% Female Count 15 7 7 % within IV 55.6% 24.1% 46.7% % within DV 20.5% 15.2% 36.8% Total Count 73 46 19 % within IV 67.0% 42.2% 35.2% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 37 27 20 % within IV 48.1% 35.1% 29.9% % within DV 62.7% 56.3% 50.0% Female Count 22 21 20 % within IV 47.8% 45.7% 41.7% % within DV 37.3% 43.8% 50.0% Total Count 59 48 40 % within IV 48.0% 39.0% 34.8% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 31 33 15 % within IV 36.9% 39.3% 32.6% % within DV 57.4% 68.8% 60.0% Female Count 23 15 10 % within IV 56.1% 36.6% 31.3% % within DV 42.6% 31.3% 40.0% Total Count 54 48 25 % within IV 43.2% 38.4% 32.1% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group Base 2011 Closure 2011 Treatment Sex Male Count 27 20 % within IV 42.9% 30.8% % within DV 42.2% 50.0% Female Count 37 20 % within IV 61.7% 33.9% % within DV 57.8% 50.0% Total Count 64 40 % within IV 52.0% 32.3% % within DV 100.0% 100.0% Control Sex Male Count 36 28 % within IV 49.3% 41.2% % within DV 56.3% 54.9% Female Count 28 23 % within IV 45.2% 34.8% % within DV 43.8% 45.1% Total Count 64 51 % within IV 47.4% 38.1% % within DV 100.0% 100.0% Indicator 12: Increases leadership level Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Región X Metropolitan Count 12 10 8 % within IV 54.5% 45.5% 44.4% % within DV 22.2% 25.0% 28.6% North Count 7 4 3 % within IV 36.8% 21.1% 37.5% % within DV 13.0% 10.0% 10.7% Northwest Count 11 7 3 % within IV 57.9% 36.8% 30.0% % within DV 20.4% 17.5% 10.7% South Count 7 2 2 % within IV 36.8% 10.5% 14.3% % within DV 13.0% 5.0% 7.1% East-Northeast Count 17 17 12 % within IV 47.2% 47.2% 35.3% % within DV 31.5% 42.5% 42.9% Total Count 54 40 28 % within IV 47.0% 34.8% 33.3% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 42 25 11 % within IV 75.0% 44.6% 36.7% % within DV 57.5% 54.3% 57.9% North Count 8 5 3 % within IV 66.7% 41.7% 60.0% % within DV 11.0% 10.9% 15.8% Northwest Count 6 3 0 % within IV 85.7% 42.9% 0.0% % within DV 8.2% 6.5% 0.0% South Count 6 5 3 % within IV 46.2% 38.5% 42.9% % within DV 8.2% 10.9% 15.8% East-Northeast Count 11 8 2 % within IV 52.4% 38.1% 16.7% % within DV 15.1% 17.4% 10.5% Total Count 73 46 19 % within IV 67.0% 42.2% 35.2% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group Base 2010 Survey 2010 Closure 2010 Treatment Región X Metropolitan Count 9 8 6 % within IV 45.0% 40.0% 28.6% % within DV 15.3% 16.7% 15.0% North Count 9 7 5 % within IV 40.9% 31.8% 33.3% % within DV 15.3% 14.6% 12.5% Northwest Count 15 14 5 % within IV 71.4% 66.7% 35.7% % within DV 25.4% 29.2% 12.5% South Count 11 8 7 % within IV 52.4% 38.1% 31.8% % within DV 18.6% 16.7% 17.5% East-Northeast Count 15 11 17 % within IV 38.5% 28.2% 39.5% % within DV 25.4% 22.9% 42.5% Total Count 59 48 40 % within IV 48.0% 39.0% 34.8% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 20 22 13 % within IV 37.7% 41.5% 35.1% % within DV 37.0% 45.8% 52.0% North Count 10 7 4 % within IV 35.7% 25.0% 30.8% % within DV 18.5% 14.6% 16.0% Northwest Count 11 10 5 % within IV 68.8% 62.5% 83.3% % within DV 20.4% 20.8% 20.0% South Count 11 5 0 % within IV 61.1% 27.8% 0.0% % within DV 20.4% 10.4% 0.0% East-Northeast Count 2 4 3 % within IV 20.0% 40.0% 60.0% % within DV 3.7% 8.3% 12.0% Total Count 54 48 25 % within IV 43.2% 38.4% 32.1% % within DV 100.0% 100.0% 100.0% Indicator 12: Increases leadership level Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 9 6 % within IV 47.4% 31.6% % within DV 14.1% 15.0% North Count 12 10 % within IV 66.7% 55.6% % within DV 18.8% 25.0% Northwest Count 10 7 % within IV 45.5% 31.8% % within DV 15.6% 17.5% South Count 13 6 % within IV 65.0% 30.0% % within DV 20.3% 15.0% East-Northeast Count 20 11 % within IV 45.5% 24.4% % within DV 31.3% 27.5% Total Count 64 40 % within IV 52.0% 32.3% % within DV 100.0% 100.0% Control Región X Metropolitan Count 19 17 % within IV 41.3% 37.8% % within DV 29.7% 33.3% North Count 7 4 % within IV 43.8% 25.0% % within DV 10.9% 7.8% Northwest Count 8 11 % within IV 47.1% 64.7% % within DV 12.5% 21.6% South Count 14 9 % within IV 50.0% 31.0% % within DV 21.9% 17.6% East-Northeast Count 16 10 % within IV 57.1% 37.0% % within DV 25.0% 19.6% Total Count 64 51 % within IV 47.4% 38.1% % within DV 100.0% 100.0% Indicator 12: Increases leadership level Indicator 13: Makes modernization proposals Type of group TOTAL Base 2009 Survey 2009 Closure 2009 Treatment Count 44 56 41 % within IV 38.3% 48.7% 48.8% % within DV 100.0% 100.0% 100.0% Control Count 50 38 18 % within IV 45.9% 34.9% 33.3% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group Base 2010 Survey 2010 Closure 2010 Treatment Count 48 48 45 % within IV 38.7% 39.0% 46.4% % within DV 100.0% 100.0% 100.0% Control Count 51 52 26 % within IV 41.1% 41.6% 40.6% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group Base 2011 Closure 2011 Treatment Count 27 58 % within IV 20.9% 45.0% % within DV 100.0% 100.0% Control Count 46 54 % within IV 34.1% 39.4% % within DV 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group Base 2009 Survey 2009 Closure 2009 Treatment Sex Male Count 30 27 26 % within IV 44.1% 44.3% 53.1% % within DV 68.2% 48.2% 63.4% Female Count 14 29 15 % within IV 29.8% 53.7% 42.9% % within DV 31.8% 51.8% 36.6% Total Count 44 56 41 % within IV 38.3% 48.7% 48.8% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 39 26 14 % within IV 47.6% 32.5% 35.9% % within DV 78.0% 68.4% 77.8% Female Count 11 12 4 % within IV 40.7% 41.4% 26.7% % within DV 22.0% 31.6% 22.2% Total Count 50 38 18 % within IV 45.9% 34.9% 33.3% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 38 38 30 % within IV 49.4% 49.4% 44.8% % within DV 79.2% 69.1% 56.6% Female Count 10 17 23 % within IV 21.7% 37.0% 47.9% % within DV 20.8% 30.9% 43.4% Total Count 48 55 53 % within IV 39.0% 44.7% 46.1% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 40 36 23 % within IV 47.6% 42.9% 50.0% % within DV 76.9% 78.3% 63.9% Female Count 12 10 13 % within IV 29.3% 24.4% 40.6% % within DV 23.1% 21.7% 36.1% Total Count 52 46 36 % within IV 41.6% 36.8% 46.2% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group Base 2011 Closure 2011 Treatment Sex Male Count 15 30 % within IV 23.8% 46.2% % within DV 62.5% 53.6% Female Count 9 26 % within IV 15.0% 44.1% % within DV 37.5% 46.4% Total Count 24 56 % within IV 19.5% 45.2% % within DV 100.0% 100.0% Control Sex Male Count 26 29 % within IV 35.6% 42.6% % within DV 56.5% 53.7% Female Count 20 25 % within IV 32.3% 37.9% % within DV 43.5% 46.3% Total Count 46 54 % within IV 34.1% 40.3% % within DV 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group BASE 2009 SURVEY 2009CLOSURE 2009 Treatment Región X Metropolitan Count 12 13 6 % within IV 54.5% 59.1% 33.3% % within DV 27.3% 23.2% 14.6% North Count 7 11 3 % within IV 36.8% 57.9% 37.5% % within DV 15.9% 19.6% 7.3% Northwest Count 5 3 5 % within IV 26.3% 15.8% 50.0% % within DV 11.4% 5.4% 12.2% South Count 4 9 8 % within IV 21.1% 47.4% 57.1% % within DV 9.1% 16.1% 19.5% East-Northeast Count 16 20 19 % within IV 44.4% 55.6% 55.9% % within DV 36.4% 35.7% 46.3% Total Count 44 56 41 % within IV 38.3% 48.7% 48.8% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 28 20 10 % within IV 50.0% 35.7% 33.3% % within DV 56.0% 52.6% 55.6% North Count 5 7 1 % within IV 41.7% 58.3% 20.0% % within DV 10.0% 18.4% 5.6% Northwest Count 2 2 0 % within IV 28.6% 28.6% 0.0% % within DV 4.0% 5.3% 0.0% South Count 4 5 2 % within IV 30.8% 38.5% 28.6% % within DV 8.0% 13.2% 11.1% East-Northeast Count 11 4 5 % within IV 52.4% 19.0% 41.7% % within DV 22.0% 10.5% 27.8% Total Count 50 38 18 % within IV 45.9% 34.9% 33.3% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group BASE 2010 SURVEY 2010 CLOSURE 2010 Treatment Región X Metropolitan Count 8 11 8 % within IV 40.0% 55.0% 38.1% % within DV 16.7% 20.0% 15.1% North Count 13 12 7 % within IV 59.1% 54.5% 46.7% % within DV 27.1% 21.8% 13.2% Northwest Count 7 6 5 % within IV 33.3% 28.6% 35.7% % within DV 14.6% 10.9% 9.4% South Count 5 8 13 % within IV 23.8% 38.1% 59.1% % within DV 10.4% 14.5% 24.5% East-Northeast Count 15 18 20 % within IV 38.5% 46.2% 46.5% % within DV 31.3% 32.7% 37.7% Total Count 48 55 53 % within IV 39.0% 44.7% 46.1% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 21 25 20 % within IV 39.6% 47.2% 54.1% % within DV 40.4% 54.3% 55.6% North Count 16 9 4 % within IV 57.1% 32.1% 30.8% % within DV 30.8% 19.6% 11.1% Northwest Count 3 3 3 % within IV 18.8% 18.8% 50.0% % within DV 5.8% 6.5% 8.3% South Count 8 4 5 % within IV 44.4% 22.2% 29.4% % within DV 15.4% 8.7% 13.9% East-Northeast Count 4 5 4 % within IV 40.0% 50.0% 80.0% % within DV 7.7% 10.9% 11.1% Total Count 52 46 36 % within IV 41.6% 36.8% 46.2% % within DV 100.0% 100.0% 100.0% Indicator 13: Makes modernization proposals Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 5 11 % within IV 26.3% 57.9% % within DV 20.8% 19.6% North Count 4 6 % within IV 22.2% 33.3% % within DV 16.7% 10.7% Northwest Count 3 9 % within IV 13.6% 40.9% % within DV 12.5% 16.1% South Count 4 10 % within IV 20.0% 50.0% % within DV 16.7% 17.9% East-Northeast Count 8 20 % within IV 18.2% 44.4% % within DV 33.3% 35.7% Total Count 24 56 % within IV 19.5% 45.2% % within DV 100.0% 100.0% Control Región X Metropolitan Count 20 17 % within IV 43.5% 37.8% % within DV 43.5% 31.5% North Count 2 9 % within IV 12.5% 56.3% % within DV 4.3% 16.7% Northwest Count 4 6 % within IV 23.5% 35.3% % within DV 8.7% 11.1% South Count 7 9 % within IV 25.0% 31.0% % within DV 15.2% 16.7% East-Northeast Count 13 13 % within IV 46.4% 48.1% % within DV 28.3% 24.1% Total Count 46 54 % within IV 34.1% 40.3% % within DV 100.0% 100.0% Indicator 13: Makes modernization proposals Indicator 14: Teaches political courses Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Count 12 28 20 % within IV 10.4% 24.3% 23.8% % within DV 100.0% 100.0% 100.0% Control Count 14 13 8 % within IV 12.8% 11.9% 14.8% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group Base 2010 Survey 2010 Closure 2010 Treatment Count 23 23 23 % within IV 18.5% 18.7% 23.7% % within DV 100.0% 100.0% 100.0% Control Count 25 26 11 % within IV 20.2% 20.8% 17.2% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group Base 2011 Closure 2011 Treatment Count 4 30 % within IV 3.1% 23.3% % within DV 100.0% 100.0% Control Count 15 22 % within IV 11.1% 16.1% % within DV 100.0% 100.0% Indicator 14: Teaches political courses Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Sex Male Count 10 16 11 % within IV 14.7% 26.2% 22.4% % within DV 83.3% 57.1% 55.0% Female Count 2 12 9 % within IV 4.3% 22.2% 25.7% % within DV 16.7% 42.9% 45.0% Total Count 12 28 20 % within IV 10.4% 24.3% 23.8% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 8 10 6 % within IV 9.8% 12.5% 15.4% % within DV 57.1% 76.9% 75.0% Female Count 6 3 2 % within IV 22.2% 10.3% 13.3% % within DV 42.9% 23.1% 25.0% Total Count 14 13 8 % within IV 12.8% 11.9% 14.8% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group Base 2010 Survey 2010 Closure 2010 Treatment Sex Male Count 21 21 14 % within IV 27.3% 27.3% 20.9% % within DV 91.3% 72.4% 53.8% Female Count 2 8 12 % within IV 4.3% 17.4% 25.0% % within DV 8.7% 27.6% 46.2% Total Count 23 29 26 % within IV 18.7% 23.6% 22.6% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 21 16 9 % within IV 25.0% 19.0% 19.6% % within DV 80.8% 84.2% 64.3% Female Count 5 3 5 % within IV 12.2% 7.3% 15.6% % within DV 19.2% 15.8% 35.7% Total Count 26 19 14 % within IV 20.8% 15.2% 17.9% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group Base 2011 Closure 2011 Treatment Sex Male Count 1 16 % within IV 1.6% 24.6% % within DV 33.3% 55.2% Female Count 2 13 % within IV 3.3% 22.0% % within DV 66.7% 44.8% Total Count 3 29 % within IV 2.4% 23.4% % within DV 100.0% 100.0% Control Sex Male Count 10 12 % within IV 13.7% 17.6% % within DV 66.7% 54.5% Female Count 5 10 % within IV 8.1% 15.2% % within DV 33.3% 45.5% Total Count 15 22 % within IV 11.1% 16.4% % within DV 100.0% 100.0% Indicator 14: Teaches political courses Type of group BASE 2009 SURVEY 2009 CLOSURE 2009 Treatment Región X Metropolitan Count 3 6 4 % within IV 13.6% 27.3% 22.2% % within DV 25.0% 21.4% 20.0% North Count 2 3 2 % within IV 10.5% 15.8% 25.0% % within DV 16.7% 10.7% 10.0% Northwest Count 3 3 2 % within IV 15.8% 15.8% 20.0% % within DV 25.0% 10.7% 10.0% South Count 0 3 4 % within IV 0.0% 15.8% 28.6% % within DV 0.0% 10.7% 20.0% East-Northeast Count 4 13 8 % within IV 11.1% 36.1% 23.5% % within DV 33.3% 46.4% 40.0% Total Count 12 28 20 % within IV 10.4% 24.3% 23.8% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 6 8 4 % within IV 10.7% 14.3% 13.3% % within DV 42.9% 61.5% 50.0% North Count 4 2 1 % within IV 33.3% 16.7% 20.0% % within DV 28.6% 15.4% 12.5% Northwest Count 1 1 0 % within IV 14.3% 14.3% 0.0% % within DV 7.1% 7.7% 0.0% South Count 0 0 0 % within IV 0.0% 0.0% 0.0% % within DV 0.0% 0.0% 0.0% East-Northeast Count 3 2 3 % within IV 14.3% 9.5% 25.0% % within DV 21.4% 15.4% 37.5% Total Count 14 13 8 % within IV 12.8% 11.9% 14.8% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group Base 2010 Encuesta 2010 Cierre 2010 Treatment Región X Metropolitan Count 4 5 2 % within IV 20.0% 25.0% 9.5% % within DV 17.4% 17.2% 7.7% North Count 8 5 5 % within IV 36.4% 22.7% 33.3% % within DV 34.8% 17.2% 19.2% Northwest Count 4 1 4 % within IV 19.0% 4.8% 28.6% % within DV 17.4% 3.4% 15.4% South Count 1 6 4 % within IV 4.8% 28.6% 18.2% % within DV 4.3% 20.7% 15.4% East-Northeast Count 6 12 11 % within IV 15.4% 30.8% 25.6% % within DV 26.1% 41.4% 42.3% Total Count 23 29 26 % within IV 18.7% 23.6% 22.6% % within DV 100.0% 100.0% 100.0% Control Región X Metropolitan Count 10 12 9 % within IV 18.9% 22.6% 24.3% % within DV 38.5% 63.2% 64.3% North Count 8 4 1 % within IV 28.6% 14.3% 7.7% % within DV 30.8% 21.1% 7.1% Northwest Count 1 0 2 % within IV 6.3% 0.0% 33.3% % within DV 3.8% 0.0% 14.3% South Count 4 1 2 % within IV 22.2% 5.6% 11.8% % within DV 15.4% 5.3% 14.3% East-Northeast Count 3 2 0 % within IV 30.0% 20.0% 0.0% % within DV 11.5% 10.5% 0.0% Total Count 26 19 14 % within IV 20.8% 15.2% 17.9% % within DV 100.0% 100.0% 100.0% Indicator 14: Teaches political courses Type of group BASE 2011 CLOSURE 2011 Treatment Región X Metropolitan Count 0 3 % within IV 0.0% 15.8% % within DV 0.0% 10.3% North Count 1 3 % within IV 5.6% 16.7% % within DV 33.3% 10.3% Northwest Count 0 4 % within IV 0.0% 18.2% % within DV 0.0% 13.8% South Count 0 6 % within IV 0.0% 30.0% % within DV 0.0% 20.7% East-Northeast Count 2 13 % within IV 4.5% 28.9% % within DV 66.7% 44.8% Total Count 3 29 % within IV 2.4% 23.4% % within DV 100.0% 100.0% Control Región X Metropolitan Count 5 8 % within IV 10.9% 17.8% % within DV 33.3% 36.4% North Count 0 1 % within IV 0.0% 6.3% % within DV 0.0% 4.5% Northwest Count 1 5 % within IV 5.9% 29.4% % within DV 6.7% 22.7% South Count 3 3 % within IV 10.7% 10.3% % within DV 20.0% 13.6% East-Northeast Count 6 5 % within IV 21.4% 18.5% % within DV 40.0% 22.7% Total Count 15 22 % within IV 11.1% 16.4% % within DV 100.0% 100.0% Indicator 14: Teaches political courses Indicator 15: Accountability Index Indicator 15: Accountability Index Type of group Base 2009 Survey 2009 Closure 2009 Treatment 40.1 37.5 45.2 Control 36.1 34.4 36.4 Indicator 15: Accountability Index Type of group Base 2010 Survey 2010 Closure 2010 Treatment 29.5 29.5 43.9 Control 34.6 34.6 31.2 Indicator 15: Accountability Index Type of group Base 2011 Closure 2011 Treatment 29.5 42.6 Control 34.6 39.8 Indicator 15: Accountability Index Type of group Base 2009 Survey 2009 Closure 2009 Male 40.6 34.7 45.1 Female 33.1 38.2 37.3 Indicator 15: Accountability Index Type of group Base 2010 Survey 2010 Closure 2010 Male 28.8 28.8 45.7 Female 36.8 36.8 38.6 Indicator 15: Accountability Index Type of group Base 2011 Closure 2011 Male 36.2 45.2 Female 27.6 36.7 Indicator 15: Accountability Index Region Base 2009 Survey 2009 Closure 2009 Metropolitan 34.56 33.42 36.04 North 45.58 35.02 45.63 Northwest 35.04 32.46 47.57 South 41.64 33.24 42.98 East-Northeast 38.48 43.27 45.42 Indicator 15: Accountability Index Region Base 2010 Survey 2010 Closure 2010 Metropolitan 35.16 35.16 35.57 North 27.67 27.67 46.37 Northwest 31.24 31.24 41.17 South 28.50 28.50 47.03 East-Northeast 33.04 33.04 46.43 Indicator 15: Accountability Index Region Base 2011 Closure 2011 Metropolitan 38.35 44.37 North 31.79 43.44 Northwest 29.07 37.17 South 33.16 35.64 East-Northeast 27.78 43.26 6. Knowledge Gain Exams Distributions 2009 Entry Exam Final Exam Original Scale Total Valid 250 250 250 Missing 0 0 0 51.15 79.500 78.22 .972 .5483 .300 51.00 80.000 79.00 47 80.0 78 15.368 8.6689 4.743 .058 -.255 -.931 .154 .154 .154 -.579 -.174 2.233 .307 .307 .307 Std. Deviation Skewness Std. Error of Skewness Kurtosis Std. Error of Kurtosis Statistics N Mean Std. Error of Mean Median Mode 2010 Entry Exam Final Exam Original Scale Total Valid 267 267 267 Missing 0 0 0 66.536 91.155 83.509251 .6076 .2934 .3401307 67.500 92.400 84.200000 65.0 90.0 79.3000a 9.9284 4.7942 5.5577813 -.438 -.474 -1.503 .149 .149 .149 .728 1.068 3.941 .297 .297 .297 Mode Std. Deviation Skewness Std. Error of Skewness Kurtosis Std. Error of Kurtosis Statistics N Mean Std. Error of Mean Median 2011 Entry Exam Final Exam Original Scale Total Valid 258 269 269 Missing 11 0 0 65.269380 90.954 82.958 .6881473 .3692 .3591 65.000000 91.000 83.700 62.5000 98.0 84.8 11.0532824 6.0549 5.8895 -1.074 -.730 -1.354 .152 .149 .149 4.556 .569 2.963 .302 .296 .296 Kurtosis Std. Error of Kurtosis Statistics N Mean Std. Error of Mean Median Mode Std. Deviation Skewness Std. Error of Skewness General Lineal Model: Index 1 x Sex x Region. Cohort 2009 Measure:Knowdege Gain Index 1 Dependent Variable 1 Entry Exam 2 Final Exam Within-Subjects Factors Value Label N Sex 1 Male 136 2 Female 114 Region 1 Metropolitan 48 2 North 41 3 Northwest 40 4 South 39 5 East 41 6 Northeast 41 Between-Subjects Factors Sex Region Mean Std. Deviation N Metropolitan 65.26 16.013 27 North 46.65 13.279 23 Northwest 53.26 16.599 19 South 51.35 10.329 23 East 56.39 12.619 23 Northeast 37.48 7.521 21 Total 52.29 15.586 136 Metropolitan 67.29 9.334 21 North 44.56 12.118 18 Northwest 54.10 7.449 21 South 39.00 13.609 16 East 56.17 11.516 18 Northeast 34.50 5.916 20 Total 49.79 15.059 114 Metropolitan 66.15 13.415 48 North 45.73 12.669 41 Northwest 53.70 12.482 40 South 46.28 13.143 39 East 56.29 11.998 41 Northeast 36.02 6.868 41 Total 51.15 15.368 250 Metropolitan 87.407 4.4658 27 North 76.304 7.7191 23 Northwest 75.789 5.8365 19 South 74.783 8.0451 23 East 87.391 6.5487 23 Northeast 77.619 4.6419 21 Total 80.257 8.3293 136 Metropolitan 89.286 4.5513 21 North 77.222 8.4405 18 Northwest 76.905 5.1177 21 South 70.000 8.3666 16 East 83.333 6.6421 18 Northeast 73.000 5.2315 20 Total 78.596 9.0110 114 Metropolitan 88.229 4.5534 48 North 76.707 7.9538 41 Northwest 76.375 5.4287 40 South 72.821 8.4132 39 East 85.610 6.8186 41 Northeast 75.366 5.4072 41 Total 79.500 8.6689 250 Final Exam Male Female Total Descriptive Statistics Entry Exam Male Female Total Tests of Within-Subjects Contrasts Measure:Knowdege Gain Source Index 1 Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Index 1 Linear 100312.840 1 100312.840 1346.545 .000 .850 1.000 Index 1 * Sex Linear 23.392 1 23.392 .314 .576 .001 .086 Index 1 * Region Linear 4233.819 5 846.764 11.366 .000 .193 1.000 Index 1 * Sex * Region Linear 386.253 5 77.251 1.037 .397 .021 .367 Error(Index 1) Linear 17730.162 238 74.496 Tests of Between-Subjects Effects Measure:Knowledge Gain Transformed Variable:Average Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Intercept 2061238.877 1 2061238.877 19227.677 .000 .988 1.000 Sex 504.787 1 504.787 4.709 .031 .019 .580 Region 28300.106 5 5660.021 52.798 .000 .526 1.000 Sex * Region 1463.161 5 292.632 2.730 .020 .054 .818 Error 25513.995 238 107.202 Knowledge Gain: Bonferroni Lower Bound Upper Bound North 15.968* 1.5569 .000 11.352 20.584 Northwest 12.150* 1.5674 .000 7.503 16.797 South 17.636* 1.5783 .000 12.956 22.316 East 6.236* 1.5569 .001 1.620 10.853 Northeast 21.492* 1.5569 .000 16.876 26.109 Metropolitan -15.968* 1.5569 .000 -20.584 -11.352 Northwest -3.818 1.6271 .297 -8.642 1.006 South 1.668 1.6376 1.000 -3.187 6.524 East -9.732* 1.6170 .000 -14.526 -4.937 Northeast 5.524* 1.6170 .011 .730 10.319 Metropolitan -12.150* 1.5674 .000 -16.797 -7.503 North 3.818 1.6271 .297 -1.006 8.642 South 5.486* 1.6475 .015 .601 10.371 East -5.914* 1.6271 .005 -10.738 -1.089 Northeast 9.342* 1.6271 .000 4.518 14.167 Metropolitan -17.636* 1.5783 .000 -22.316 -12.956 North -1.668 1.6376 1.000 -6.524 3.187 Northwest -5.486* 1.6475 .015 -10.371 -.601 East -11.400* 1.6376 .000 -16.256 -6.544 Northeast 3.856 1.6376 .290 -1.000 8.712 Metropolitan -6.236* 1.5569 .001 -10.853 -1.620 North 9.732* 1.6170 .000 4.937 14.526 Northwest 5.914* 1.6271 .005 1.089 10.738 South 11.400* 1.6376 .000 6.544 16.256 Northeast 15.256* 1.6170 .000 10.462 20.051 Metropolitan -21.492* 1.5569 .000 -26.109 -16.876 North -5.524* 1.6170 .011 -10.319 -.730 Northwest -9.342* 1.6271 .000 -14.167 -4.518 South -3.856 1.6376 .290 -8.712 1.000 East -15.256* 1.6170 .000 -20.051 -10.462 Northeast Metropolitan North Northwest South East Multiple Comparisons (I) Region (J) Región Mean Difference (I-J) Std. Error Sig. 95% Confidence Interval General Lineal Model: Index 1 x Sex x Region. Cohort 2010 Measure:Knowdege Gain Index 1 Dependent Variable 1 Entry Exam 2 Final Exam Within-Subjects Factors Value Label N Sex 1 Male 159 2 Female 108 Region 1 Metropolitan 48 2 North 46 3 Northwest 45 4 South 44 5 East 42 6 Northeast 42 Between-Subjects Factors Sex Region Mean Std. Deviation N Metropolitan 70.839 11.9808 31 North 67.593 9.6447 27 Northwest 68.111 6.7955 27 South 65.673 12.3604 26 East 68.523 7.0988 22 Northeast 69.038 9.4360 26 Total 68.365 9.8846 159 Metropolitan 62.176 14.5784 17 North 63.421 6.1920 19 Northwest 62.667 7.7308 18 South 60.833 8.4453 18 East 65.375 9.4686 20 Northeast 68.906 7.0692 16 Total 63.843 9.4066 108 Metropolitan 67.771 13.4722 48 North 65.870 8.5670 46 Northwest 65.933 7.5931 45 South 63.693 11.0823 44 East 67.024 8.3600 42 Northeast 68.988 8.5193 42 Total 66.536 9.9284 267 Metropolitan 90.887 3.0787 31 North 89.574 5.2270 27 Northwest 92.056 4.8957 27 South 93.173 5.1282 26 East 91.045 2.9549 22 Northeast 90.481 5.8008 26 Total 91.192 4.7128 159 Metropolitan 90.676 3.0667 17 North 90.989 3.9471 19 Northwest 91.556 4.3314 18 South 92.722 5.6262 18 East 88.200 6.2148 20 Northeast 92.969 4.4999 16 Total 91.100 4.9332 108 Metropolitan 90.813 3.0433 48 North 90.159 4.7449 46 Northwest 91.856 4.6339 45 South 92.989 5.2777 44 East 89.690 4.9436 42 Northeast 91.429 5.4242 42 Total 91.155 4.7942 267 Final Exam Male Female Total Descriptive Statistics Entry Exam Male Female Total Tests of Within-Subjects Contrasts Measure:Knowdege Gain Source Index 1 Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Index 1 Linear 80444.367 1 80444.367 1582.857 .000 .861 1.000 Index 1 * Sex Linear 613.181 1 613.181 12.065 .001 .045 .933 Index 1 * Region Linear 736.231 5 147.246 2.897 .015 .054 .844 Index 1 * Sex * Region Linear 203.147 5 40.629 .799 .551 .015 .286 Error(Index 1) Linear 12959.679 255 50.822 Tests of Between-Subjects Effects Measure:Knowledge Gain Transformed Variable:Average Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Intercept 3159584.472 1 3159584.472 48927.804 .000 .995 1.000 Sex 622.878 1 622.878 9.646 .002 .036 .872 Region 316.172 5 63.234 .979 .431 .019 .348 Sex * Region 380.967 5 76.193 1.180 .319 .023 .417 Error 16466.998 255 64.576 Knowledge Gain: Bonferroni Lower Bound Upper Bound North 1.278 1.1724 1.000 -2.197 4.752 Northwest .397 1.1791 1.000 -3.096 3.891 South .951 1.1860 1.000 -2.563 4.465 East .935 1.2006 1.000 -2.623 4.492 Northeast -.917 1.2006 1.000 -4.474 2.641 Metropolitan -1.278 1.1724 1.000 -4.752 2.197 Northwest -.880 1.1914 1.000 -4.411 2.650 South -.327 1.1982 1.000 -3.877 3.224 East -.343 1.2127 1.000 -3.936 3.250 Northeast -2.194 1.2127 1.000 -5.788 1.399 Metropolitan -.397 1.1791 1.000 -3.891 3.096 North .880 1.1914 1.000 -2.650 4.411 South .554 1.2047 1.000 -3.016 4.123 East .537 1.2191 1.000 -3.075 4.150 Northeast -1.314 1.2191 1.000 -4.926 2.299 Metropolitan -.951 1.1860 1.000 -4.465 2.563 North .327 1.1982 1.000 -3.224 3.877 Northwest -.554 1.2047 1.000 -4.123 3.016 East -.016 1.2258 1.000 -3.648 3.616 Northeast -1.867 1.2258 1.000 -5.500 1.765 Metropolitan -.935 1.2006 1.000 -4.492 2.623 North .343 1.2127 1.000 -3.250 3.936 Northwest -.537 1.2191 1.000 -4.150 3.075 South .016 1.2258 1.000 -3.616 3.648 Northeast -1.851 1.2400 1.000 -5.525 1.823 Metropolitan .917 1.2006 1.000 -2.641 4.474 North 2.194 1.2127 1.000 -1.399 5.788 Northwest 1.314 1.2191 1.000 -2.299 4.926 South 1.867 1.2258 1.000 -1.765 5.500 East 1.851 1.2400 1.000 -1.823 5.525 Northeast 95% Confidence Interval Metropolitan North Northwest South East Multiple Comparisons (I) Region (J) Región Mean Difference (I-J) Std. Error Sig. General Lineal Model: Index 1 x Sex x Region. Cohort 2011 Measure:Knowdege Gain Index 1 Dependent Variable 1 Entry Exam 2 Final Exam Within-Subjects FactorsValue Label N Sex 1 Male 126 2 Female 132 Region 1 Metropolitan 49 2 North 46 3 Northwest 46 4 South 35 5 East 41 6 Northeast 41 Between-Subjects Factors Sex Region Mean Std. Deviation N Metropolitan 61.876923 10.5937834 26 North 70.444444 10.5131176 27 Northwest 69.342105 7.1122989 19 South 70.735294 8.7840723 17 East 67.236842 7.8126827 19 Northeast 69.166667 8.9934617 18 Total 67.883333 9.6455316 126 Metropolitan 57.095652 11.9128456 23 North 71.473684 7.8977382 19 Northwest 64.166667 14.3111038 27 South 61.944444 8.4259559 18 East 60.340909 9.2355245 22 Northeast 62.608696 11.9534671 23 Total 62.774242 11.7501770 132 Metropolitan 59.632653 11.3714260 49 North 70.869565 9.4377697 46 Northwest 66.304348 12.0501168 46 South 66.214286 9.5750025 35 East 63.536585 9.1851716 41 Northeast 65.487805 11.1273805 41 Total 65.269380 11.0532824 258 Metropolitan 90.481 5.1585 26 North 89.519 5.3051 27 Northwest 92.237 5.2767 19 South 90.794 3.0211 17 East 86.611 5.2988 19 Northeast 96.528 4.2443 18 Total 90.862 5.5610 126 Metropolitan 88.370 5.2184 23 North 91.105 5.2694 19 Northwest 93.704 4.0697 27 South 93.361 3.1984 18 East 83.300 6.1215 22 Northeast 96.391 4.0112 23 Total 91.088 6.3446 132 Metropolitan 89.490 5.2416 49 North 90.174 5.2907 46 Northwest 93.098 4.6087 46 South 92.114 3.3323 35 East 84.834 5.9247 41 Northeast 96.451 4.0633 41 Total 90.978 5.9643 258 Final Exam Male Female Total Descriptive Statistics Entry Exam Male Female Total Tests of Within-Subjects Contrasts Measure:Knowdege Gain Source Index 1 Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Index 1 Linear 81721.214 1 81721.214 1439.811 .000 .854 1.000 Index 1 * Sex Linear 851.643 1 851.643 15.005 .000 .057 .971 Index 1 * Region Linear 2238.376 5 447.675 7.887 .000 .138 1.000 Index 1 * Sex * Region Linear 355.183 5 71.037 1.252 .286 .025 .441 Error(Index 1) Linear 13962.536 246 56.758 Tests of Between-Subjects Effects Measure:Knowledge Gain Transformed Variable:Average Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Intercept 3081491.440 1 3081491.440 42713.397 .000 .994 1.000 Sex 844.851 1 844.851 11.711 .001 .045 .926 Region 4125.079 5 825.016 11.436 .000 .189 1.000 Sex * Region 513.639 5 102.728 1.424 .216 .028 .498 Error 17747.286 246 72.143 Knowledge Gain: Bonferroni Lower Bound Upper Bound North -5.960515* 1.2330148 .000 -9.615345 -2.305684 Northwest -5.139862* 1.2330148 .001 -8.794693 -1.485032 South -4.603061* 1.3292021 .009 -8.543005 -.663118 East .375859 1.2712024 1.000 -3.392166 4.143883 Northeast -6.408288* 1.2712024 .000 -10.176312 -2.640263 Metropolitan 5.960515* 1.2330148 .000 2.305684 9.615345 Northwest .820652 1.2523321 1.000 -2.891438 4.532742 South 1.357453 1.3471408 1.000 -2.635663 5.350570 East 6.336373* 1.2899480 .000 2.512784 10.159962 Northeast -.447773 1.2899480 1.000 -4.271362 3.375816 Metropolitan 5.139862* 1.2330148 .001 1.485032 8.794693 North -.820652 1.2523321 1.000 -4.532742 2.891438 South .536801 1.3471408 1.000 -3.456315 4.529918 East 5.515721* 1.2899480 .000 1.692132 9.339310 Northeast -1.268425 1.2899480 1.000 -5.092014 2.555164 Metropolitan 4.603061* 1.3292021 .009 .663118 8.543005 North -1.357453 1.3471408 1.000 -5.350570 2.635663 Northwest -.536801 1.3471408 1.000 -4.529918 3.456315 East 4.978920* 1.3821789 .006 .881945 9.075894 Northeast -1.805226 1.3821789 1.000 -5.902201 2.291748 Metropolitan -.375859 1.2712024 1.000 -4.143883 3.392166 North -6.336373* 1.2899480 .000 -10.159962 -2.512784 Northwest -5.515721* 1.2899480 .000 -9.339310 -1.692132 South -4.978920* 1.3821789 .006 -9.075894 -.881945 Northeast -6.784146* 1.3264977 .000 -10.716074 -2.852219 Metropolitan 6.408288* 1.2712024 .000 2.640263 10.176312 North .447773 1.2899480 1.000 -3.375816 4.271362 Northwest 1.268425 1.2899480 1.000 -2.555164 5.092014 South 1.805226 1.3821789 1.000 -2.291748 5.902201 East 6.784146* 1.3264977 .000 2.852219 10.716074 Northeast 95% Confidence Interval Metropolitan North Northwest South East Multiple Comparisons (I) Region (J) Región Mean Difference (I￾J) Std. Error Sig. General Lineal Model: Index 2 x Sex x Region. Cohort 2009 Measure:Knowdege Gain Index 2 Dependent Variable 1 Entry Exam 2 Total Within-Subjects Factors Value Label N Sex 1 Male 136 2 Female 114 Region 1 Metropolitan 48 2 North 41 3 Northwest 40 4 South 39 5 East 41 6 Northeast 41 Between-Subjects Factors Sex Region Mean Std. Deviation N Metropolitan 65.26 16.013 27 North 46.65 13.279 23 Northwest 53.26 16.599 19 South 51.35 10.329 23 East 56.39 12.619 23 Northeast 37.48 7.521 21 Total 52.29 15.586 136 Metropolitan 67.29 9.334 21 North 44.56 12.118 18 Northwest 54.10 7.449 21 South 39.00 13.609 16 East 56.17 11.516 18 Northeast 34.50 5.916 20 Total 49.79 15.059 114 Metropolitan 66.15 13.415 48 North 45.73 12.669 41 Northwest 53.70 12.482 40 South 46.28 13.143 39 East 56.29 11.998 41 Northeast 36.02 6.868 41 Total 51.15 15.368 250 Metropolitan 78.56 7.668 27 North 77.52 6.178 23 Northwest 77.05 3.045 19 South 76.78 3.977 23 East 80.26 3.306 23 Northeast 78.10 2.844 21 Total 78.09 5.068 136 Metropolitan 81.29 5.676 21 North 78.56 4.706 18 Northwest 77.67 2.781 21 South 74.88 3.442 16 East 79.44 3.468 18 Northeast 77.80 2.895 20 Total 78.39 4.341 114 Metropolitan 79.75 6.936 48 North 77.98 5.538 41 Northwest 77.38 2.888 40 South 76.00 3.839 39 East 79.90 3.360 41 Northeast 77.95 2.837 41 Total 78.22 4.743 250 Final Exam Male Female Total Descriptive Statistics Entry Exam Male Female Total Tests of Within-Subjects Contrasts Measure:Knowdege Gain Source Index 2 Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Index 2 Linear 93901.196 1 93901.196 1281.328 .000 .843 1.000 Index 2 * Sex Linear 222.203 1 222.203 3.032 .083 .013 .411 Index 2 * Region Linear 10092.530 5 2018.506 27.544 .000 .367 1.000 Index 2 * Sex * Region Linear 403.689 5 80.738 1.102 .360 .023 .390 Error(Index 2) Linear 17441.657 238 73.284 Tests of Between-Subjects Effects Measure:Knowledge Gain Transformed Variable:Average Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Observed Powera Intercept 2031800.867 1 2031800.867 22923.696 .000 .990 1.000 Sex 153.699 1 153.699 1.734 .189 .007 .259 Region 14525.333 5 2905.067 32.776 .000 .408 1.000 Sex * Region 1054.657 5 210.931 2.380 .039 .048 .753 Error 21094.705 238 88.633 Knowledge Gain: Bonferroni Lower Bound Upper Bound North 11.09* 1.416 .000 6.90 15.29 Northwest 7.41* 1.425 .000 3.18 11.64 South 11.81* 1.435 .000 7.55 16.06 East 4.85* 1.416 .011 .65 9.05 Northeast 15.96* 1.416 .000 11.76 20.16 Metropolitan -11.09* 1.416 .000 -15.29 -6.90 Northwest -3.68 1.479 .202 -8.07 .70 South .71 1.489 1.000 -3.70 5.13 East -6.24* 1.470 .000 -10.60 -1.88 Northeast 4.87* 1.470 .016 .51 9.23 Metropolitan -7.41* 1.425 .000 -11.64 -3.18 North 3.68 1.479 .202 -.70 8.07 South 4.40 1.498 .055 -.05 8.84 East -2.56 1.479 1.000 -6.95 1.83 Northeast 8.55* 1.479 .000 4.16 12.94 Metropolitan -11.81* 1.435 .000 -16.06 -7.55 North -.71 1.489 1.000 -5.13 3.70 Northwest -4.40 1.498 .055 -8.84 .05 East -6.96* 1.489 .000 -11.37 -2.54 Northeast 4.15 1.489 .086 -.26 8.57 Metropolitan -4.85* 1.416 .011 -9.05 -.65 North 6.24* 1.470 .000 1.88 10.60 Northwest 2.56 1.479 1.000 -1.83 6.95 South 6.96* 1.489 .000 2.54 11.37 Northeast 11.11* 1.470 .000 6.75 15.47 Metropolitan -15.96* 1.416 .000 -20.16 -11.76 North -4.87* 1.470 .016 -9.23 -.51 Northwest -8.55* 1.479 .000 -12.94 -4.16 South -4.15 1.489 .086 -8.57 .26 East -11.11* 1.470 .000 -15.47 -6.75 Northeast 95% Confidence Interval Metropolitan North Northwest South East Multiple Comparisons (I) Region (J) Región Mean Differen ce (I-J) Std. Error Sig. Bivariate correlations between exams 2009 Mean Std. Deviation N Entry Exam 51.15 15.368 250 Final Exam 79.50 8.669 250 Total 78.22 4.743 250 Descriptive Statistics 2009 Entry Exam Final Exam Total Pearson Correlation 1 .493** .227** Sig. (2-tailed) .000 .000 N 250 250 250 Pearson Correlation .493** 1 .513** Sig. (2-tailed) .000 .000 N 250 250 250 Pearson Correlation .227** .513** 1 Sig. (2-tailed) .000 .000 N 250 250 250 Correlations Entry Exam Final Exam Total 2010 Mean Std. Deviation N Entry Exam 66.54 9.928 267 Final Exam 91.15 4.794 267 Total 83.51 5.558 267 Descriptive Statistics 2010 Entry Exam Final Exam Total Pearson Correlation 1 .127* .277** Sig. (2-tailed) .038 .000 N 267 267 267 Pearson Correlation .127* 1 .308** Sig. (2-tailed) .038 .000 N 267 267 267 Pearson Correlation .277** .308** 1 Sig. (2-tailed) .000 .000 N 267 267 267 Correlations Entry Exam Final Exam Total 2011 Mean Std. Deviation N Entry Exam 65.27 11.053 258 Final Exam 90.95 6.055 269 Total 82.96 5.890 269 Descriptive Statistics 2011 Entry Exam Final Exam Total Pearson Correlation 1 .167** .194** Sig. (2-tailed) .007 .002 N 258 258 258 Pearson Correlation .167** 1 .469** Sig. (2-tailed) .007 .000 N 258 269 269 Pearson Correlation .194** .469** 1 Sig. (2-tailed) .002 .000 N 258 269 269 Entry Exam Final Exam Total Correlations 7. Evaluation Survey Q1. Sum of Promotions MEANS Male Female Treatment 0.60 0.65 Control 0.90 0.6 SUMS Male b1 Female b2 Total Treatment a1 27 42 69 Control a2 19 15 34 Total 46 57 103 B A Summary ANOVA Source SS df MS F P < A 9.361 1 9.361 29.044 0.001 B 0.925 1 0.925 2.869 0.025 A x B 2.759 1 2.759 8.559 ns S/AB 40.888 126.866 0.322 Total 53.932 129.866 Q2. Sum of Reform Proposals MEANS Male Female Treatment 0.78 0.82 Control 0.71 0.7 SUMS Male b1 Female b2 Total Treatment a1 35 53 88 Control a2 15 20 35 Total 50 73 123 B A Summary ANOVA Source SS df MS F P < A 21.465 1 21.465 53.823 0.001 B 4.042 1 4.042 10.136 0.025 A x B 1.291 1 1.291 3.238 ns S/AB 50.594 126.866 0.399 Total 77.393 129.866 Type of group Total Yes No Treatment Sex Male Count 14 14 28 % within IV 50.0% 50.0% 100.0% % within DV 33.3% 51.9% 40.6% Female Count 28 13 41 % within IV 68.3% 31.7% 100.0% % within DV 66.7% 48.1% 59.4% Total Count 42 27 69 % within IV 60.9% 39.1% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 7 6 13 % within IV 53.8% 46.2% 100.0% % within DV 46.7% 66.7% 54.2% Female Count 8 3 11 % within IV 72.7% 27.3% 100.0% % within DV 53.3% 33.3% 45.8% Total Count 15 9 24 % within IV 62.5% 37.5% 100.0% % within DV 100.0% 100.0% 100.0% Success of reform proposals Sum of Reform Approvals MEANS Male Female Treatment 0.93 0.89 Control 0.90 0.5 SUMS Male b1 Female b2 Total Treatment a1 42 58 100 Control a2 19 14 33 Total 61 72 133 B A Summary ANOVA Source SS df MS F P < A 34.302 1 34.302 81.749 0.001 B 0.925 1 0.925 2.204 ns A x B 3.370 1 3.370 8.031 0.01 S/AB 53.234 126.866 0.420 Total 91.831 129.866 Q3. Sum of Promoting Training MEANS Male Female Treatment 0.60 0.60 Control 0.67 0.7 SUMS Male b1 Female b2 Total Treatment a1 27 39 66 Control a2 14 19 33 Total 41 58 99 A Summary ANOVA Source SS df MS F P < A 8.322 1 8.322 25.623 0.001 B 2.208 1 2.208 6.800 0.025 A x B 0.374 1 0.374 1.153 ns S/AB 41.202 126.866 0.325 Total 52.106 129.866 Type of group Total Yes No Treatment Sex Male Count 13 12 25 % within IV 52.0% 48.0% 100.0% % within DV 34.2% 52.2% 41.0% Female Count 25 11 36 % within IV 69.4% 30.6% 100.0% % within DV 65.8% 47.8% 59.0% Total Count 38 23 61 % within IV 62.3% 37.7% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 7 4 11 % within IV 63.6% 36.4% 100.0% % within DV 41.2% 57.1% 45.8% Female Count 10 3 13 % within IV 76.9% 23.1% 100.0% % within DV 58.8% 42.9% 54.2% Total Count 17 7 24 % within IV 70.8% 29.2% 100.0% % within DV 100.0% 100.0% 100.0% Succes of training proposals Sum of Training Approvals MEANS Male Female Treatment 0.24 0.38 Control 0.38 0.6 SUMS Male b1 Female b2 Total Treatment a1 11 25 36 Control a2 8 15 23 Total 19 40 59 B A Summary ANOVA Source SS df MS F P < A 1.291 1 1.291 3.319 ns B 3.370 1 3.370 8.660 0.01 A x B 0.374 1 0.374 0.962 ns S/AB 49.364 126.866 0.389 Total 54.400 129.866 Type of group Total Yes No Treatment Sex Male Count 23 20 43 % within IV 53.5% 46.5% 100.0% % within DV 40.4% 40.8% 40.6% Female Count 34 29 63 % within IV 54.0% 46.0% 100.0% % within DV 59.6% 59.2% 59.4% Total Count 57 49 106 % within IV 53.8% 46.2% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 15 6 21 % within IV 71.4% 28.6% 100.0% % within DV 46.9% 40.0% 44.7% Female Count 17 9 26 % within IV 65.4% 34.6% 100.0% % within DV 53.1% 60.0% 55.3% Total Count 32 15 47 % within IV 68.1% 31.9% 100.0% % within DV 100.0% 100.0% 100.0% Q4. Participation in elections Type of group Total Yes No Treatment Sex Male Count 8 14 22 % within IV 36.4% 63.6% 100.0% % within DV 36.4% 43.8% 40.7% Female Count 14 18 32 % within IV 43.8% 56.3% 100.0% % within DV 63.6% 56.3% 59.3% Total Count 22 32 54 % within IV 40.7% 59.3% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 5 8 13 % within IV 38.5% 61.5% 100.0% % within DV 41.7% 61.5% 52.0% Female Count 7 5 12 % within IV 58.3% 41.7% 100.0% % within DV 58.3% 38.5% 48.0% Total Count 12 13 25 % within IV 48.0% 52.0% 100.0% % within DV 100.0% 100.0% 100.0% Candidate in elections Sum of Won Election MEANS Male Female Treatment 0.13 0.25 Control 0.38 0.3 SUMS Male b1 Female b2 Total Treatment a1 6 16 22 Control a2 8 8 16 Total 14 24 38 B A Summary ANOVA Source SS df MS F P < A 0.275 1 0.275 0.710 ns B 0.764 1 0.764 1.972 ns A x B 0.764 1 0.764 1.972 ns S/AB 49.162 126.866 0.388 Total 50.966 129.866 Dialogue Type of group Total No Yes Treatment Sex Male Count 18 27 45 % within IV 40.0% 60.0% 100.0% % within DV 50.0% 36.5% 40.9% Female Count 18 47 65 % within IV 27.7% 72.3% 100.0% % within DV 50.0% 63.5% 59.1% Total Count 36 74 110 % within IV 32.7% 67.3% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 4 17 21 % within IV 19.0% 81.0% 100.0% % within DV 26.7% 51.5% 43.8% Female Count 11 16 27 % within IV 40.7% 59.3% 100.0% % within DV 73.3% 48.5% 56.3% Total Count 15 33 48 % within IV 31.3% 68.8% 100.0% % within DV 100.0% 100.0% 100.0% Participation in discussions Type of group Total No Yes Treatment Sex Male Count 39 6 45 % within IV 86.7% 13.3% 100.0% % within DV 40.2% 46.2% 40.9% Female Count 58 7 65 % within IV 89.2% 10.8% 100.0% % within DV 59.8% 53.8% 59.1% Total Count 97 13 110 % within IV 88.2% 11.8% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 20 1 21 % within IV 95.2% 4.8% 100.0% % within DV 46.5% 20.0% 43.8% Female Count 23 4 27 % within IV 85.2% 14.8% 100.0% % within DV 53.5% 80.0% 56.3% Total Count 43 5 48 % within IV 89.6% 10.4% 100.0% % within DV 100.0% 100.0% 100.0% Has been an observer of Citizens Participation /OAS Type of group Total No Yes Treatment Sex Male Count 13 32 45 % within IV 28.9% 71.1% 100.0% % within DV 48.1% 38.6% 40.9% Female Count 14 51 65 % within IV 21.5% 78.5% 100.0% % within DV 51.9% 61.4% 59.1% Total Count 27 83 110 % within IV 24.5% 75.5% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 4 17 21 % within IV 19.0% 81.0% 100.0% % within DV 20.0% 60.7% 43.8% Female Count 16 11 27 % within IV 59.3% 40.7% 100.0% % within DV 80.0% 39.3% 56.3% Total Count 20 28 48 % within IV 41.7% 58.3% 100.0% % within DV 100.0% 100.0% 100.0% Member of a political/social network Type of group Total No Yes Treatment Sex Male Count 13 32 45 % within IV 28.9% 71.1% 100.0% % within DV 31.7% 46.4% 40.9% Female Count 28 37 65 % within IV 43.1% 56.9% 100.0% % within DV 68.3% 53.6% 59.1% Total Count 41 69 110 % within IV 37.3% 62.7% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 6 15 21 % within IV 28.6% 71.4% 100.0% % within DV 42.9% 44.1% 43.8% Female Count 8 19 27 % within IV 29.6% 70.4% 100.0% % within DV 57.1% 55.9% 56.3% Total Count 14 34 48 % within IV 29.2% 70.8% 100.0% % within DV 100.0% 100.0% 100.0% Coordination meetings Accountability Type of group Total No Yes Treatment Sex Male Count 19 26 45 % within IV 42.2% 57.8% 100.0% % within DV 43.2% 39.4% 40.9% Female Count 25 40 65 % within IV 38.5% 61.5% 100.0% % within DV 56.8% 60.6% 59.1% Total Count 44 66 110 % within IV 40.0% 60.0% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 6 15 21 % within IV 28.6% 71.4% 100.0% % within DV 33.3% 50.0% 43.8% Female Count 12 15 27 % within IV 44.4% 55.6% 100.0% % within DV 66.7% 50.0% 56.3% Total Count 18 30 48 % within IV 37.5% 62.5% 100.0% % within DV 100.0% 100.0% 100.0% Inform the community Type of group Total No Yes Treatment Sex Male Count 29 16 45 % within IV 64.4% 35.6% 100.0% % within DV 40.8% 41.0% 40.9% Female Count 42 23 65 % within IV 64.6% 35.4% 100.0% % within DV 59.2% 59.0% 59.1% Total Count 71 39 110 % within IV 64.5% 35.5% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 13 8 21 % within IV 61.9% 38.1% 100.0% % within DV 43.3% 44.4% 43.8% Female Count 17 10 27 % within IV 63.0% 37.0% 100.0% % within DV 56.7% 55.6% 56.3% Total Count 30 18 48 % within IV 62.5% 37.5% 100.0% % within DV 100.0% 100.0% 100.0% Make meetings Type of group Total No Yes Treatment Sex Male Count 18 27 45 % within IV 40.0% 60.0% 100.0% % within DV 39.1% 42.2% 40.9% Female Count 28 37 65 % within IV 43.1% 56.9% 100.0% % within DV 60.9% 57.8% 59.1% Total Count 46 64 110 % within IV 41.8% 58.2% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 11 10 21 % within IV 52.4% 47.6% 100.0% % within DV 45.8% 41.7% 43.8% Female Count 13 14 27 % within IV 48.1% 51.9% 100.0% % within DV 54.2% 58.3% 56.3% Total Count 24 24 48 % within IV 50.0% 50.0% 100.0% % within DV 100.0% 100.0% 100.0% Encourages its leaders Type of group Total No Yes Treatment Sex Male Count 26 19 45 % within IV 57.8% 42.2% 100.0% % within DV 37.7% 46.3% 40.9% Female Count 43 22 65 % within IV 66.2% 33.8% 100.0% % within DV 62.3% 53.7% 59.1% Total Count 69 41 110 % within IV 62.7% 37.3% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 16 5 21 % within IV 76.2% 23.8% 100.0% % within DV 45.7% 38.5% 43.8% Female Count 19 8 27 % within IV 70.4% 29.6% 100.0% % within DV 54.3% 61.5% 56.3% Total Count 35 13 48 % within IV 72.9% 27.1% 100.0% % within DV 100.0% 100.0% 100.0% Offer up public declarations Type of group Total No Yes Treatment Sex Male Count 35 10 45 % within IV 77.8% 22.2% 100.0% % within DV 37.6% 58.8% 40.9% Female Count 58 7 65 % within IV 89.2% 10.8% 100.0% % within DV 62.4% 41.2% 59.1% Total Count 93 17 110 % within IV 84.5% 15.5% 100.0% % within DV 100.0% 100.0% 100.0% Control Sex Male Count 21 0 21 % within IV 100.0% .0% 100.0% % within DV 44.7% .0% 43.8% Female Count 26 1 27 % within IV 96.3% 3.7% 100.0% % within DV 55.3% 100.0% 56.3% Total Count 47 1 48 % within IV 97.9% 2.1% 100.0% % within DV 100.0% 100.0% 100.0% File sworn statements of assets