anu Janu This prep Int Rig uary 31, 201 uary 2014 s publication w pared by Dev ternal ghts an 4 was produced vTech System Evalua nd Just d for review s, Inc. under ation o tice II by the United Contract No of USA (PRAJ d States Agen o. AID-OAA￾AID/Ca II) ncy for Intern M-11-00026. ambod ational Deve ia’s Pr lopment. It w ogram as on Photo Credit: LICADHO i PROGRAM CYCLE SERVICE CENTER INTERNAL EVALUATION OF USAID/CAMBODIA’S PROGRAM ON RIGHTS AND JUSTICE II (PRAJ II) January 31, 2014 Submitted to: Elizabeth Callender Contracting Officer’s Representative USAID/Policy, Planning, and Learning Authors: Nancy E. Horn, Ph.D., Team Leader Melissa Patsalides, USAID Washington Peoulida Ros, USAID Cambodia Phea Sat, USAID Cambodia Yoke Sudarbo, USAID Indonesia Submitted by: DevTech Systems, Inc. Contract AID-OAA-M-11-00026 DISCLAIMER: The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii ACKNOWLEDGEMENTS The evaluation team wants to thank the many individuals and stakeholder groups that provided the team with the time for interviews that resulted in this report: the United States Agency for International Development (USAID), the East-West Management Institute (EWMI), The Asia Foundation (TAF), the League for the Promotion and Defense of Human Rights (in Cambodia) (LICADHO), the Cambodian Human Rights and Development Association (ADHOC), the Cambodian Center for Human Rights (CCHR), the Samreth Law Group (SLG), the Vishnu Law Group (VSLG), Legal Aid of Cambodia (LAC), the Independent Democracy of Informal Economic Association (IDEA), Prey Lang, and the Phnom Penh Municipal Court (PPMC). While the team acknowledges that many more organizational representatives could have been interviewed, time allocated for this learning evaluation was insufficient to do so. The team wants to give special thanks to USAID/Cambodia, USAID/Indonesia, and USAID/Washington, Policy, Planning, and Learning Bureau/ Learning, Evaluation and Research Office (PPL/LER) for supporting this learning evaluation by providing the team members and funding to conduct the evaluation. Sustaining democratic reforms is not easy in any context. The team’s experience in Cambodia confirms that enduring change remains elusive. The team particularly wants to acknowledge the very strong commitment to reform and the unflinching stance that many of the sub-grantees have demonstrated in persevering to produce incremental change, for even these can lead to systemic reform. The PRAJ II Evaluation Team iii TABLE OF CONTENTS ACKNOWLEDGEMENTS ........................................................................................................... II ACRONYMS .................................................................................................................................. V EXECUTIVE SUMMARY........................................................................................................... VII 1. INTRODUCTION ..................................................................................................................... 1 Evaluation Purpose and Evaluation Questions ...................................................................................................................... 1 Team Learning Plan ..................................................................................................................................................................... 2 2. PROJECT BACKGROUND ...................................................................................................... 3 3. METHODOLOGY ..................................................................................................................... 5 Evaluation Team ........................................................................................................................................................................... 5 Evaluation Design Process ......................................................................................................................................................... 5 Limitations of the Evaluation ..................................................................................................................................................... 5 4. FINDINGS AND DISCUSSION .............................................................................................. 7 Project Design .............................................................................................................................................................................. 7 Project Implementation .............................................................................................................................................................. 9 Achievements Reported ............................................................................................................................................................ 9 5. CONCLUSIONS AND LESSONS LEARNED...................................................................... 29 Project Design and Implementation (Q2) ............................................................................................................................ 29 Degree to Which the Objectives Were Achieved (Q1) .................................................................................................. 29 Sustainability (Q3) ..................................................................................................................................................................... 30 6. RECOMMENDATIONS FOR FUTURE D&G PROGRAMMING ....................................... 31 Recommendations related to Project Design and Implementation (Q2) ...................................................................... 31 Recommendations to Overcome Hindrances or Leverage Achievements (Q1) ......................................................... 32 ANNEXES .................................................................................................................................... 35 ANNEX 1: STATEMENT OF WORK .................................................................................................................................. 36 ANNEX II: EWMI STATEMENT OF DIFFERENCES ........................................................................................................ 41 ANNEX III: USAID STATEMENT OF DIFFERENCES ...................................................................................................... 49 ANNEX IV: CHANGES IN PMP AND M&E INDICATORS ........................................................................................... 52 ANNEX V: TAF MONITORING AND EVALUATION REPORT FOR PRAJ II GRANTEES .................................. 57 ANNEX VI: SUMMARY OF GRANTS UNDER PRAJ II ................................................................................................... 73 ANNEX VII: DATA COLLECTION INSTRUMENTS ....................................................................................................... 76 ANNEX VIII: SCHEDULE OF DATA COLLECTION INTERVIEWS AND LEARNING ACTIVITIES .................. 87 ANNEX IX: BIBLIOGRAPHY ................................................................................................................................................. 91 ANNEX X: DISCLOSURE OF ANY CONFLICTS OF INTEREST ................................................................................ 93 iv TABLES TABLE 1: PRAJ II OVERALL REPORTED ACHIEVEMENTS ................................................................................. 9 TABLE 2: GRANTS MADE BY EWMI AND TAF IN SUPPORT OF HR NGOS ............................................ 13 TABLE 3: DATABASE “HITS” ..................................................................................................................................... 14 TABLE 4: CONTRIBUTORY FACTORS LEADING TO SUCCESSFUL CONSTRUCTION AND MAINTENANCE OF HR VIOLATIONS AND COURT DATABASE OUTCOMES ........................... 17 TABLE 5: GRANTS MADE BY EWMI AND TAF IN SUPPORT OF CBOS/CSOS/GNS.............................. 20 TABLE 6: OUTCOME ACHIEVED FROM GRANTS AWARDED BY TAF IN ACCORDANCE WITH USAID-DEFINED INDICATORS....................................................................................................................... 20 TABLE 7: CONTRIBUTORY FACTORS LEADING TO SUCCESSFUL CBO/CSO/GN OUTCOMES ... 24 TABLE 8: GRANTS MADE BY EWMI IN SUPPORT OF LEGAL AID AND PILFS ........................................ 25 TABLE 9: NUMBER OF CASES REPRESENTED AND NUMBER OF CLIENTS BENEFITTED BY LEGAL AID NGOS FROM 2008-2012 ........................................................................................................................... 26 TABLE 10: CONTRIBUTORY FACTORS LEADING TO POSITIVE LEGAL AID OUTCOMES ............... 28 TABLE 11: SAMPLE LOGIC MODEL IN FUTURE PROGRAMMING ............................................................... 31 v ACRONYMS ABA American Bar Association ABARLA American Bar Association Rule of Law Initiative AEC Action for Environment and Community ADHOC Cambodian Human Rights and Development Association AusAID Australian Agency for International Development BAKC Bar Association of the Kingdom of Cambodia BCV Building Community Voice CA Cooperative Agreement CBO Community Based Organizations CCC Cooperation Committee of Cambodia CCD Community Capacities for Development CCHR Cambodian Center for Human Rights CCJAP Cambodia Criminal Justice Assistance Program CDA Child Development Association CDP Cambodian Defenders Project CDPS Community Development for Peace and Sustainability (see CPN) CED Community Economic Development CEDAC Centre d’Etude et de Developpement Agricole Cambodgien CEELI Central European and Eurasian Law Initiative CHRAC Cambodian Human Rights Action Committee CLEC Community Legal Education Center CLJR Commission on Legal and Justice Reform COP Chief of Party CPN Community Peace Building Network (includes CDPS) CSO Civil Society Organization CTIP Cambodian Trafficking in Persons (database) CVS Cambodian Volunteers for Society CYN Cambodian Youth Network DANIDA Danish International Development Agency DG Democracy and Governance DKA Day Kou Aphiwat ECCC Extraordinary Chambers in the Courts of Cambodia EWMI East-West Management Institute FACT Fisheries Action Coalition Team FIT Financial Improvement Team GBV Gender-Based Violence GIZ German development assistance organization GN Grassroots Network HR Human Rights ICSO Indigenous Community Support Organization IBJ International Bridges to Justice IDEA Independent Democracy of Informal Economic Association IHRD International Human Rights Day IRAM Indigenous Rights Active Members Network IT Information Technology LAC Legal Aid of Cambodia LANGO Law on Associations vi LICADHO League for the Promotion and Defense of Human Rights (in Cambodia) LPD Law on Peaceful Demonstrations LSCW Legal Services for Children and Women MAFF Ministry of Agriculture, Forestry and Fisheries M&E Monitoring and Evaluation MOE Ministry of Environment MOI Ministry of Interior MOJ Ministry of Justice MOWA Ministry of Women’s Affairs NGO Non-Governmental Organization ODC Open Development Cambodia PILF Public Interest Law Firms PLCN Prey Lang Community Network PMP Performance Management Plan PPL/LER Policy, Planning, and Learning Bureau/ Learning, Evaluation and Research Office PPMC Phnom Penh Municipal Court PRA Participatory Rapid Appraisal PRAJ I First Phase of the Program on Rights and Justice PRAJ II Second Phase of the Program on Rights and Justice PUC Pannasastra University of Cambodia QR Quarterly Reports RAJP Royal Academy of Judicial Professionals RFA Request for Application RFP Request for Proposal RGC Royal Government of Cambodia RISC Returnee Integration Support Center ROL Rule of Law RULE Royal University of Law and Economics SIDA Swedish International Development Cooperation Agency SI Social Impact SLG Samreth Law Group SOW Scope of Work TAF The Asia Foundation TIP Trafficking in Persons TOR Terms of Reference USAID United States Agency for International Development USG United States Government VSLG Vishnu Law Group WMC Women’s Media Center of Cambodia vii EXECUTIVE SUMMARY The project evaluated is the United States Agency for International Development (USAID) Cambodia Program on Rights and Justice II (PRAJ II), currently being implemented by the East-West Management Institute (EWMI). The program began its first course in 2003 (as PRAJ I). Beginning in 2008, PRAJ II (a US $20 million program) has focused on civil society strengthening. Originally scheduled to close in September 2013, PRAJ II has been extended one year (with an additional two million dollars of funding). The dual purpose of the evaluation was to: 1) Provide critical information to USAID/Cambodia to inform the design and strategy of its next generation of Democracy and Governance (DG) programming; and 2) Develop the capacity of USAID staff in the field of evaluation by including them as members of the evaluation team. The approach to this internal evaluation emerged out of a USAID/Washington, Policy, Planning, and Learning Bureau/ Learning, Evaluation and Research Office (PPL/LER) initiative to build USAID staff capacity to better commission, manage and use evaluations by conducting one as part of an evaluation team. Three questions posed in the evaluation Scope of Work (SOW) guided this evaluation: 1. To what extent have these three key project objectives (of five) been achieved? What have been the main promoters or hindrances to the achievement of these objectives?  Objective 1: Improve Collection and Use of Justice System Data  Objective 4: Strengthen Cambodian Constituencies Supporting Justice Sector Reform  Objective 5: Strengthen Capacity of Legal Profession and Civil Society to Advise and Represent Poor Citizens 2. Have the key sections (as noted in Question 1) of the original project design been successfully addressed or has implementation demonstrated that these elements of the design were insufficient or incorrect suppositions?0F0F 1  The evaluation should focus on those key objectives listed in Question 1, and refer to the Request for Application (RFA) and its development hypotheses, and sub-hypothesis, for this question. 3.Which individual Non-Governmental Organizations (NGOs) or strategic partnerships between NGOs show evidence of sustainability, in terms of a) capacity to function effectively, b) having a constituency which is interested enough to maintain the organization, or c) financially sustainable (or moving in a direction of sustainability outside of sole-USAID grants)?  Are networks or partnerships, or constituency groups, cooperating or partnering effectively, and thus could be considered ‘sustainable’ in continuing as partnerships that achieve results, in terms of a) ability to work together on an issue, and b) achieving appreciable changes/actions with such partnerships? Project Background The aim of the PRAJ is to focus on building judicial reform and strengthening Cambodia’s society after a scarring history. While both iterations of PRAJ had similar objectives to create judicial reform, the focus changed considerably as a result of the mid-term evaluation; the Rule of Law (ROL) focus was dropped and building civil society was emphasized. Several new strategies were implemented, although no change was made to the Cooperative Agreement (CA). A new Performance Management Plan (PMP) was 1 In a subsequent discussion with USAID, it was agreed that Question 2 would be dropped or reworked due to the many changes the project underwent over the course of implementation. viii developed, although indicators were not sufficiently fine-tuned to capture the incremental changes1F1F 2 made by sub-grantees. In the last few years of project implementation, USAID directed the East-West Management Institute (EWMI) to: submit weekly reports (requested by the previous Ambassador), in addition to quarterly reports; develop an on-line platform to present information on illegal land concessions; and to support an organization addressing the needs of refugees being deported from the United States back to Cambodia. Team Members and Data Collection Methodology The team assembled to conduct this evaluation included: Nancy E. Horn, Ph.D., Team Leader, DevTech Consultant; Melissa Patsalides, USAID/Washington, PPL/LER; Peoulida Ros, USAID/Cambodia, Program Office; Phea Sat, USAID/Cambodia, Democracy & Governance (DG) Office; and Yoke Sudarbo, USAID/Indonesia, DG Office. Using a mixed methods approach, the team conducted a document review, interviewed sub-grantee leaders, government officials, and members of networks using a key informant interview, focus group interview, or participatory rapid appraisal (PRA) format. When interviews had to be conducted in Khmer, Ms. Ros and Mr. Sat took the lead; otherwise the lead was shared among the team members. Findings and Discussion Achievements. The evaluation team was able to document a number of achievements. Under Objective 1, several databases were developed: three documenting human rights violations,2F2F 3 one supportive of the expansion of the Trafficking in Persons (TIP) database, one supportive of the Gender-Based Violence (GBV) database, and a database focusing on criminal cases heard at the Phnom Penh Municipal Court (PPMC). The TIP and GBV database served as a template for the PPMC database (with modifications) into which data on pre-trial detention and time lapses in the issuing of final judgments (among many other variables) were entered. In the last two years, EWMI also supported the development of a database that tracked the perpetrators of illegal pharmaceutical sales. EWMI staff provided technical assistance for the development of these databases and purchased computers for use in the PPMC. Under Objective 4, approximately US $7 million was awarded in the small grants program: The Asia Foundation (TAF) supported grants to approximately 12 Human Rights NGOs and Community Based Organization (CBOs)/Civil Society Organizations (CSOs)/Grassroots Networks (GNs). EWMI reportedly supported programmatic grants to: two Human Rights NGOs; seven Public Interest Law Firms (PILFs) and Legal Aid NGOs; a media organization; a GN; and supported specific activities for several smaller organizations. TAF and EWMI created their own grants manuals and then followed their own internal procedures for selecting grantees. EWMI made the final selection of grantees and forwarded them to USAID for review. Once awarded, each grantee was provided grant management training. TAF also provided any technical assistance needed when conducting field visits. Each grantee was also responsible for submitting its own monitoring reports that included programmatic and financial information based on the TAF Monitoring & Evaluation (M&E) system. EWMI monitoring included some field visitation. Under Objective 5, EWMI provided grants to legal aid organizations and PILFs to expand and build capacity of lawyers to provide assistance to poor clients who were being detained under the range of existing laws in Cambodia – criminal, civil, business, human rights violations, etc. 2 Incremental changes include steps that can potentially lead to a particular PIRS (performance indicator reference sheet) defined indicator used to measure change during project implementation. 3 The databases were developed by the following organizations: League for the Promotion and Defense of Human Rights (in Cambodia) (LICADHO), the Cambodian Human Rights and Development Association (ADHOC), and the Cambodian Center for Human Rights (CCHR). ix Challenges. PRAJII was based on a theory of change that posited a central role of accessibility of information in generating judicial reform. When direct reform interventions with government were not as productive as envisioned, Objective 3 (focused on supporting reform at central levels within the Ministry of Justice and the Bar Association) was eliminated. The M&E framework was insufficient to capture the many incremental changes that were occurring by at the sub-grantees working at the provincial and local levels. PRAJ II leadership were required to complete many reporting requirements that limited available time to create linkages between objectives and among sub-grantees and affected the goal of enhanced peer learning. Due to security concerns, several database owners (Human Rights NGOs) could not share openly the information from their databases in order to not compromise the safety of those who provided the information. Constituency building and sustainability of sub-grantees are both challenges owing to their donor￾dependency. The legal aid sector is severely challenged by a shortage of funding as well as the low number of law school graduates who become practicing lawyers, thus limiting the number of individuals who can be represented in court cases or receive advice when accused. Recommendations As a result of the research conducted and the findings and conclusions/lessons learned reported, the team generated the following recommendations for USAID in its efforts to design the next DG project iteration and for EWMI to consider in how to address some outstanding issues (even though only a few months of project implementation remain): 1) Project Design and Implementation USAID is one player among many attempting to achieve democratic reform in Cambodia. Therefore, in the future the Mission would be well-served by forming an association/ coalition of actors seeking to create a more democratic society and/or assume a leadership role in the Technical Working Group (TWG) focused on this area. The TWG is a group of donors and implementers in the area of democracy and governance who meet monthly or quarterly to share common concerns. With the information shared on activities and results, the TWG could develop a systems perspective in planning and measurement, taking into account all the actors engaged and what their perspectives, relationships and contributions are to achieving reform. The results of forming this association/collaboration might be seen in the pooling of donor funds to support democratic reform. With a common planning and M&E approach, success might be more easily achieved through streamlined indicators across this sector’s evaluation field. This would facilitate capturing incremental change in a timely manner, as those receiving small grants could be responsible for writing one progress report per quarter. While some sub-grantees have expressed a desire to receive direct funding, others have not, largely due to the political ramifications of receiving direct funding from USAID. Pooled donor funding could reduce this stigma and facilitate greater sharing of data collected and entered into databases. Agreements would have to be reached, memoranda of understanding would have to be signed, and by-laws would have to be developed by members of this new association to assure collaboration and continuity. Such collaboration would also prevent widespread and time￾consuming competition in writing several proposals. Each potential sub-grantee would fill an informational niche (so as to prevent overlapping data collection) and would be responsible for collecting data within that domain. In order to support a network of local and international actors who can share their work and their reporting on results achieved in human rights, government reform and civil society, the Mission should x consider a plan for a greatly modified approach to M&E. The recommended approach would include fewer and more targeted performance indicators, and a more robust effort to capture incremental change and the contributions of various actors in the system. M&E approaches that recognize this reality, such as developmental evaluation, outcome harvesting, or contribution analysis (a modified form of which we have attempted in the findings on each of the Objectives above), among others, will be far better suited to helping the Mission understand what its contribution has been. One suggestion to capture these incremental changes is to develop a logic model, as presented below: A major finding is that the reporting burdens affected the ability of the implementer to execute programs. It is recommended that the Mission consider the information it needs for management decision making and limit its reporting requirements to no more (in frequency or content) than that. USAID should reconsider working with the Royal Government of Cambodia (RGC) at all levels (district, provincial and national) on different reform measures, by focusing greater attention on provincial and local government leaders (a network of “champions” for particular reforms) where collaborative inroads have already been made by NGOs/CBOs/ CSOs/GNs. Organizational sustainability is the result of many factors, including the following: 1) commitment to an organizational vision, mission and values; 2) the creation and following of systems and procedures; 3) good leadership and management; 4) political will; and 5) a sustainable source of funding (among others). Mobilizing and building the capacity of organizations and networks requires a long-term plan that meets an organization and its members where they currently are, ascertaining their strengths and weaknesses, building upon what they have through workshops and real-life exercises, and then coaching and mentoring them through a learning process – complete with time-related benchmarks – until the leaders and members of the organization become self-sustaining. Identifying skilled facilitators who know the Cambodian context (including its history and the culture of fear many Cambodians experience when considering how to address – if at all – human rights violations and land rights), are aware of cultural issues in organizations, and are armed with a toolkit of learner-centered activities would be critical in taking NGOs/CBOs/CSOs/GNs to the next level. Prior to any new initiative, considerable research and/or a baseline study should be undertaken to analyze the current political atmosphere and the potential for reform at all levels. This research is necessary to obtain an accurate reading of the RGC’s readiness and willingness to reform. While it may appear that members of the RGC and the courts are deeply entrenched in the status quo, the results of the most recent election indicated a change in the mind of the voting population such that the opposition won 26 more seats. Dissent and the willingness to change need to be measured at all levels and in all types of communities. Given the historical (and even current) use of military and police force, an accurate picture needs to be taken of where the Cambodian people are in terms of their willingness to act on their convictions. 2) Objective 1 USAID should consider further support of database development, expansion and data analysis by:  Ensuring that data obtained and entered are complementary to that of other organizations, and that information is shared – to the extent feasible – among database owners.  Supporting a marketing strategy for database products to reduce overwhelming donor dependency.  Advocating with the appropriate RGC entities for appropriate budget allocations to support database development and maintenance.  Supporting collaborative public/private partnerships to develop mutual appreciation for data, its storage, and its use in improving HUMAN RIGHTS in Cambodia. xi  Supporting the development of stronger linkages with international human rights NGOs and international media to ensure a continuous global focus on Cambodia.  Supporting training in transforming database entries into reports that suggest new policy directions and actions on the part of RGC entities.  Supporting the training of citizen journalists/witnesses capable of reporting human rights violations to human rights NGOs for database entry. 3) Objective 4 USAID should consider continuing the small grants program by:  Expanding the breadth of CBOs/CSOs/GNs receiving grant funding to create a wider base for the demand side of democratic reform (i.e., tapping into organizations funded by other donors and keeping abreast of new organizations’ missions as they develop).  Encouraging sub-grantees to find ways of collaborating with different levels and different agencies of the RGC when they make inroads toward reform. Ideas on how to do this should be generated by the organizations themselves in a forum established for this purpose.  Providing a planned program of learner-centered capacity building to sub-grantees to create strong, self-standing CBOs/CSOs/GNs capable of generating their own revenue.  Identifying participatory approaches to strengthening GNs that will not transform them into NGOs but rather help them to identify their own pattern of growth and development based in the mobilization of local resources.  Blending targeted income-generating strategies with democracy building activities among GNs so that they are capable of being self-supporting (as the Mission’s Supporting Forestry and Biodiversity project will be attempting to do this year).  Helping CBOs/CSOs/GNs to develop their own roadmap to success that includes expanded community participation, advocacy campaigns, livelihood development and other elements identified by the organizations themselves.  Supporting regularly-scheduled meetings among sub-grantees in different locations with learning, coaching and mentoring agendas. 4) Objective 5 USAID should consider the following recommendations to build a legal aid culture:  Offer scholarships to attend law school in return for two years of service to a Public Interest Law Firm (PILF) or legal aid NGO upon graduation.  Fund more internship opportunities for law school graduates to practice in a PILF.  Fund legal aid NGOs to teach would-be lawyers how to advocate for expanded legal aid support by RGC.  Strengthen the Bar Association of the Kingdom of Cambodia (BACK) referral system to remove obstacles preventing lawyers from participating fully in the legal aid system.  Support BAKC to create a system to monitor the quality of education provided to future lawyers (to address purchasing of degrees and to create “checks and balances” in legal education).  Provide continuous training on changes in the law and the legal system to PILFs and legal aid NGOs, as well as members of the court, so that appropriate laws are referenced in trials and appropriate systems and procedures are followed.  Support the creation of a legal aid database to monitor case flow that can be accessed by all lawyers defending the poor.  Support PILFs and legal aid NGOs who intend to take on private cases in terms of financial management and case handlings.  Human Rights NGOs should continue to take on legal aid cases to extend the legal aid culture. xii This page is intentionally left blank. 1 1. INTRODUCTION This learning evaluation of USAID/Cambodia’s Program on Rights and Justice II (PRAJ II) has two purposes: 1) Provide critical information to USAID/Cambodia to inform the design and strategy of its next generation of Democracy and Governance (DG) programming; and 2) Develop the capacity of USAID staff in the field of evaluation by including them as members of the evaluation team. Evaluation Purpose and Evaluation Questions The evaluation Scope of Work (SOW) [see Annex I] focused on three of PRAJ II’s original five objectives:3F3F 4 Question 1 (Q1). To what extent have these three key project objectives been achieved? What have been the main promoters or hindrances to the achievement of these objectives?  Objective 1: Improve Collection and Use of Justice System Data o 1.1 Improve NGO human rights monitoring and database efforts  Objective 4: Strengthen Cambodian Constituencies Supporting Justice Sector Reform o 4.1 Advocacy and land and livelihood rights training to rural communities o 4.2 Technical support and targeted grants for Community Peace Building Network and Human Rights NGOs o 4.3 Targeted NGO grants program in support of national level reform agenda o 4.4 Encourage public-private dialogue, cooperation, and engagement between NGO partners and government officials  Objective 5: Strengthen Capacity of Legal Profession and Civil Society to Advise and Represent Poor Citizens o 5.1 Support new Public Interest Law Firm o 5.2 Public awareness campaign for legal aid Question 2 (Q2). Have the key sections (as noted in Question 1) of the original project design been successfully addressed or has implementation demonstrated that these elements of the design were insufficient or incorrect suppositions?4F4F 5  The evaluation should focus on those key objectives listed in Question 1, and refer to the Request for Application (RFA) and its development hypotheses, and sub-hypothesis, for this question. Question 3 (Q3). Which individual Non-Governmental Organizations (NGOs) or strategic partnerships between NGOs show evidence of sustainability, in terms of a) capacity to function effectively, b) having a constituency which is interested enough to maintain the organization, or c) financially sustainable (or moving in a direction of sustainability outside of sole-USAID grants)? 4 USAID had dropped Objective 3, working directly with the Ministry of Justice (MOJ), due to low participation. The SOW did not include Objective 2, that dealt with improving legal education, largely because most of the work had been completed about the time of the mid-term evaluation and what little remained had been integrated into the other objectives. 5 In a subsequent discussion with USAID, it was agreed that Question 2 would be dropped or reworked due to the many changes the project underwent over the course of implementation. 2  Are networks or partnerships, or constituency groups, cooperating or partnering effectively, and thus could be considered ‘sustainable’ in continuing as partnerships that achieve results, in terms of a) ability to work together on an issue, and b) achieving appreciable changes/actions with such partnerships? The overall questions guiding this evaluation are:  What have been the outcomes of all the activities undertaken under the three objectives?  How have these contributed to reaching the strategic goal? Because democracy-building projects may not achieve their goals for many years, it is important to be able to document the incremental steps taken toward achieving project goals. Hence, a critical part of this evaluation was the identification of those incremental steps and how they contribute to the achievement of the project goal. Team Learning Plan Each member of the team had different levels of training and experience in evaluation. Hence, the learning process included, among other items: 1) an initial assessment of evaluation skills; 2) learning experiences assigned over conference calls, including the generation of questions to be posed to different sets of stakeholders; 3) field data collection; 4) daily learning debriefings; 5) learning days that focused on different topics; 6) analyzing data; 7) writing and revising the report; and 8) a final assessment. The dual purpose of the evaluation posed some challenges that are highlighted as appropriate in the findings and recommendations sections. 3 2. PROJECT BACKGROUND PRAJ II was supported through Cooperative Agreement (CA) No. 442-A-00-08-00011-00, awarded to the East-West Management Institute (EWMI) in October 2008. PRAJ II was an extension of PRAJ I, CA No. 442-A-00-03-00193-00, implemented from 2003 to 2008. The goal of both projects - to strengthen the foundation of support for reform of the justice sector in Cambodia – was implemented under an overall theory of change based on the integral role that information can play in judicial reform. Emerging from this theory were five objectives: 1) Database construction and data gathering on human rights violations; 2) Capacity building of future legal professionals; 3) Capacity building of Ministry of Justice (MOJ) departments; 4) Capacity building of civil society organizations; and 5) Expanding and equipping the legal aid sector to represent poor clients (in all cases ranging from criminal to civil to human rights abuses). A challenge existed in the development of these objectives because they addressed the needs of both the supply side (government) and the demand side (civil society) of democratic reform. Within EWMI, each objective had a team leader. The Program’s sub-contractor, the American Bar Association Rule of Law Initiative (ABARLA) implemented Objective 2; and sub-contractor, The Asia Foundation (TAF), implemented approximately half of the small grants program to support the work of human rights non-governmental organizations (NGOs), and selected civil society organizations (CSO), community-based organizations (CBO), and grassroots networks (GN). EWMI administered the other half of the small grants program for legal aid organizations and Public Interest Law Firms (PILF), and to selected CSOs/CBOs/GNs with whom it had already built a strong relationship. EWMI also provided specific event (rather than programmatic) grants to a range of organizations. In 2012, Objective 3 was dropped from the Program owing largely to budget cuts and challenges the government experienced in meeting its commitments. USAID determined that Objective 2 should not be a direct part of this evaluation as ABARLA’s participation had ended in 2012. A summary of the changes resulting from shifts in project focus can be found in Annex II – Changes in PMP and M&E Indicators. As a result of many of these changes PRAJ II’s work focused more heavily on CBO/CSO/GN capacity building, recording human rights violations in databases, and supporting the development of PILFs and legal aid NGOs. By 2012, the level of funding available for small grants had declined, necessitating a reduction and/or reassignment of personnel. In 2013 (originally planned as the last year of the project), PRAJ II prepared a proposal and was granted a one-year extension and US $2 million to finish its work, write the close-out report and transition out of Cambodia in 2014.5F5F 6 The small grants program, as well as EWMI’s development of the Internet platform, Open Development Cambodia (ODC), constituted the bulk of project activities in the last few years. Grantees conducted their work under separate project objectives with EWMI sponsoring periodic “reflections” to help foster greater collaboration. Owing to the way in which data were collected by several grantees, little sharing of information occurred. Under Objective 1, several databases were developed: three documenting human rights violations,6F6F 7 one supportive of the expansion of the Trafficking in Persons (TIP) database, one supportive of the Gender￾Based Violence (GBV) database, and a database focusing on criminal cases heard at the Phnom Penh 6 This extension will allow USAID sufficient time to develop a new, follow-on procurement. 7 The databases were developed by the following organizations: League for the Promotion and Defense of Human Rights (in Cambodia) (LICADHO), the Cambodian Human Rights and Development Association (ADHOC), and the Cambodian Center for Human Rights (CCHR). 4 Municipal Court (PPMC). The TIP and GBV database actually served as a template for the PPMC database (with modifications) into which data on pre-trial detention and time lapses in the issuing of final judgments (among many other variables) were entered. Over the past two years, EWMI also supported the development of a database that tracked the perpetrators of illegal pharmaceutical sales. EWMI staff provided technical assistance for the development of these databases and purchased computers for use in the PPMC. Under Objective 4, approximately US $7 million was awarded in the small grants program: The Asia Foundation (TAF) supported grants to approximately 12 Human Rights NGOs and Community Based Organization (CBOs)/Civil Society Organizations (CSOs)/Grassroots Networks (GNs). EWMI supported programmatic grants to: two Human Rights NGOs; seven Public Interest Law Firms (PILFs) and Legal Aid NGOs; a media organization; a GN; and supported specific activities for several smaller organizations (see Annex III – TAF M&E and Evaluation Report for PRAJ II Grantees and Annex IV – Summary of PRAJ II Grants). TAF and EWMI created their own grants manuals and then followed their own internal procedures for selecting grantees. EWMI made the final selection of grantees and forwarded them to USAID for review. Once awarded, each grantee was provided grant management training. TAF also provided any technical assistance needed when conducting field visits. Each grantee was also responsible for submitting its own monitoring reports that included programmatic and financial information based on the TAF Monitoring & Evaluation (M&E) system. EWMI monitoring included some field visitation. Under Objective 5, EWMI provided grants to legal aid organizations and PILFs to expand and build the capacity of lawyers to provide assistance to poor clients who were being detained under the range of existing laws in Cambodia – criminal, civil, business, human rights violations, etc. The training and capacity building conducted under each objective took on many forms. Under PRAJ I, international consultants were hired to deliver workshops that some participants reported were not sufficiently contextualized for use in Cambodia; others were very satisfied with the training. As a result of these mixed responses, PRAJ II changed its capacity building/training delivery mechanism in favor of “information sharing” or “reflection” meetings in order to facilitate an open forum for grantees to discuss issues they were dealing with. EWMI has stated that a learning plan and assessment was created for these meetings. However, meetings focused more on human rights training than capacity building, which may have posed a hindrance for some grantees. Over the course of project implementation, several activities (outside the specifics included in the CA) were added based on USAID’s request:  EWMI provided grant support to the Returnee Integration Support Center (RISC) for reintegration of Cambodians who had been deported from the United States.  EWMI laid the foundation of establishing Open Development Cambodia (ODC) as an NGO and supported the development of its website and underlying database.  PRAJ II was required to write weekly (in addition to quarterly) reports for USAID. 5 3. METHODOLOGY Evaluation Team The evaluation team consisted of five people: Nancy E. Horn, Ph.D., Team Leader, DevTech Consultant; Melissa Patsalides, USAID/Washington, PPL/LER; Peoulida Ros, USAID/Cambodia, Program Office; Phea Sat, USAID/Cambodia, Democracy & Governance (DG) Office; and Yoke Sudarbo, USAID/Indonesia, DG Office. Evaluation Design Process Data collection entailed a mixed methods approach that included document reviews, key informant interviews, focus group interviews, and a participatory rapid appraisal (PRA) technique that requested participants to draw three time-sequence pictures. The team commenced the evaluation with a document review that included: 70-90-page quarterly reports (QRs), the Cooperative Agreement (CA), the Performance Monitoring Plan (PMP), a mid-term evaluation, a USAID Inspector General Audit Report, PRAJ I Evaluation and Close-Out Report, a 2012 Mission civil society assessment, and selected information on several grantees. On the basis of the document review, each team member was assigned to identify information gaps and develop questions to be posed to individuals and sets of stakeholders in order to answer the overall evaluation questions. The Team Leader generated the evaluation design and included these questions within the design. The data collection instruments appear in Annex V – Stakeholder Data Collection Instruments. Ms. Ros made all appointments with selected stakeholders, which included USAID, EWMI and TAF, as well as a sample of grantees. The team conducted stakeholder interviews from December 3 – 16, 2013 (see Annex VI – Schedule of Interviews and Research Activities), and then analyzed the findings, prepared a debriefing presentation for USAID (December 19, 2013), and began to draft sections of the report. Limitations of the Evaluation Time was a key constraint limiting several aspects of this evaluation: 1) reviewing the volume of documents provided; 2) the production of assignments prior to field research; and 3) the number and type of organizations at which data collection could occur (thus making the findings, conclusions and recommendations illustrative rather than comprehensive). For example, the team was not able to interview other donors or organizations not involved in PRAJ II to learn the specifics of what else is being done in the sector, thus preventing the team from being able to recommend other partners and Cambodian organizations with which to work in future. The length and level of the QRs (70-90 pages each) prevented all team members from reading them in-depth. Consequently, the Team Leader developed a summary of the main points raised in each report so that all the team members would have at least a basic understanding of how PRAJ II was implemented. Over the course of PRAJ II implementation there were many changes in the project’s Performance Monitoring Plan/M&E Plan documents and the indicators therein. This made it difficult to discern how decisions were made and how different elements of the project, as currently implemented, came into being. While the evaluation team recognizes that several documents could have supplemented the analysis conducted, the evaluation’s findings, conclusions, and recommendations are limited to those documents available at the time of data collection (See Annex VII – Bibliography). For example, a request for 6 close-out reports on sub-grantees (which may have reported outcome information) was made during initial interviews with EWMI. It was not until after the completion of the field work and analysis that an offer was made to provide the reports. The content of these reports could not be included in the analysis given to the time constraints for the evaluation. 7 4. FINDINGS AND DISCUSSION This section is organized to respond to the questions posed in the evaluation Scope of Work (SOW) through the exploration of different themes that emerged during data collection. Statements made are based on responses to interview questions and a review of the documents provided. The section begins with overall findings and issues related to project design and implementation, and then moves on to an analysis of the themes that explain the findings related to Objectives 1, 4, and 5. Project Design Theory of Change. PRAJ II’s theory of change was based on the notion that increased availability of information and training of Ministry of Justice (MOJ) staff on human rights violations, court procedures, and land disputes would ultimately lead to the recognition by the Royal Government of Cambodia (RGC) of its need to be transparent, follow the Rule of Law (ROL), and provide a governance structure that respects the rights of citizens. To that end, PRAJ II supported the investigation of human rights violations, the creation of several human rights databases, addressed organizational development and training needs of CBOs/CSOs/GNs, and supported the development of a legal aid culture in defending the poor. Under Objective 3, it also trained a number of MOJ staff on court administration at different levels. However, challenges in implementing what was learned during those trainings and generating system-wide reform raised questions about the implementation of objective (according to the indicators identified to measure progress). The theory of change was based on a supply and demand construct. The supply side of this construct sought to strengthen the ability of justice sector actors to create and promote reform, while the demand side sought to build a broad-based constituency for reform outside of the government. During implementation USAID and EWMI found that the implicit assumption that the central government would fully engage in the reform had not sufficiently been met. Specifically, PRAJ II’s mid￾term evaluation pointed out the shortcomings of this bifurcated approach, largely due to a highly politicized and at times, uncooperative Bar Association of the Kingdom of Cambodia (BAKC), Ministry of the Interior (MOI) and MOJ, and declining resources. Consequently, the Mission viewed the work under Objective 3 as ineffective, dropped it, and significantly scaled back Objective 2. Thus a rule of law and human rights approach was altered into more of a civil society and legal aid intervention. The changes in design were manifest primarily in indicators, rather than in a reconceptualization of the remaining objectives. The basis for the design modification (which was not covered by an amendment to the CA) in 2011-12 may have been interpreted too broadly as every key informant interviewed for this evaluation identified productive working relationships with government entities at provincial and district levels.7F7F 8 Incremental advances to reform agendas occurred as a result of the regular interactions and working relationships between PRAJ II sub-grantees and these government entities (e.g., legal aid organizations with specific courts, human rights NGOs with prison administrators, local cadastral authorities or specific ministry leadership, and Prey Lang Community Network (PLCN) members and district forestry officials). These findings demonstrate that there are additional ways to work toward reform with RGC counterparts than the original design conceived and merits further exploration by the Mission to determine which RGC entities at what level are proponents and even willing partners in reform efforts. 8 Subsequent to the field work, the team learned that AusAid had similar experiences in working with the MOJ and came to the same conclusion, citing http://www.ode.dfat.gov.au/publications/documents/lawjustice-cambodia-case￾study.pdf. However, as noted under limitations above, the team did not have the time to interview other donors and other non-PRAJ II direct stakeholders. 8 A final aspect of the design that the evaluation team questioned is the limited breadth of constituencies that were provided with grants to work on reform. Although EWMI sought to expand the demand side of the theoretical construct by widely advertising its grant program, it was not successful in eliciting proposals from a broader organizational base.8F8F 9 As envisioned in the design of PRAJ II (per the CA), a more serious pursuit of public-private dialogue and cooperation could expand the breadth of constituencies for reform.9F9F 10 To address concerns that the population of NGOs/CBOs/CSOs/GNs willing to work on political issues is necessarily small in a restrictive political environment, the Mission might consider whether there are additional avenues to building constituencies for reform in addition to an overt human rights or political small grants program. M&E Framework. The Monitoring & Evaluation (M&E) system did not properly align with the needs of PRAJ II. The project’s M&E plan and the indicators it collected changed each year, which reflect both the Mission’s and EWMI’s efforts to improve the M&E system. However, the Mission asked EWMI to collect standard F indicators which were not adequate measures of the intended results of the project, nor were they required by any Washington reporting guidance. The strategy in collecting information on these indicators utilized by EWMI and TAF called for self-reporting, which was a challenge for sub￾grantees, as acknowledged by all key informants, sub-grantees as well as EWMI and TAF. For example, the project collected data under the legal aid objective on number of poor clients represented and number of cases handled by target organizations. However, given the length of time often needed for legal aid cases to flow through the court system, the small size of legal aid firms and NGOs, and the few organizations providing legal aid, these numbers could change very little from one year to the next, let alone from one quarter to the next. A better indicator might be a qualitative rubric that captures specific actions taken on a given case. This, accompanied by overall numbers, would more accurately capture project progress in the legal aid sector. A more important issue is that of the overall framework for M&E for the project. The kinds of outcomes PRAJ II sought to achieve are difficult to measure largely because changes are incremental and results often cannot be seen during the period of project implementation. If EMWI or TAF had had sufficient M&E specialists on staff to support PRAJ II, they may have easily identified more effective means of capturing incremental changes. Such information could have led the Mission to make different decisions about eliminating activities.10F10F 11 Reporting. In 2011, the Mission asked EWMI to provide weekly, in addition to quarterly, reports (in the Mission out-brief, staff indicated this was in response to a previous Ambassador’s request). The trickle￾down effect was that EWMI created greater reporting requirements for sub-grantees. EWMI generated a reporting format, initially named “bullet reports,” but the information requested often required more lengthy explanations. One NGO interviewed stated that the weekly report amounted to 50 pages and required a full-time report writer; another organization interviewed refused to comply with the requirement because it took too much staff time. The evidence gathered is conflicting as to whether 9 This may have been because the RGC may have determined much of the work EWMI supported as a threat and so other organizations did not want to be involved, but the team cannot confirm this. 10 It is unclear, owing to Cambodia’s political realities, exactly which organizations/corporations would fall under the category of “private” in establishing these partnerships as those entities, if successful, are well connected to government or to the ruling party. Interviews held with EWMI did not detail any efforts, and so the team makes this finding. 11 If the project had been able to capture the positive changes that sub-grantees were noticing in local and provincial officials, they might have found additional ways to press for reform within the RGC rather than ending all work under Objective 3 early. 9 this weekly reporting was mandatory and/or useful to sub-grantees; EWMI and USAID were the only interviewees who said they benefited from the continuance of the weekly reports. The QRs developed by EWMI were incredibly lengthy (averaging 70-90 pages), reported on every activity rather than outcomes or incremental progress, included information that was repeated from one section to another, and had no summary section for quick reference. USAID staff indicated that they skimmed the reports for specific information needs or tasks. This made it incredibly difficult if not impossible to exercise comprehensive oversight as significant information was essentially buried in all the details. Project Implementation PRAJ II Adaptations. Over the course of PRAJ II implementation (2008-2013, and continued via a one￾year extension through 2014), project objectives were adapted to include several requests from USAID (including funding for RISC, ODC, and dropping of Objective 3, among others) that were not incorporated into the CA. With its changing emphasis to supporting the growth of civil society, legal aid NGOs and PILFs, managerial staff time had to be reallocated within EWMI and TAF. Achievements Reported In response to a request from the USAID Mission to understand the changes that took place during the course of PRAJ II, the evaluation team generated Table 1, on the basis of data from quarterly reports and stakeholder interviews. The table sets forth a number of achievements as reported to which PRAJ II contributed,1F1F 12 separated out (by the evaluation team) into short-, medium- and long-term changes: Table 1: PRAJ II Overall Reported Achievements Short-Term Changes Medium-Term Changes Long-Term Changes Objective 1 Criminal Case database established PPMC using database; lawyers and legal aid NGOs using database Pre-trial detention times shortened for juveniles and accused felons Data on case flow shared with courts in training Court personnel more aware of their shortcomings in moving cases along Case deliberation time shortened Evidence in support of defending the accused collected Judges and prosecutors use and act upon evidence presented Accused are no longer guilty until proven innocent; evidence more frequently requested Objective 4 Gathering of formal and non￾formal organizations for periodic sharing Increased linkages between NGOs and grassroots groups Unified civil society contributes to opposition of LANGO in 2012 12 Due to the contribution of multiple actors to many of the medium to long-term changes, none of these changes should be construed as attributable to the efforts of one project or donor. In fact, several PRAJ II grantees received funds from other donor organizations for work similar to that undertaken as part of PRAJ II. 10 Table 1: PRAJ II Overall Reported Achievements Short-Term Changes Medium-Term Changes Long-Term Changes HR organizations collaborate more consistently on specific high profile cases/issues Contribution of multiple organizations to UPR; release of periodic thematic reports revealing HR violations Increased pressure on RGC for reform, both from international community as well as within Cambodia PLCN unifies multiple villages PLCN people see the benefit of cooperating with local authorities and begin some collaboration (notifying them of forest patrols or even doing patrols together) Local authorities sympathize with PLCN and don’t see them as adversarial per se; PLCN sees the benefit of cooperating with local authorities Women taking a strong role in civil society (ADHOC mentioned beginning work with them in 2003 and now seeing fruits of that on HR Day in 2013) Civil society (with some contribution from PRAJ II) had influence on the Law on Peaceful Demonstrations (LPD), including an amendment that protects organizers of demonstrations CS, PRAJ II and MOI developed implementation guide for LPD Training to CS groups on protections under LPD (partnered with OHCHR to do it) Organizers can’t be ‘set up’ for criminal charges for organizing demonstration The implementation guide created conditions so favorable for demonstrators that MOI insiders were reported to have said the minister must have been ‘tricked’ into signing it. Key informants (IDEA and PLCN) cited training and understanding of their rights under this law Peaceful demonstrations after 2013 elections Objective 5 Court and prosecutor attitudes toward detainees improved Accused treated more fairly and defended more appropriately More cases result in dismissal due to erroneous accusation Legal aid organizations formed and trained; exploit every provision of law in their Courts more carefully following procedural requirements – they are more More cases decided on the basis of the law rather than on bribing and corruption 11 Table 1: PRAJ II Overall Reported Achievements Short-Term Changes Medium-Term Changes Long-Term Changes approach to defending cases careful when dealing with certain legal aid entities now PILF/Legal aid NGOs collaborating with local authorities and other government entities Mutual understanding between the two sides Earlier notification to PILFs of need for lawyer Source: Data compiled through quarterly reports and stakeholder interviews conducted by the evaluation team Challenges Integration of PRAJ II Activities – Owing to the separate focus of each of PRAJ II’s objectives, it appears that not much integration of activities under separate objectives was undertaken. For example, under Objective 1, difficulties arose in sharing database information among sub-grantees, although respondents from these organizations reported cooperating in producing several themed reports. Information on human rights violations was not necessarily shared with CBOs/CSOs/ GNs (target populations under Objective 4) so that they would be more aware of others experiencing the same difficulties (perhaps to be the basis of greater on-the-ground collaboration). Under Objective 5, the human rights NGOs defended poor clients, but successful strategies were not necessarily shared with PILFs and legal aid NGOs. Missed Opportunities12F12F 13 – CBO/CSO/GN sub-grantees requested a number of interventions that addressed their need to maintain and/or acquire skills to generate income. PLCN forest patrols were supported by PRAJ II, but while patrols were out, the household suffered the loss of at least one unit of labor. This meant that whatever income-generating strategy was being employed within the household had to be shouldered by those who remained behind. A more comprehensive strategy that took both human rights and income generation into account might have produced a more committed and larger PLCN membership.13F13F 14 When asked about different ways in which EWMI collaborated with other donors or organizations working in support of DG activities, the team learned that the Law and Judicial Reform Technical Working Group (TWG) had become defunct for lack of a donor facilitator and that such high level, sector-specific collaboration was no longer operational. Collaboration between PRAJ II and other donors did exist to achieve specific objectives, e.g., with GIZ (the German international assistance organization) in constructing and collecting data for the gender-based violence (GBV) database, and with a range of donors to create a joint position statement designed to shape Cambodia’s National Strategic Development Plan. Other donor collaboration emerged from sub-grantees themselves, such as human rights NGOs when funding was sought from multiple donors. 13 Areas where PRAJ II could have taken reasonable steps to achieve a stronger outcome. 14 The team was informed by USAID and EWMI that another USAID-funded project would address this issue, but we were not provided any information on the project – Supporting Forests and Biodiversity – as it would address forest patrols directly under its 2014 work plan (which was outside the scope of this evaluation). 12 Another missed opportunity in working more productively with CBOs/CSOs/GNs was joint planning to achieve outcomes. Using participatory methodologies to involve key stakeholders in identifying jointly￾held notions of cause and effect relationships as well as the links in the chain of reform at the local, provincial and national levels could lead to more substantial outcomes. Participants who utilize these methods do not require a high level of M&E expertise in order to contribute to the planning process, but rather, it requires a commitment of time and attention from USAID and its implementing partners (IP). When participatory methodologies that empower beneficiaries were utilized during PRAJ II implementation, it was in an ad hoc or issue-specific way rather than systematic or higher level.14F14F 15 Capacity building constituted another missed opportunity as the abundance of empowering models for doing this was not tapped into. Capacity building requires a systematic approach that begins with conducting a needs assessment and continues through the development of learner-centered activities offered by a skilled facilitator who is knowledgeable about the country and the specific issues of the people attending. The ad hoc “sharing” and “reflection” approaches taken to capacity building prevented the rigorous development of organizational self-sufficiency.15F15F 16 Shifting Priorities – The refocusing of PRAJ II activities away from issues relating to the Rule of Law and reform of the MOJ and toward building civil society and a legal aid culture created a need to downsize EWMI. At about the same time, USAID requested that EWMI create a more comprehensive platform to access data that addressed human rights violations, especially land concessions. The resulting creation of Open Development Cambodia (ODC) shifted the workload of at least 12 EWMI employees to create this database, gather data for entry and create an open portal to access this information via a website. Although only two PLCN leaders were interviewed (from two villages in Kampong Thom Province), both were not satisfied with the type of attention they received after having been trained in defending their rights. Only local EWMI staff visited PLCN villages; and reflection meetings (held after each major forest patrol) were conducted in provincial towns or Phnom Penh, not in their villages as they requested. Additionally, there appears to be considerable ambiguity about guiding the growth of GNs – should they follow an “organic” growth path, or should they become NGOs? The two leaders interviewed stated they wanted to pursue the latter.16F16F 17 EWMI was requested to establish/support an NGO – RISC - that would address the needs of Cambodian deportees from the US. Unfamiliar with Khmer language after having resided several years in the US, devoid of resources and family connections, and needing cultural re-orientation, the deportees had to be re-educated and re-integrated into their own society. While EWMI was able to respond to this Ambassadorial request, no change was made to the CA as it was conceptualized as a human rights activity. In another instance of shifting priorities, EWMI learned of an illegal operation in pharmaceutical production and distribution. To address these illegalities, EWMI provided funding (not reported in the grants documents shared with the evaluation team) to track cases that prosecuted violators in one 15 EWMI has clarified that they moved away from more conventional methods of training due to time constraints and the fact that other donors were already doing this, but the interviews held and project documents reviewed cannot confirm this. 16 EWMI has clarified that their program officer working with Prey Lang and IDEA generated very tailored capacity building interventions. However, the team’s interviews did not yield this information. 17 Because there is the potential for considerable variance about a network growth strategy among PLCN leaders from the hundreds of villages involved in Prey Lang in four provinces, it behooves EWMI and USAID to ascertain the direction this should take so that the next project can reflect these priorities. 13 location and then constructed a link to similar cases in other locations so that an accused could not escape prosecution. While this was important work in the Cambodian environment, it was an “add-on” that was pursued without changing the CA and the evaluation team did not find clear evidence of how it directly supported the overall goals of the project. A major shift in work focus emerged when the RGC promulgated a new NGO law – LANGO - that would restrict NGO/CBO/CSO/GN activities significantly. With the activities of so many PRAJ II grantees in jeopardy, EWMI worked with local and international organizations to try and prevent the law from being enacted – an effort that brought CBO/CSO/GN solidarity to a new level. The concerted effort produced the desired result, especially because elections were at hand, but local newspapers recently wrote that the law will be re-opened in 2014. It is not clear whether donors and members of civil society can amass a similar campaign as several donors have stopped funding ROL initiatives. OBJECTIVE 1 – Improve Collection and Use of Justice System Data This sub-section concentrates on the achievements and challenges presented in the following three aspects of the project: data collection, creation of the databases, and database use. The sub-section begins with the grants record in support of human rights NGOs and then analyzes the data obtained through interviews with three of these NGOs and the PPMC: LICADHO, ADHOC and CCHR. Achievements Databases - Databases documenting human rights violations (CCHR (Sithi.org), LICADHO, and ADHOC), criminal court procedures (PPMC), trafficking in persons (C-TIP), gender-based violence (GBV), and land encroachment resulting from government land concessions (ODC) were either established or supported through PRAJ II’s small grants program (see Table 2). Without the funding provided, many of the achievements documented in this report might not have been possible. Table 2: Grants Made by EWMI and TAF in Support of HR NGOs Grantee/Type Purpose No. of Years Total Amount EWMI CCHR/ HR NGO Trial Monitoring; database development; periodic thematic reports 5 $162,357 CHRAC/ HR NGO HR and advocacy 3 $23,350 RISC/HR NGO Provide support services for returning deportees from US 4 $127,761 TAF ADHOC/ HR NGO Trial investigation in land cases 6 $343,439 LICADHO/ HR NGO Investigation of all activities related to government infringement on people’s rights 6 $1,236,278 CHRAC/ HR NGO Investigation of all HR violations 5 $203,471 CCHR/ HR NGO Creation of database; use of DB in generating thematic reports 5 $433,930 CDA/ HR NGO HR and land violation awareness raising 4 $105,551 Source: EWMI and TAF reports collected by the Evaluation Team 14 Website “hits” have steadily increased for all information either directly available on databases (CCHR and ODC) or through published reports compiled by each organization (ADHOC, LICADHO and CCHR). However, the “hits” recorded were skewed by hackers in Beijing (as in the case of LICADHO), and spiked during the commune and National Assembly Elections (as in the case of ODC). Greater accessibility was made possible to Cambodians when CCHR (9/10) and LICADHO (6/11) launched Khmer versions of their websites. CCHR won two international awards for its website: 1) “The Rights to Freedom” award of the Information Society Innovation Award in 8/2011 (for innovative ICT solutions contributing to social development in the Asia Pacific region); and 2) the “Communications for Social Change” award of 2013 from the University of Queensland Centre for Communication and Social Change. Table 3: Database “Hits” Database Time Period Visits Unique Visitors Pageviews17F17F 18 Average % of Pageviews from Cambodia Global Social Media Followers, Likes & Views18F18F 19 ADHOC 1/11-12/13 192,121 71,615 857,098 51% 1,342 CCHR: Sithi.org 9/09-11/13 343,673 164,147 259,427 24% 172,131 LICADHO 1/09-11/13 637,562 264,931 2,028,827 31% 941,537 ODC 8/11-11/13 449,129 311,375 1,048,889 59% 9,851 Source: EWMI Database monitoring software. The information is not complete due to the more recent use of Google Analytics. Support for Data Collection on Human Rights Abuses - The owners of two of the major human rights databases collect information through on-site observation: ADHOC in all 24 provinces; and LICADHO in 12 provinces plus Phnom Penh (and will be opening another provincial office in 2014). LICADHO focuses on all types of human rights violations involving state institutions, while ADHOC focuses more on land rights, human rights and legal aid, and women and children. CCHR, the third major human rights database owner, collects information from public sources such as newspapers, through HR activists and trade unionists they have trained, and, in specific cases, sends its investigators out to collect information. CCHR also documents human rights abuses reported on its hotline. CCHR manages the website, Sithi.org., which is a public access portal to all information on its database. CCHR also collects data on fair trial monitoring utilizing its staff to monitor procedures at the PPMC and Kandal Provincial Courts and the Court of Appeals. To monitor trials, they utilize checklists that include specific criteria for fair trials to determine whether legal rights are being upheld. The checklists, as well as the database, have been revised several times to include an expanding set of data and to accommodate changes in the system. PPMC collects data on criminal cases that have been recorded by clerks who work at the prosecutor’s office, investigative judge’s office and offices of trial judges, and have responsibility for recording all case activities. PPMC has a number of contractual staff to enter the collected data into the system. The data is verified and cleaned by a court administration official, and the database itself is overseen by the court’s 18 For CCHR, these data are available only from 10/11-11/13. 19 Dates on which each of the organizations joined Twitter, Facebook and YouTube range from 5/09 to 1/13 so cumulative numbers are more an indicator of the length of time the organization was a member. 15 chief clerk. Information on this database is available only to members of the court; it is not available to the general public. For the TIP/GBV database, the MOJ received reports on TIP prosecutions from courts. The reports were submitted initially in hard copy, but with TA and material support provided by PRAJ II, electronic submissions were enabled. In the fourth quarter of 2012, the MOJ moved the GBV and C-TIP databases to the Department of Criminal Affairs, but no staff was tasked with data entry. The current status of the databases is not known, although EWMI’s extension proposal identified support for the GBV database. ODC, which was launched as an online platform by EWMI staff, collects information from various sources such as NGO reports, websites of companies who have received economic land concessions (ELC), RGC websites, and the Royal Gazette (the government’s official publication of recently passed laws and other regulations). ODC also collects and publishes maps of land concessions the RGC has granted to foreign and local companies, and land law violations (including one that prohibits the RGC from granting over 10,000 ha. of state land to the same private company). Database Use - The reports generated from the data entered on their databases by LICADHO and ADHOC and posted on their websites, and the CCHR database and reports posted on its website (Sithi.org) are regularly accessed by international bodies and media. These human rights NGOs generate thematic reports that are used to create press releases, hold press conferences and support advocacy campaigns. Many of the reports are used by international human rights groups to assert pressure on RGC for reform. One of the human rights organizations, LICADHO, has established good relations with prison system officials due, in part, to the lack of RGC attention to prison reform and corruption. By working collaboratively with these officials, LICADHO has managed to collect data on 9,878 inmates in 13 provinces so far.19F19F 20 The C-TIP database is of special interest to donors (the US Department of State wanted better information on TIP prosecutions because Cambodia at that time was on the international watch list). A EWMI staff member stated that the RGC also found the database very helpful in increasing TIP prosecution rates. The database produced information from 2010 to 2012, but has ceased to do so owing to continuing personnel allocation problems. The C-TIP and GBV databases were being used to improve prosecution rates in both areas. When the database was established in 2010, the clearance rate (prosecution to final verdict) was 16%, but after one year, the clearance rate doubled. EWMI attributed this increase to incorporating the data into training for court officials (through the power of shame vis a vis other courts or motivation to do better in one’s own court). EWMI also said that the Ministry of Women’s Affairs (MOWA) recently stated that they had increased TIP/GBV prosecution rates; however, this could not be confirmed. C-TIP II is currently under Winrock management. The PPMC Criminal Courts database has improved case management and prosecution clearance rates. Court officials interviewed at PPMC said the database has also helped judges keep track of the detention period of the criminally accused to ensure that legal pre-trial limits are observed. A lawyer at a legal aid NGO who used to oversee a model court project supported this finding. The lawyer also stated that the facts entered into the database prevent judges from making decisions on a non-legal basis. For example, fair trial data illustrated that accused felons and juveniles appear in court without 20 TAF’s Monitoring and Evaluation Report for PRAJ II Grantee 16 representation (which violates the law); judges had to reconsider how to manage such cases. Data from several interviews indicate that the detention period for juveniles has been reduced (although several Phnom Penh newspaper articles declared that this is still a problem). The MOJ has recently promised that a website – to be produced in 2014 – will make PPMC database information publicly available on the rate of case disposition (see The Phnom Penh Post, “Justice Ministry to keep track of courts’ case lists,” December 13, 2013, p.4). Although overall reform is slow in coming, this step is moving the courts in the right direction, according to an ADHOC observer. Legal aid organizations and PILFs use the data from several databases to collect evidence and file complaints with government institutions on a range of rights violations. Government Acceptance and Use of Data Collected - Objective and consistent data collection systems have yielded strong evidence of the RGC’s abuses. Human rights NGOs unanimously believe that well-documented data makes it impossible for the RGC to deny any evidence-based allegations. As a human rights NGO leader noted, “Trial monitoring… is having an impact because the courts can never deny their reports. When they raise the issue of felony and juvenile defendants who don’t have representation, the courts acknowledge they must do something.” He added that a cabinet member directly contacted him and told him that on the basis of the reports he received, he is now lobbying a “tycoon” cited in the reports to return 20,000 ha. of land to the state. Another HR NGO representative said that in some cases the government was not happy with reports from NGOs that highlighted RGC illegalities, but that the dissatisfaction was short-lived because the organization’s activities continued. In fact, in this organization’s case, many prison directors would sit down with the NGO leader and complain about government officials, hoping the NGO could do something about it. Challenges Technical - Database construction and maintenance require significant time and resources, especially when they are first constructed. Language is also a problem; some website-linked databases only allow for English searches, while others use Khmer. Linking events in different geographic locations is also problematic due to the lack of convention on the spelling of place names. Access by GNs - Due to technical skill requirements, availability of computers, and limited access to Internet facilities, GNs are unable to access the information they need to lead evidence-based advocacy campaigns. Creation and Maintenance of ODC Database –USAID directed and funded EWMI (with earmarked funds for anti-corruption funding) to create a public, online platform to provide more “neutral” information to support reform in response to the perception that many NGOs were presenting biased information. The database includes a law compendium; trends in land concessions, mining, soils and water systems, including maps; data on companies operating in Cambodia; geographical information; and information on RGC activities. Two years before ODC establishment, CCHR, a PRAJ II sub-grantee, had already established a similar database (Sithi.org). One HR NGO leader stated that he did not object to the addition of another such database and website, but he questioned why EWMI did not fund a local organization to do it instead of EWMI undertaking the work itself. He stated that Cambodian organizations with appropriate expertise were capable of creating and maintaining the ODC database (it was not clear whether EWMI announced an RFP for creation of this database). However, EWMI argued that before ODC establishment, PRAJ funded the NGO Forum Land Center, a platform to track land concessions and related land cases, for about a year, but it was not productive. Though the CCHR representative stated that ODC is similar to its own database, EWMI responded that ODC is very different. 17 This finding is not questioning the efficacy of the database itself, as attested to the increasing number of “hits” from internal and international organizations; rather, based on the interviews, the team has concerns with the locus of the development of this database as well as the funding that launched the initiative. The nature of the evaluation did not allow the Team to determine: 1) if EWMI circulated an RFP; 2) whether other, capable organizations did not respond due to the sensitive nature of the database to be developed; 3) the overall rationale for the development of the ODC database by EWMI staff; 4) the relocation of EWMI/ODC staff to another location; and 5) the registration of ODC as an NGO as, according to EWMI, ODC represents a new strategy that was born out of EWMI’s work with grassroots groups.20F20F 21 Sharing Database Information – The way in which LICADHO and ADHOC collect their information has prevented them from sharing their databases. LICADHO argued that their capability to get donor funds is based on its database; those provide the data must be protected from being exposed to danger. ADHOC also keeps its database confidential for security concerns. An ADHOC senior staff member reported that if the data were published, it could be destroyed by Internet hackers. However, he said that ADHOC was open to those who approach the organization for selected data. Since LICADHO and ADHOC have staff on the ground to collect information, the two organizations question why they have to share information with those who do not. Only CCHR makes its database available to the public through its online portal, Sithi.org. CCHR did not have any concern with respect to safety and security of the information, and believes that those who provide information are better protected if they are publicly known and connected to organizations that can provide support. Incremental Changes As noted in the analysis of achievements and challenges, changes occur incrementally. Rather than using the PMP or M&E plan indicators, which attempt to measure high level results attributable to the project, Table 4, compiled based on a series of our interviews with PRAJII implementers and sub-grantees, documents the potential contributory factors that can lead to outcomes. Table 4: Contributory Factors Leading to Successful Construction and Maintenance of HR Violations and Court Database Outcomes Contributory Factors Outcomes Hiring and training of individuals to gather data on HR violations from published resources (CCHR), trial monitoring in courts, and other locations Data on HR violations and evidence to represent the accused are collected and entered into a database. GBV illegalities posted on Sithi.org used to prosecute abusive partners. RGC and members of the court access the database for evidence. Developing or using already-existing data collection forms to ensure that all relevant evidence is included Creating a hotline (CCHR) for HR violations to be reported by others Timely reporting of HR violations. 21 Source: EWMI Comments on Draft Report on Internal Evaluation of USAID/Cambodia’s Program on Rights and Justice II (PRAJ II), January 22, 2014, p. 7 18 Contributory Factors Outcomes Creating relationships with local governmental agencies and members of courts to allow for a presence to collect data Presence of HR NGO staff in courts to collect HR violations accepted and facilitated by the course. Data collection facilitated by local authorities in a collaborative mode. Members of the court accept evidence in support of defendants Developing and maintaining the database Verified and widely distributed data used in leading advocacy campaigns. Criminal case data used to alert judges of an accused’s pre-trial detention period and prompts them to more timely action. Criminal case data used by legal aid lawyers to update criminal case status and to prepare for client representation in court Entering all data in a timely manner Cleaning the data Creating a portal (CCHR) for interested parties to access data (Sithi.org) Creating websites in English and Khmer to enable access to data/reports Creating and maintaining Criminal Case Database by PPMC Continually updating database to include changing type of information obtained Generating periodic thematic reports Thematic reports produced, citing concrete evidence of HR abuses from the databases. Reports utilized in defending HR violators and prosecuting cases against RGC. Writing press releases and holding press conferences to announce thematic report availability Creating partnerships with other like-minded donors and implementing organizations to gather relevant data and create reports Judges beholden to majority political party and so do not listen to the merits of the case. Cases brought against HR defenders lead to conviction in first instance, but released on appeal due to evidence presented. Source: Compiled by the evaluation team based on interviews and implementing partner and donor documentation Incremental change in the court process – PPMC’s criminal case database has partly contributed to the improvement of criminal case prosecutions. Prior to the establishment of the database, judges relied largely on recording case activities in registration books. This manual recording system led to problems in exceeding pre-trial detention periods. With the database, investigative/trial judges regularly issue “Pre-trial Detention Reminder Reports” which have led to a reduction in pre-trial detention periods of the accused. As of June 30, 2013, the court had input 5,277 cases at the prosecutor stage, 4,814 cases at the investigation stage, and 2,916 cases at the trial stage.21F21F 22 The PPMC database has also facilitated incremental change in legal aid proceedings. With the database, lawyers can easily request information on the case status of their clients, scheduling, and evidence from court clerks. Better prepared to defend their clients, legal aid lawyers assert the evidence they have gathered, thus forcing judges to be more cautious about the decisions they hand down. 22 EWMI, Program on Rights and Justice (PRAJ II), 19th QUARTERLY REPORT for Award Period October 2008 – September 2013 Report for the period April – June 2013 19 One specific outcome of the establishment of the PPMC database that requires highlighting is legal aid lawyers, provincial and other courts are now requesting that their courts create a similar database so that they, too, can become more efficient. Sustainability While most human rights NGOs are donor dependent globally, some have identified strategies to reduce this dependency. Until fairly recently in Cambodia, donors were fully supportive of database development and the creation of periodic thematic reports that addressed various HR violations. However, the donor pool has decreased threatening the sustainability of Human Rights NGOs. The government’s donor dependency and limited political will are also a concern for database sustainability. PRAJ II’s RGC partners rely on donors/NGOs almost totally to establish and/or maintain their databases. Database operations have suffered from a lack of MOJ commitment to allocate a budget to assign personnel to manage the databases. OBJECTIVE 4 – Strengthen Cambodian Constituencies Supporting Justice Sector Reform PRAJ II implemented a multi-pronged approach to reach its goals, one of which was to address the organizational development and capacity-building needs of CBOs/CSOs/GNs so that they could better advocate for reform. The issues these organizations sought to address included HR violations, corruption evidenced in the inconsistent application of the law, land and livelihood encroachment, and similar issues. Owing to time constraints in the field, the evaluation team was able to meet with only two of these organizations, IDEA – a largely urban-based organization seeking to protect the rights of people working in the informal sector, and PLCN, a rural network of grassroots villagers seeking to stop government from giving away their forest land to local and international concessions thus threatening their livelihoods. Achievements According to available documentation, PRAJ II supported approximately 14 NGOs/CBOs/CSOs/GNs in its small grants program. 20 Table 5: Grants Made by EWMI and TAF in Support of CBOs/CSOs/GNs Grantee/Type Purpose No. of Years Total Amount (US$) EWMI WMC/Media Support production of HR films 3 143,148 IDEA/GN Support development of HR for various income-generating constituency groups 4 102,215 BCV Participation in Community Media Forum 2 2,000 NGO Forum National Consultation Workshops 2 3,000 CVS Youth Forum on Dam 1 700 CCC Media consultant on ways to address LANGO 1 2,000 TAF AEC/GN Advocacy and constituency building 2 $95,509 CCD/GN Community organizing 1 $330,415 CLEC/GN Legal and skills training 1 $290,000 DKA/GN Women and youth leadership development 1 $86,495 FACT/GN Constituency building 1 $40,871 ICSO Establish and equip Indigenous People’s Working Group 5 $422,000 Cord/CDPS/CPN GN Constituency building 4+ CCD $324,657+ Source: Compilation of EWMI and TAF grant reports Table 6 presents the USAID-defined indicators used by PRAJ II to document outcomes achieved by the 12 sub-grantees supported by the work of TAF. Table 6: Outcome Achieved from Grants Awarded by TAF in Accordance with USAID￾Defined Indicators Organization No. of Communities Organized No. of Participants Trained No. of Citizens Participating in Constituency Building2F2F 23 No. of Advocacy Initiatives23F23F 24 CBOs/CSOs/GNs Action for Environment and Communities (AEC) 22 *663 329 *6,535 4495 89 23 Constituency Building was defined by TAF as “the whole range of meetings and discussions that help groups to organize themselves to engage in their own advocacy. . . . It does not include forums for bringing issues to the attention of media and broad public and/or for engaging with the targets of advocacy such as company representatives and government authorities or does it include training exercises” (TAF, 2013, pp. 21-22) 24 This measure was highly problematic because each activity, i.e., meeting, signing of document, telephone call, leading up to an advocacy campaign or event was counted as an advocacy initiative. An advocacy initiative was defined by TAF as “an action to address a problem that has been identified as important to constituents or clients” (TAF, 2013, p. 20). 21 Table 6: Outcome Achieved from Grants Awarded by TAF in Accordance with USAID￾Defined Indicators Organization No. of Communities Organized No. of Participants Trained No. of Citizens Participating in Constituency Building2F2F 23 No. of Advocacy Initiatives23F23F 24 Children Development Association (CDA) n/a *902 1,111 *7,215 8,753 130 Community Capacities for Development (CCD) 60 (includes 28 youth groups) *719 797 *2,644 2833 8 Community Legal Education Center (CLEC) n/a *191 209 49 (clients represented) *102 3,097 (citizens advised) Community Peace Building Network n/a *1,478 1,241 *6,971 9,332 333 Dai Ku Aphiwat (DKA) 33 (14 women, 10 youth, 9 savings) *267 345 *1,485 1,893 7 Fisheries Action Coalition Team (FACT) 11 *33 78 *1,116 908 28 Indigenous Community Support Organization (ICSO) n/a *958 1,980 *10,420 995 254 HR NGOs Cambodian Human Rights Development Association (ADHOC) 484 (land cases investigated) *1,683 1,834 *4,826 10,857 1,874 League for the Defense and Promotion of Human Rights (LICADHO) 1,938 (HR cases investigated) 9,878 (prisoners interviewed) *66 320 (clients represented) 24,959 Cambodian Human Rights Action Committee (CHRAC) 124 (HR cases investigated) n/a *1,184 3,265 573 Cambodian Center for Human Rights (CCHR) 925,107 (database visitors) *160 420 *96 71 1,328 (cases referred) 3,207 Source: Compilation of EWMI and TAF grant reports. * Women Advocacy on Land Rights – Advocacy initiatives undertaken by several HR NGOs and CBOs/CSOs/GNs (with funds from a range of donors) gave rise to 266,950 RGC actions that either resulted in affected populations agreeing to accept the return of partial amounts of land, land swaps, allowance of farmers to continue to cultivate the land, and/or monetary compensation for land given to others. These land actions affected 126,395 families. 22 PLCN is most concerned about the land concessions the RGC is giving to Vietnamese and Chinese rubber and lumber companies because 1) the land, by law, may belong to them as an indigenous people,24F24F 25 and 2) with the cutting down of the forest, traditional livelihoods in resin collection and gathering of medicinal herbs have been threatened. PRAJ II trained PLCN members in forestry land rights and supported them in conducting forestry encroachment patrols.25F25F 26 PLCN has, over time, created relationships with the local Forestry Department, who help them identify where forest clearing is taking place. As a result of the data collected in these local and broader patrols, PLCN, along with other forest-based networks, have been very successful in focusing nationwide attention on the issue of economic land concessions granted by the RGC. Pressure to stop the concessions has led, in a handful of instances, to the RGC rescinding its concessions and retuning the land to local inhabitants. Advocacy on Livelihood Rights26F26F 27 – EWMI has supported the development of IDEA’s organizational and managerial capacity in mobilizing different informal sector entrepreneurs, such as tuk-tuk drivers, street side restaurant owners, petty commodity traders, day laborers, and others. To date, IDEA has been successful in helping tuk-tuk and moto drivers in 1) addressing extortion issues by tax officials as members renew their vehicle licenses; 2) having the cost of petrol prices reduced; and 3) creating parking spaces around markets that allowed drivers to increase their number of customers. Each sub-group involved in IDEA experiences characteristic problems: street side restaurants rely on public sanitation to clean the areas around their restaurants; petty commodity traders and fresh produce vendors rely on the use of “their” space on a daily basis to sell their goods; small scale manufacturers rely on steady prices for inputs, etc. As corruption and other issues arise, IDEA mobilizes its members to address any interference these informal sector workers are experiencing, and continues with its campaigns until there is resolution. The successes they have experienced in Phnom Penh have led to an increasing number of members who pay dues to IDEA (currently at 30% of membership). The evaluation team was informed by USAID that similar successes have not been experienced in provincial capitals. Working With the System – CBOs/CSOs/GNs are very supportive of working with local government officials. PLCN works with the local offices of the Forestry Department, as do other networks where RGC has given away land concessions (e.g., mutual investigation of land irregularities in Battambang by ADHOC and CHRC). LICADHO has worked with officials of the prison system to provide prisoners health care and other necessities. IDEA worked with MOI’s Demand for Good Governance Project (DFGG) and RGC’s Anti-Corruption Unit (ACU) to address corruption in vehicle tax collection, and worked together to file complaints against 90 tax officials, 30 of whom were penalized for their actions. In the resolution of the case, moto and tuk-tuk drivers now pay the transportation tax through a bank, thus reducing the potential for corruption. By addressing needs as they arise through collaborating with local and provincial governments, CBOs/CSOs/GNs have been able to achieve small victories that set precedents for further legal reform. Challenges Capacity Building – Once a grantee received an award administered by TAF, TAF provided financial orientation and then followed up on field monitoring visits with any issues the grantee had. EWMI created a local support infrastructure that was responsible for oversight of PLCN activities. Sub- 25 By law, indigenous people cannot claim collective land ownership in an area unless they are officially recognized as indigenous people or communities. The evaluation team is not sure of their status. 26 This activity should be continued under the USAID-funded Supporting Forestry and Biodiversity project. 27 PRAJ II may have supported the defense of livelihood rights with other organizations, but the evaluation team has no direct knowledge of them owing to the limitations in the number of organizations the team could interview. 23 grantees are also provided financial management training by Forum SYD, a Swedish-based NGO that formed a Financial Improvement Team (FIT), to ensure proper management of the grants awarded to CBOs/CSOs/GNs. Attendance at these financial management workshops was supported by EWMI grants. Organizational development and other capacity building activities are offered only informally and on demand, thus not addressing contextual issues in how to lead and manage specific types of organizations. An attempt was made with GNs to participate in a Commune Investment Plan in 34 communes, but not many results have emerged. Constituency Building – IDEA sought to increase its constituencies by strengthening its ability to solve problems of sub-sets of informal workers that prevented or hindered individual livelihood development. However, interviews with the EWMI team did not reveal an explicit strategy for strengthening these organizations. The two PLCN core leaders the team interviewed had a vision for expanding network constituencies to other villages in Kampong Thom province, and then on to other provinces (they seemed to not be aware of the hundreds of villages in four provinces that are already involved). They believed that the key to being able to maintain their land and their livelihoods was increasing the numbers of people who participated in advocacy campaigns. Several things are currently preventing them from moving forward on this vision: 1) PLCN is not officially registered and cannot receive external funding directly before it is (current grants to PLCN are channeled through CPN); 2) the lack of educational opportunities for adults that would facilitate their ability to use computers to access information on the Internet to develop other advocacy campaigns; and 3) the iconic reputation of PLCN as “trouble” and “instigators” that is constantly pitted against RGC. This has led to the need for PLCN to continuously work on building good relationships with local government departments so that, through negotiation, citizens can maintain their land and livelihoods. While EWMI staff spoke about an “organic” growth of networks, i.e., internally-motivated and designed growth, the two core leaders of PLCN interviewed had a different view. Desiring access to external resources, the two core leaders had visions of PLCN becoming an NGO and then obtaining donor funding for the network’s activities. This outward orientation challenges the idea of organic growth. PRAJ II should focus more attention on building capacity among PLCN members to generate their own internal resources through livelihood development. The recent growth in demand for organic rice, for example, among other agricultural products, could have been satisfied by PLCN farmers if they were provided some training in how to cultivate these varieties more efficiently; the growth in demand for indigenous pharmaceuticals is also a source for livelihood development. PLCN’s request to focus its activities more broadly on democracy building and livelihood development, to date, has not been undertaken by PRAJ II but may be taken up by the USAID-funded Supporting Forestry and Biodiversity project under another development objective. Adding Value – Concern was expressed in several interviews about the value that each of the sub￾grantees brought to the project, emphasizing that more of a patron/client relationship emerged rather than a true partnership. Interviewees envision a different type of relationship in which donors share the insights they have with sub-grantees, and vice versa. Donors bring international linkages and specific types of expertise; local organizations bring local knowledge, skills, and a willingness to explore different ways to bring about reform. According to one leader, the blend of these skills will lead to the achievement of something larger than indicators warrant. Incremental Change PRAJ II monitoring focused more on the system changes it could achieve and less on the incremental changes/outcomes sub-grantees were creating in the course of implementing the plans presented in their 24 proposals. This was likely due to the indicators used to document outcome, which did not prove useful in recognizing the influence of the sub-grantees. In terms of creating an enabling environment PRAJ II was lauded by PLCN core leaders for teaching them about their rights, building their confidence in defending these rights, and encouraging them to act in concert with others to achieve desired results. They hailed the work of PRAJ II for reducing each community’s isolation by bringing communities together in a network. IDEA thanked PRAJ II for similar inputs that helped them organize their network into sub-groups and then teaching them different advocacy strategies that enabled them to mobilize informal workers and entrepreneurs into specific interest groups. The opportunity PRAJ II missed out on was documenting these lower-level organizational and niche outcomes that form an integral part of democracy building. In Table 7, many of the contributory factors that, incrementally, can lead to systemic reform are presented. Table 7: Contributory Factors Leading to Successful CBO/CSO/GN Outcomes Contributory Factors Outcomes Informing CBO/CSO/GN members of their rights Increased advocacy campaigns based in fact and HR understood by CBOs/CSOs/GNs Learning how to mobilize members and others in advocating for their rights More engaged and active civil society; greater collaboration across the spectrum of GNs, with concomitant reduction in isolation Collaborating with different levels of government to solve local problems Increased collaboration between RGC agencies and CBOs/CSOs/GNs Sending commendations to local government for their support and success in solving local problems Institutionalized collaboration between local governments and CBOs/CSOs/GNs Identifying specific issues that need to be addressed by constituent groups A map of priority issues Identifying a strategy on how to address different issues An effective and efficient reform strategy Gathering evidence on how the law is being transgressed or is oppressive Advocacy campaigns to revise laws Consulting with relevant parties on how to change oppressive laws Strategic process of revising law Consulting with PILFs or private law firms on whether to bring cases against those transgressing the law Class action on the transgressed law Organizing GNs to focus on land issue in their advocacy campaigns RGC returns hectares to local population in specific cases Use of media to reveal HR violations Changes in voter behavior More informed and confident GN membership Ability to represent themselves in international, national, provincial and local fora; launch their own advocacy campaigns Source: Compiled by the evaluation team based on interviews and available documentation Sustainability Of all the organizations at which the team conducted interviews, only IDEA had developed strategies for sustainability. For example, IDEA has envisioned having a savings and/or microfinance scheme to support its members in needs of credit. Organizationally, IDEA will continue to collect membership fees to support HQ operations, lead advocacy campaigns and continue its work in improving the environment for informal sector workers. IDEA also needs to consider means of developing alternative revenue streams to continue its work. 25 OBJECTIVE 5 – Strengthen Capacity of Legal Profession and Civil Society to Advise and Represent Poor Citizens Two main actors, PILFs and legal aid NGOs, work under Objective 5, under two sub-objectives: 1) Support New Public Interest Law Firm and 2) Public Awareness Campaign for Legal Aid. Under the first sub-objective, Samreth Law Group (SLG) was established with a PRAJ II grant in 2009 as the first PILF in Cambodia. In 2012 Vishnu Law Group (VLG) emerged as the second PILF as the result of a split from SLG. These two law firms were registered with BAKC. Under the second sub-objective, campaigns have been held to educate Cambodians, especially those living in the provinces, on legal aid services available for them. This has been done through radio/TV programs, public forum sessions, NGOs, community outreach activities arranged by legal aid NGOs, and by the national issuing of RFPs for legal aid support by PRAJ II. Achievements Establishment of PILFs and Legal Aid NGOs - PRAJ II, recognizing the need for financial input to establish and help develop the legal aid sector and PILFs, provided funding under the small grants program to the following organizations: Working Inside the System – Legal aid organizations and PILFs have found cooperation with the government useful. One PILF has cooperated with the Cambodian Human Rights Committee (CHRC), using the Committee’s influence to process cases faster and enhance evidence collection. Another PILF worked with the Anti-Corruption Unit to pressure the courts to follow regulated systems and procedures. One legal aid NGO holds regular meetings (not under PRAJ II grants) with prosecutors, other members of the court, and other government officials that have resolved local issues. Table 8: Grants Made by EWMI in Support of Legal Aid and PILFs Grantee/Type Purpose No. of Years Total Amount LAC/Legal Aid Provide legal aid services for in all cases, including land 6 $918,813 IBJ/Legal Aid Provide legal aid services in criminal cases 4 $298,860 LSCW/Legal Aid Provide legal aid services to victims of gender-based violence 5 $518,646 SLG/PILF Provide legal aid services in all cases 4 $340,086 VLG/PILF Provide legal aid services in all cases 2+ $80,468+ CDP/Legal Aid Provide legal aid services in all cases 2 $167,325 BAKC/Bar Assoc. Provide legal aid services in all cases; lawyer support 2 $95,176 Source: Implementing partner reports Legal Aid NGOs and PILFs27F27F 28– interviewed PILFs stated that they operate differently from NGOs that also defend the poor. PILFs reported to be more selective of the clients they defend (taking only high profile cases), put extra effort into collecting evidence, and allocate sufficient time to prepare. While the team found that the structural context of PILFs and NGOs defending clients pro bono differ (PILFs are registered with BAKC; NGOs with MOI), their activities and strategies are the same. They both 1) 28 One factor differentiating PILFs from legal aid NGOs was identified by EWMI: PILFs take on a range of high profile cases that have serious political implications, while legal aid NGOs characteristically stay away from “hot” cases. EWMI cited a case that involved former Khmer Rouge solders. However, it may be a matter of perspective as to whether one case is “hotter” than another since most land cases, in particular, involve well-connected, powerful tycoons and other political elites. 26 serve public interest and represent poor clients who have been affected by land conflicts and HR violations; 2) conduct awareness-raising programs and provide training on relevant laws and rights to the communities they support; and 3) see the value in collaborating with RGC agencies and have been diligent in finding ways to work with local government. The existence of both of these institutions has resulted in an increase in the number of poor clients represented. Legal Aid Beneficiaries - Under PRAJ II, there are PILF, legal aid NGOs and HR NGOs who defend HR violations and the poor, based on different types of cases. As shown in Table 8, legal aid and HR NGOs defending the indigent have represented 5,047 cases and investigated 2,546 other cases, all of which have benefitted more than 19,689 clients over the period 2008 to 2012.28F28F 29 Among the 2,150 cases represented by LAC, 48 dealt with land ownership, and 732 with women and juveniles. IBJ takes on criminal cases; and LSCW addresses cases dealing with human trafficking, rape, domestic violence, and abuse. HR NGOs (ADHOC, LICADHO, and CHRAC) deal with land ownership/conflict and human rights cases. Table 9: Number of Cases Represented and Number of Clients Benefitted by Legal Aid NGOs from 2008-201229F29F 30 Legal Aid/HR NGOs # of Cases Represented/Investigated/Counseled # of Clients Who Benefitted from Legal Aid Counsel30F30F 31 LAC 2,150 (1,611 cases represented in courts and 539 cases with legal counseling) 13,238 IBJ 1,259 (1,025 cases represented in courts and 234 cases with legal counseling) 191 clients released on bail LSCW 1,116 (939 cases represented in courts and 177 cases with legal counseling) 1,253 CDP 512 (412 cases represented and 100 cases with legal counseling) 826 RISC - 79 returnees served CLEC 10 (on land and livelihood rights) 49 clients represented and 3,199 advised ADHOC 484 land cases investigated 468 clients implicated in land disputes represented LICADHO 1,938 case of human rights investigated 386 clients represented CHRAC 124 human rights cases investigated - Source: TAF, Monitoring and Evaluation Report for PRAJ II Grantee, November 2013; EWMI, PRAJ II EWMI Grantee Summary Report, December 2013. Out of 1,611 cases represented by LAC in court, 320 (20%) were either won or dismissed. Although this might appear to be a low percentage, it must be borne in mind that, at the same time, LAC worked 29 The Grantee Summary Reports the evaluation team received did not include cases defended by the PILFs. 30 Not every legal aid NGO received grants during the period of 2008 to 2012. Some NGOs received grants later than the others. Also, only some selected NGOs are presented in the table based on their similar activities of case representation. 31 The number of clients who benefitted is not on equal to the number of cases represented because some cases do not proceed to courts and are settled through counseling/advise, negotiations to be released on bail, etc. 27 closely with the court to reduce excessive pre-trial detention. For IBJ, 191 clients were released on bail, allowing the accused to await trial at home. As a result of ADHOC’s interventions in the period of FY2009-2010, some 5,511 families either had their land returned or were allowed to continue cultivating 21,883 ha. Challenges Establishment of PILFs – Establishing and maintaining PILFs is very complicated and donor-dependent. A group of lawyers must take the initiative to launch a PILF and then obtain funding to support their activities. Despite the high demand for legal aid services, the RGC has allocated very little funding for the defense of the indigent. This amount is channeled through the MOJ (not BAKC) although it is unclear which unit actually has oversight. Funding for legal aid was recently increased by 50% (up to US$75,000), but the amount available – given the size of the demand – is inadequate. Another barrier to pro bono representation of the poor is the number of lawyers within Cambodia. There are currently approximately 800 lawyers in Cambodia who have passed the Bar, but only 600 are practicing lawyers. Of this number, a very small percentage is providing legal aid services. Issues in Leadership/Management of PILFs and NGOs Defending the Poor – Complications in leading/managing a PILF resulted in the split of the first PILF into two. The split resulted in a loss of donor support and a number of its staff by the first PILF (which currently employs only five lawyers and so is unable to take on a greater case load). One strategy employed by a PILF and a legal aid NGO to ensure sustainability was to take on both paying and pro bono cases, with the former subsidizing the latter. Issues arose in managing financially as well as in managing the case load because legal aid cases demand more time. Perceptions of Corruption and State-Controlled Media - BAKC is viewed as corrupt and slow in their operations. Many case referrals are late, resulting in lawyers having insufficient time to collect evidence to represent clients adequately. As a result, many lawyers who are assigned legal aid cases by BAKC refuse to take them on. BAKC itself has put many restrictions on lawyer practices. For example, BAKC rules prevent lawyers from advertising their services to the public. Consequently, as, reported to the evaluation team, legal aid awareness-raising programs via radio talk shows were often censured.31F31F 32 Difficulties in Gathering Evidence – Lawyers are often prevented from gathering evidence to defend their clients largely due to corruption, the presumption that the poor are guilty until proven innocent, lack of understanding of the rules of evidence, and lack of understanding of the law. In other cases, legal aid lawyers are prevented from visiting with their clients to obtain even basic information. Incremental Changes In establishing a legal aid system, PRAJ II generated modest indicators to measure their activities. However, as the project changed – especially as a result of the Mid-Term Evaluation – new indicators were developed to monitor the progress of PILFs and legal aid NGOs. These indicators, such as number of pro bono clients successfully defended, were inappropriate due to the very lengthy defense process. Due to the inappropriate indicators used, capturing the incremental changes that were occurring was not possible. Through the evaluation, however, the team was able to document utilizing available records and interviews many changes as well as the factors that have contributed to these changes. They are presented in Table 10. 32 This BAKC regulation may have been enacted prior to the 2013 elections to prevent any challenges raised against the incumbent party; it may not be enforced currently. As the team did not interview BAKC, this potential finding cannot be confirmed. 28 Table 10: Contributory Factors Leading to Positive Legal Aid Outcomes Contributory Factors Outcomes As a result of continuous training in the law and in court procedures: members of the court are more knowledgeable of the system; and judges and prosecutors are more knowledgeable of the law. Courts follow systems and procedures:  Judges consult documented evidence in support of defendants  Judges are more circumspect before issuing final judgment  Judges are more likely to uphold the law PILFs and NGOs defending pro bono clients work with the court on behalf of their clients;  Regular meetings held with prosecutors and judges to discuss cases  Use of PPMC and other databases to establish evidence of innocence Regular interaction between PILFs/legal aid NGOs and members of the court create opportunities for mutual learning and greater cooperation in defending the indigent. PILFs and legal aid NGOs defending the indigent work with government units (CHRC, Anti￾Corruption Unit, MOJ and MOI) to obtain their support in defending their clients. Greater credence of evidence presented in defending a client resulting in adherence to the law in the presentation and final deliberation of a case Lawyers assiduously seek evidence to defend their clients, interviewing witnesses, gathering data, documenting events, etc., as well as investigating the appropriate laws in preparing their cases. Lawyers offer iron-clad evidence and fully exploit the law in defense of their clients resulting in an increase in defendants “proven” innocent. Lawyers are more knowledgeable of the law, more experienced in representing their clients in front of the courts, and stronger in collecting evidence. The courts recognize, accept and rely on the evidence amassed in defense of a client resulting in greater adherence to the law. Source: Compiled by the evaluation team utilizing implementing partner records and interview notes. Sustainability PILFs and legal aid NGOs are largely donor dependent. When one legal aid NGO accepted private clients to subsidize its poor clients, one of its donors viewed it as a conflict of interest and ceased funding. This loss of funds coupled with weak financial management caused it to go bankrupt. PRAJ II is unable to claim success in assuring the sustainability of the legal aid sector, as it states in the CA: “the firm will engage with the private sector, starting with the private lawyer community… a key step towards sustainability”.32F32F 33 The limited number of lawyers available to join PILFs, the limited resources to build and sustain a legal aid culture, and the internal corruption of selected legal aid entities all work against the achievement of this objective. 33 Cooperative Agreement No. 442-A-00-08-00011-00 Titled “Rule of Law and Human Rights in Cambodia”, September 23, 2008, page 22. 29 5. CONCLUSIONS AND LESSONS LEARNED Project Design and Implementation (Q2) Project Design – Following the mid-term evaluation, PRAJ II leadership learned that an implicit assumption (level of commitment expressed by the central RGC entities with which it was trying to partner) in the theory of change had not been met. As a result, the mission and EWMI shifted from implementing a ROL project to a CBO/CSO/GN and legal aid sector capacity building project. The support for the growth of civil society included “arming” them with information (websites and databases, public awareness campaigns, funding for investigations, forest patrols, etc.); creating opportunities for them to find common areas of interest for broader initiatives (arranging meetings, providing networking opportunities); and targeted training where needed (technical legal areas for legal aid orgs). The challenge to PRAJ II was that the indicators utilized in the PMP and the M&E plan could not account for all the incremental changes that resulted from all the activities reported in the QRs. Without changes documented in an amendment to the CA, which functioned as the design document guiding PRAJ II, a thorough reconsideration of indicators was not undertaken. M&E Framework - The M&E Framework for PRAJ II did not capture incremental results as well as the relative contributions of USAID and others, instead seeking to use traditional performance indicators to understand what changes were attributable to USG efforts (F standard indicators). The meaningful and interesting progress this evaluation documented in terms of incremental results could not be captured by PRAJ II monitoring systems due to the lack of an appropriate, inclusive and incremental change M&E framework. That outcomes were not documented was a weakness in the framework. Reporting - The current reporting system places a great burden on sub-grantees who are not staffed to meet such onerous requirements (one sub-grantee indicated that they dedicate one staff person to it full-time). Another primary grantee indicated that they sacrifice time they would otherwise spend mentoring or working with sub-grantees to fulfill these reporting requirements. Degree to Which the Objectives Were Achieved (Q1) OBJECTIVE 1 Databases are widely seen as central to evidence-based advocacy and institutional reform by NGOs/CBOs/ CSOs/GNs and the RGC. Evidence can exert pressure in various RGC entities to take action against human rights violations. Data is very helpful in improving institutional performance and its use has resulted in better law enforcement and the implementation of criminal procedure. Regardless of the usefulness of the databases, however, the RGC has often been non-responsive in supporting their continuous development. OBJECTIVE 4 HR awareness has been extended with every grant PRAJ II has awarded to NGOs/CBOs/CSOs/ GNs owing to the project’s provision of initial training in HR. Sub-grantees, in fulfilling the goals of their respective proposals, have both challenged and collaborated with RGC entities to adjust laws, regulations, and to prosecute HR violators. Accomplishments of each organization have not been shared adequately to state that sub-grantees have learned from each other. PRAJ II did not fully address the delivery of contextually appropriate and learner-based strategies for capacity building that helped form self-sufficient organizations; instead, most remain donor-dependent. OBJECTIVE 5 In trying to establish a legal aid culture, poor citizens have been represented and provided with legal counseling. PILFs were established and supported to enhance legal aid services. However, without governmental support through budget allocation, the sector will remain donor-dependent. 30 Sustainability (Q3) No Non-Governmental Organization interviewed or strategic partnership between NGOs interviewed demonstrated sustainability. Of all the organizations with which the team conducted interviews, only IDEA had developed strategies for sustainability. Constituency building and sustainability of the interviewed NGOs are both challenges owing to their donor-dependency. The human rights and legal aid sectors are severely challenged by a shortage of funding as well as the low number of law school graduates who become practicing lawyers. 31 6. RECOMMENDATIONS FOR FUTURE D&G PROGRAMMING Recommendations related to Project Design and Implementation (Q2) USAID is one player among many attempting to achieve democratic reform in Cambodia. Therefore, in future the Mission would be well-served by either forming an association/ coalition of actors seeking to create a more democratic society and/or assuming a leadership role in the donor TWG. With the information shared on activities and results, develop a systems perspective in planning and measurement, taking into account all the actors engaged and what their perspectives, relationships and contributions are to achieving reform. The results of forming this association/collaboration might be seen in the pooling of donor funds to support democratic reform. With a common planning and M&E approach, success might be more easily achieved by creating a few common indicators in one progress report per quarter for those receiving small grants in order to capture incremental change. While some sub-grantees have expressed a desire to receive direct funding, others have not, largely due to the political ramifications of receiving direct funding from USAID. Pooled donor funding could reduce this stigma and facilitate greater sharing of data collected and entered into databases. Agreements would have to be reached, memoranda of understanding would have to be signed, and by-laws would have to be developed by members of this new association to assure collaboration and continuity. Such collaboration would also prevent widespread and time-consuming competition in writing several proposals. Each potential sub-grantee would fill an informational niche (so as to prevent overlapping data collection) and would be responsible for collecting data within that domain. Table 11: Sample Logic Model in Future Programming Activities Outputs Short –Term Outcomes Longer-Term Outcomes Development of databases (including data gathering and entry on HR violations and court procedures) Thematic reports Increased number of advocacy campaigns Changes in local, provincial and national systems, procedures and laws Meetings and networking opportunities among Sub￾grantees Reduction in sense of isolation; increased sense of solidarity Expanded participation in advocacy on chosen issues Civil society functions as a check to RGC actions Long-term capacity building plan developed for categories of NGOs/ CBOs/CSOs/GNs Leaders of organizations prepared to mobilize constituencies and lead/ manage organization Mobilization of broader constituencies for advocacy; greater accuracy in reporting RGC reform Source: Generated by the Evaluation Team during post evaluation learning sessions. Firmly ensconced in a network of local and international actors who share what they are doing and reporting on the results they are achieving in human rights, government reform and civil society, the Mission should plan for a greatly modified approach to M&E in any follow-on activities, with fewer performance indicators in the traditional sense, and far more robust efforts at capturing incremental change as well as contributions of various actors in the system. M&E approaches that recognize this reality, such as developmental evaluation, outcome harvesting, or contribution analysis (a modified form of which we have attempted in the findings on each of the Objectives above), among others, will be far 32 better suited to helping the Mission understand what its contribution has been. One suggestion to capture these incremental changes is to develop a logic model (see Table 11). Participatory methodologies, including outcome mapping, participatory impact pathways analysis, and others that would allow the USAID Mission and its implementers to convene key stakeholders to identify jointly-held notions of cause and effect relationships as well as the links in the chain of reform at the local, provincial and national levels in order to greatly improve the M&E plan. This would lead to stakeholders feeling a greater sense of buy-in and understanding of relative contribution to achieve desired outcomes into the evaluation. The Mission should reconsider the information it needs for management decision making and limit its reporting requirements to no more (in frequency or content) than necessary. USAID should reconsider working with the RGC at all levels on different reform measures, by focusing greater attention on provincial and local government leaders (a network of “champions” for particular reforms) where collaborative inroads have already been made by NGOs/CBOs/ CSOs/GNs. Organizational sustainability is the result of many factors, including the following: 1) commitment to an organizational vision, mission and values; 2) the creation and following of systems and procedures; 3) good leadership and management; 4) political will; and 5) a sustainable source of funding (among others). Mobilizing and building the capacity of organizations and networks requires a long-term plan that meets an organization and its members where they currently are, ascertaining their strengths and weaknesses, building upon what they have through workshops and real-life exercises, and then coaching and mentoring them through a learning process – complete with time-related benchmarks – until the leaders and members of the organization become self-sustaining. Identifying skilled facilitators who know the Cambodian context (including its history and the culture of fear many Cambodians experience when considering how to address – if at all – human rights violations and land rights), are aware of cultural issues in organizations, and are armed with a toolkit of learner-centered activities would be critical in taking NGOs/CBOs/CSOs/GNs to the next level. Prior to any new initiative, considerable research and/or a baseline study should be undertaken to analyze the current political atmosphere and the potential for reform at all levels. This research is necessary to obtain an accurate reading of the RGC’s readiness and willingness to reform. While it may appear that members of the RGC and the courts are deeply entrenched in the status quo, the results of the most recent election indicated a change in the mind of the voting population such that the opposition won 26 more seats. Dissent and the willingness to change need to be measured at all levels and in all types of communities. Given the historical (and even current) use of military and police force, an accurate picture needs to be taken of where the Cambodian people are in terms of their willingness to act on their convictions. Recommendations to Overcome Hindrances or Leverage Achievements (Q1) OBJECTIVE 1 USAID should consider further support of database development, expansion and data analysis by:  Ensuring that data obtained and entered are complementary to that of other organizations, and that information is shared – to the extent feasible – among database owners.  Supporting a marketing strategy for database products to reduce overwhelming donor dependency.  Advocating with the appropriate RGC entities for appropriate budget allocations to support database development and maintenance.  Supporting collaborative public/private partnerships to develop mutual appreciation for data, its storage, and its use in improving HUMAN RIGHTS in Cambodia. 33  Supporting the development of stronger linkages with international human rights NGOs and international media to ensure a continuous global focus on Cambodia.  Supporting training in transforming database entries into reports that suggest new policy directions and actions on the part of RGC entities.  Supporting the training of citizen journalists/witnesses capable of reporting human rights violations to human rights NGOs for database entry. OBJECTIVE 4 USAID should consider continuing the small grants program by:  Expanding the breadth of CBOs/CSOs/GNs receiving grant funding to create a wider base for the demand side of democratic reform (e.g., tapping into organizations funded by other donors and keeping abreast of new organizations’ missions as they develop).  Encouraging sub-grantees to find ways of collaborating with different levels and different agencies of the RGC when they make inroads toward reform. Ideas on how to do this should be generated by the organizations themselves in a forum established for this purpose.  Providing a planned program of learner-centered capacity building to sub-grantees to create strong, self-standing CBOs/CSOs/GNs capable of generating their own revenue.  Identifying participatory approaches to strengthening GNs that will not transform them into NGOs but rather help them to identify their own pattern of growth and development based in the mobilization of local resources.  Blending targeted income-generating strategies with democracy building activities among GNs so that they are capable of being self-supporting (as the Mission’s Supporting Forestry and Biodiversity project will be attempting to do this year).  Helping CBOs/CSOs/GNs to develop their own roadmap to success that includes expanded community participation, advocacy campaigns, livelihood development and other elements identified by the organizations themselves.  Supporting regularly-scheduled meetings among sub-grantees in different locations with learning, coaching and mentoring agendas. OBJECTIVE 5 - LEGAL AID USAID should consider the following recommendations to build a legal aid culture:  Offer scholarships to attend law school in return for two years of service to a Public Interest Law Firm (PILF) or legal aid NGO upon graduation.  Fund more internship opportunities for law school graduates to practice in a PILF.  Fund legal aid NGOs to teach would-be lawyers how to advocate for expanded legal aid support by RGC.  Strengthen the Bar Association of the Kingdom of Cambodia (BACK) referral system to remove obstacles preventing lawyers from participating fully in the legal aid system.  Support BAKC to create a system to monitor the quality of education provided to future lawyers (to address purchasing of degrees and to create “checks and balances” in legal education).  Provide continuous training on changes in the law and the legal system to PILFs and legal aid NGOs, as well as members of the court, so that appropriate laws are referenced in trials and appropriate systems and procedures are followed.  Support the creation of a legal aid database to monitor case flow that can be accessed by all lawyers defending the poor.  Support PILFs and legal aid NGOs who intend to take on private cases in terms of financial management and case handlings.  Human Rights NGOs should continue to take on legal aid cases to extend the legal aid culture. 34 This page is intentionally left blank. 35 ANNEXES 36 ANNEX 1: STATEMENT OF WORK Statement of Work for A Team Leader for a joint USAID/Cambodia and LER Internal Evaluation Objective LER requires the services of a qualified evaluation methods specialist/contractor to serve as the Team Leader for an internal evaluation of USAID/Cambodia’s Program on Rights and Justice II (PRAJ II). The contractor will be expected to work closely with the USAID/Cambodia and LER team to conduct a high quality evaluation of this justice sector reform, human rights and civil society development project. The contractor will also support efforts to build the capacity of USAID staff participating on the team in the field of evaluation. 1. Background Cambodia has suffered decades of political instability, war and recurrent violence which displaced populations, degraded national institutions and diminished capacity within the government. Excessive politicization has hindered efforts to advance justice sector reform as well as the failure to pass key legislation, agency infighting, direct political interference in specific cases, and widespread corruption. In order to address these problems and promote justice sector reforms, USAID/Cambodia awarded the PRAJ II project to the East-West Management Institute (EWMI), on October 1, 2008. This cooperative agreement ends on September 30, 2013 and is for approximately $20 million over five years. Due to the challenges of promoting judicial reform in Cambodia, and the real or perceived stalling by agencies responsible for the administration of justice, USAID/Cambodia decided to significantly change PRAJ II based on the findings of a mid-term evaluation conducted in April 2011. Most activities involving cooperation with the Cambodian government or affiliated bodies were discontinued, and emphasis was placed on promoting human rights and creating demand for justice. Through sub-grants, PRAJ has worked with civil society, strengthening civil society organizations, building advocacy potential of these organizations as well as networks undertaking advocacy efforts, and focusing on human rights advocacy. Advocacy efforts have been concentrated in the area of land disputes. Support has also been provided to legal aid providers to offer free legal representation to indigent criminal defendants and victims of domestic violence as part of the human rights work supporting vulnerable populations. This final performance evaluation will focus on the objective as stated in the initial RFA, “Strengthen support for the development of broad based Cambodian constituencies with common interests in supporting effective reform in the judicial sector.” As a limited scope internal evaluation, the Mission wishes to focus efforts on a few questions (the full evaluation scope of work will be provided once the evaluation methods specialist is selected): 1. To what extent have these three key project objectives been achieved? What have been the main promoters or hindrances to the achievement of these objectives? o Objective 1: Improve Collection and Use of Justice System Data o Objective 4: Strengthen Cambodian Constituencies Supporting Justice Sector Reform o Objective 5: Strengthen Capacity of Legal Profession and Civil Society to Advise and Represent Poor Citizens 2. Have the key sections (as noted in Question 1) of the original project design been successfully addressed or has implementation demonstrated that these elements of the design were insufficient or incorrect suppositions? 37 3. Which individual NGOs or strategic partnerships between NGOs show evidence of sustainability, in terms of a) capacity to function effectively, b) having a constituency which is interested enough to maintain the organization, or c) financially sustainable (or moving in a direction of sustainability outside of sole-USAID grants)? 2. Scope of Work LER and USAID/Cambodia is seeking a consultant to lead an internal evaluation of the PRAJ II project. The purpose of the internal evaluation is to provide 1) critical information to USAID/Cambodia in order to inform the design and strategy of a high priority follow-on democracy program, as well as to 2) develop the capacity of USAID/Cambodia, regional, and PPL/LER staff in the field of evaluation. The contractor will perform the required tasks as described below: 1. Conduct desk top review of documents. 2. Lead development of the draft evaluation design, work plan, and pre-departure briefing. 3. Work with individual team members to develop individual learning plan for the evaluation. 4. Work with team to finalize proposed methodologies, including the initial development of interview guides, interviews or focus groups or roundtable discussions with NGO partners, program beneficiaries and other stakeholders as identified by the evaluation team. Train team on methodologies and related tools/ instruments to be applied during the field work. 5. Finalize the selection of field sites and provide oversight to sub-teams covering different geographic areas. 6. Provide training and support in application of evaluation methodologies, synthesis of data, and data analysis. 7. Lead mid-term and final de-briefs. 8. Based on input from team members, combine drafts into one coherent draft for Mission and LER review. 9. Submit final evaluation report to USAID/Cambodia and LER. 38 3. Deliverables The consultant will ensure that the following deliverables are achieved: i. Draft Evaluation Design and Work Plan and Pre-Departure Briefings: The evaluation team will develop a draft evaluation design and work plan prior to arrival in Phnom Pen. A pre-departure briefing via conference call may be conducted with the Mission. ii. Individual learning plans: The consultant will work remotely with the individual evaluation team members to develop brief learning plans for their work on evaluation. iii. Mid-Point Review/Briefing: The evaluation team will provide a mid-point briefing to the USAID/Cambodia team, to clarify any outstanding queries that may have emerged since the initiation of the evaluation process. iv. Oral Presentation: After field work and initial data analysis, the evaluation team will provide a power point presentation on its findings and early recommendations prior to departure. v. Reports: The consultant will be required to submit the following reports: a) Draft Report: The evaluation team will present a draft report of its findings and recommendations one week after departure. Any data collection and analysis will disaggregate by gender where appropriate (required by USAID policy), and other characteristics as relevant and depending on data availability (e.g., age, geographic region). b) Final Report: The final report should meet the criteria for evaluation reports as stated in Appendix 1 of the USAID Evaluation Policy (http://www.usaid.gov/evaluation) and be delivered to USAID A/COR Elizabeth Callender within 30 days of completion of departure from Cambodia. 4. Level of Effort The contractor will accomplish the above tasks between October 15 and December 20, 2013. Fieldwork will take place between October 28 and November 16. It is envisioned that the contractor’s level of effort will be 31 days total, with time included before and after the fieldwork for document review and report drafting. A seven-day work week will be authorized to conduct field work in Cambodia, as travel in the provinces may occur on Sundays. Logistical support for travel and arranging meetings/interviews etc. will be provided by USAID Cambodia mission. Please be advised that Nov. 16-18, 2013 is a public holiday in Cambodia. The suggested level of effort and notional timeline for tasks to be completed is as follows: 39 Activity Location LOE (days) Start Date End Date Evaluation prep, document review, work on learning plan with evaluation team members US 10 TBD Travel to Cambodia 2 Conduct evaluation Cambodia 17 Dec. 2, 2013 Dec. 20, 2013 Learning days US 3 Travel home 2 Finalize evaluation report US 10 TBD Total LOE 44 5. Reporting & Coordination The contractor will report directly to Elizabeth Callender, A/COR for the Program Cycle Services Center. However, daily liaison and coordination for the evaluation team with USAID/Cambodia will be provided by Melissa Patsalides, the PPL/LER member of the evaluation team who will serve as activity manager on behalf Elizabeth Callender. The LER team member will serve as a critical link to the mission staff, in particular for organizing virtual team meetings in advance of the fieldwork and for ensuring team member contributions to report drafting after fieldwork. 5.1. Specific Qualifications LER and USAID/Cambodia requires that the tasks be carried out by a seasoned development professional that has ample experience conducting mixed method, high quality evaluations in developing countries. Priority will be given to professionals with: experience developing, conducting, and leading high quality evaluations; teaching and applying state of the art evaluation methodologies and techniques, and prior experience working with USAID. Experience leading diverse teams to deliver high quality evaluation reports is of priority importance. Specific qualifications desired include: 40 Educational Background  Master’s degree in international development, social sciences, public health, or other relevant field is required. Additional training in evaluation methodologies, operations research, and/or other monitoring and evaluation training is desired. Professional Experience:  Has proven experience leading evaluation teams, teaching the theoretical underpinning of various evaluation methodologies and applying their practical application in the field, and successfully transferring skills and knowledge to learners.  S/he should have at least five years senior experience conducting evaluations and/or designing and managing programs in international development, justice sector reform, civil society development or a related subject.  Excellent English oral and written skills are required.  S/he should have proven experience in preparing high quality reports based on participation of multiple team members/drafters.  Experience in human rights, civil society, justice sector reform, gender and/or SE Asia/Cambodia is preferred. 6. Evaluation Team Members The internal evaluations team will consist of the consultant, who will serve as the Team Leader, one LER staff member, 2 members from USAID/Cambodia (1 M&E point of contact and 1 DG officer) and 1 member from USAID/Indonesia (DG officer). The team will not exceed 5 people, and members must commit to full participation during the entire evaluation process, including preparation, field work, drafting specific sections of the report, and contributing to the finalization of the evaluation report. 41 ANNEX II: EWMI STATEMENT OF DIFFERENCES EWMI Statement of Differences with respect to Internal Evaluation of USAID/Cambodia’s Program on Rights and Justice II (January 2014) March 6, 2014 EWMI appreciates the opportunity to provide its Statement of Differences with respect to the above￾referenced Internal Evaluation. To be sure, EWMI is pleased that the evaluation team recognized many of the accomplishments of the project, and EWMI agrees with many of the team’s recommendations. The purpose of this document is to note areas in which EWMI disagrees with the findings or conclusions set forth in the report. Preliminary and General Comments As a preliminary manner, EWMI regrets that the scope of work of the evaluation did not encompass the legal education reform work encompassed under Component 2. This work has been a key component of PRAJ I and PRAJ II, and the suggestion that the work was terminated or greatly reduced upon the end of the ABA’s participation is inaccurate. The phase-out of the ABA was foreseen at the very outset of PRAJ II, as part of an effort to transition more responsibility to local staff. The title of the evaluation report, which gives the impression that it encompasses the entire project, is therefore misleading. Moreover, one who only reads the Executive Summary will likely miss the fact that an important component of the project has not been addressed by the evaluation. Spanning more than eight years of methodical work, the legal education component has introduced innovative reforms that have touched thousands of Cambodian law students, the future of the legal profession. Most impressively, many of these reforms have been made sustainable for the long-term. The advocacy skills program at RULE, the largest law faculty in the country, has been formally incorporated in the curriculum and now operates without PRAJ assistance. EWMI is currently working to ensure that both the legal ethics class and the legal skills clinic that PRAJ introduced at RULE are similarly incorporated into the curriculum before the end of the project. Client counseling and mock trial competitions, another PRAJ innovation, are now held at a number of Cambodian law faculties without any outside assistance. The national competitions are now supported in part by the faculties themselves, and EWMI will hold the 2014 competitions during the next few months. While it is too early to measure the impact of these efforts in terms of the practice of law, there is certainly testimonial evidence from leading employers in both private law practice and human rights organizations that law students that have emerged from PRAJ-supported legal education programs are the cream of the crop among young lawyers. In terms of the methodology of the evaluation, EWMI takes issue with the evaluation team’s failure to reconvene with PRAJ II staff after the team had conducted field visits and developed its initial conclusions. EWMI recognizes the time constraints on the evaluation team and the special circumstances of a “learning evaluation” intended to develop the capacity of USAID staff; nevertheless, a follow up discussion between the evaluators and PRAJ II staff would have helped place impressions in context, suggested other sources of information, and reduced misunderstandings. Such follow up discussions have been the norm in previous evaluations of PRAJ I and II. “Limitations of the Evaluation” related to EWMI EWMI disagrees with the suggestion that EWMI did not provide all the documentation requested by the evaluation team. There appears to have been a misunderstanding regarding the evaluation team’s 42 apparent request for close-out reports for small grants. The following is the list of materials requested by the team leader on 11 December: “1. Grantee summary report on who got how much for what and for how long; monitoring indicators; outcomes 2. Risk assessment forms and ranking forms used when investigating potential grantee 3. Trends and number of hits to all databases: CCHR, LIDAHCO, ADHOC, ODC, GBV, TIPS, etc. 4. Performance Indicator Reference Sheets for all Indicators in the PMP.” Regarding item (1) above, there were several single event grants under $3,000 that were identified in the grantee report submitted, but without particulars noted. In transmitting the packet on 16 December, the PRAJ II COP noted that further information could be provided if desired. Although the evaluation team leader sent a request on 17 December for additional information on PRAJ II grants (which was responded to that day, apparently satisfactorily), there was no mention made of the absent close-out reports at that time or after. During the period that EWMI was provided to review and respond to the draft report, we were contacted by a member of the evaluation team asking the amounts spent in funding several grants. In responding to that request, the PRAJ II Grants Manager also asked whether, based on the point raised in the draft report about our failure to provide small grant close-out reports, anything further was still desired. Challenges -- Integration of PRAJ II activities EWMI disagrees with the conclusion that there was “not much integration” of PRAJ activities. Despite the structural challenges presented, and some reluctance by partners to share data or fully cooperate with one another, integration of project objectives occurred. The report correctly notes that there were a number of instances where data (and it should be noted, collaborative activity) was shared across partners in the development of thematic reports. In several contexts, EWMI took steps to push the sharing of information more broadly, so that it cut across components of the project. One simple example is actually the aggregation of thematic reports so that the information was directly available to different sectors. EWMI funded CHRAC, in association with the most recent UN universal periodic review of the human rights conditions in Cambodia, to assist civil society organizations in preparing their reports, with UNOHCHR also a partner in this effort. One of CHRAC’s roles was to compile all the submissions and to collect them on-line. This allowed groups interested in particular rights, such as freedom of speech, to see reports bearing on related rule of law themes, such as access to information and for the submissions bearing on justice reform, to be seen by interested persons. This effort was further leveraged by other donor funds, which supported the hard copy publication of the compiled submissions. The report acknowledges various databases that shared information without restriction to specific objectives or sector-based partner sets. Sithi.org publishes information bearing on human rights violations but also puts emphasis that bear on the need for justice reform, such as impunity, supporting another of PRAJ’s objectives. ODC offers information on the country’s development in a format that, without an advocacy agenda like Sithi’s, also provides information bearing on rights abuses, failures in the justice system and links these across to other USAID objectives as well, such as environmental protection and increased government transparency/accountability. It is incorrect to conclude that there was not sharing of strategic litigation insights between human rights advocacy organizations on the one hand and legal aid NGOs and PILFs on the other. While grants for the two sets were managed separately (by TAF and EWMI, respectively), both were included in working groups, aimed at information sharing and joint training, that EWMI organized for public interest 43 lawyers. They not only shared successful strategies at these meetings, they created them there. In the “Big Ocean” labor trafficking case, various partners including LICADHO, ADHOC, LSCW, IBJ and LAC, discovered at one of these meetings that they all had clients who had been victimized by the same company and made plans for joining the cases and obtaining international assistance. The idea of creating a shared defenders database of criminal cases documenting common procedural errors by the courts and poor administrative management (which is among the evaluations recommendations), was agreed on – after discussion by an HR advocacy organization and a number of LA-NGOs and a PILF -- at one of these working group meetings. Sharing also occurred outside the working groups across EWMI partners. In preparing the appeal of Mam Sonando (an acknowledged success among human rights cases) and his co-defendants, insights were shared by LICADHO (though it did not have a lawyer on the case), ADHOC, Vishnu (one of PRAJ II’s PILF partners) and CCHR, alongside several private attorneys. “Missed opportunities”: Offsetting costs of PLCN Forest Patrols We agree with the evaluation team’s insight that offsetting the lost revenue opportunity associated with forest patrol time would promote the sustainability of such patrols. PRAJ II’s GNA advisor explored this issue several years ago, but received guidance from USAID that an SMSE project that promoted the value chain of non-timber forest products would address this, not PRAJ. While the assistance that project provided ended up benefitting Community Forests rather than other PLCN members, another opportunity is presented by the current USAID Supporting Forests and Biodiversity (SFB) project. Under its FY14 workplan, that project is exploring ways to factor the costs of forest patrols (which serve to protect resources supporting livelihood, such as resin) into the market price of those products, such as resin, when they are sold. In keeping with USAID’s previous guidance, the evaluation team’s suggestion of cultivating wild rice will also be transmitted by PRAJ staff to the SFB managers. Given the above, EWMI disagrees with the suggestion that the income generation issue was an opportunity missed by EWMI. “Missed Opportunities”: Donor collaboration While recognizing that EWMI collaborated with other donors on a number of initiatives, the evaluation team seems to suggest that the relative inactivity of the Law and Judicial Reform Technical Working Group, which served as a donor collaboration forum, is a missed opportunity for PRAJ II. To the contrary, EWMI has used this forum as a means for advancing policy and programming of joint interest. In the summer of 2013, EWMI proposed, through the group, collaboration on a joint position statement from donors that could impact on the shaping of Cambodia’s National Strategic Development Plan (NSDP). The effort was successful, with the contributions and collective voice of numerous donors enlisted; USAID, AUSAID, JICA, UNOHCHR, UNICEF and others participated in preparing a joint statement that pushed for inclusion of HR/ROL issues in Cambodia’s next five-year strategic plan. EWMI recently learned that this effort has borne fruit. The latest draft of the NSDP includes a number of the targets that EWMI advanced with the group’s support, including improved judicial information systems and increase public access to that information. EWMI has used this body to discuss common interests in legal aid funding programming, gaps in and opportunities for justice procedure reforms, conditions of and responses to human rights threats, mechanisms that might reduce corruption, and other programming issues of relevance to PRAJ II. EWMI staff has, at relevant points, participated in other donor technical working groups, including the group addressing land use issues. “Missed opportunities”: Joint planning with CBOs/CSOs/GNs Rather than convening collaborative venues itself, PRAJ II has supported, participated in, and promoted partner participation in the collaborative initiatives of local NGOs wherever possible. This approach is 44 intended to value and build local capacity for leadership and action. This approach was based largely on feedback and comments made by PRAJ partners and others that INGO donors should convene fewer processes themselves and be more supportive of local initiatives. With that in mind, PRAJ (both EWMI and TAF) participated in, advised, and supported partner participation in collaborative venues that included: NGO Forum’s Forest and Plantations Network and their Rivers Coalition of Cambodia; an NGO Forum led process on Joint Monitoring Indicators; the Extractive Industries Social and Environmental Impacts group of which EWMI was elected to the steering committee for three years in a row; Cambodians for Resource Revenue Transparency; Cambodia’s Climate Change Network; REDD+ dialogues and processes convened by government, CSOs, and UN agencies; a UNOHCHR regularly convened discussion on ELCs and other land developments and how to document and respond to them in a collaborative way. TAF regularly joined and provided legal advice to HR agency responding to critical land conflicts. All of these entailed significant co-planning together with partners and others. EWMI maintains that this approach was both more respectful of local NGO initiatives, contributed more to capacity building, and was more sustainable. Capacity building approaches EWMI disagrees with the conclusion that PRAJ II capacity building did not incorporate empowering models and was not systematic in its approach. PRAJ I and II followed an intended development arc. PRAJ I offered more conventional training including OD training, as well as information sessions on new development trends, consultative meetings, etc., in an effort to 1) address organizational issues deemed critical at that time, and 2) support transition from information and analysis to action. EWMI strongly disagrees with the statement that under PRAJ I, participants in EWMI’s training and capacity building workshops believed that the workshops “were not as useful as they could have been.” An independent final evaluation of PRAJ I, which was based on the detailed polling of 31 of 37 EWMI grantee partners, in-depth interviews with 13 grantees and related field visits, concluded to the contrary that PRAJ I partners found substantial value in the training they had received. By design – and based on the expressed desires of partners – EWMI moved away from conventional workshops during PRAJ II, as these were starting to take up too much time, especially since other donors were doing similar training and partners were reporting that they needed to shift their focus to the actual work. Our shift moved away from setting up meetings and trainings to participating in various meetings called by the NGOs themselves, and providing advice, helping to resource, etc. The organic process that PRAJ II adopted was not without structure. A PRAJ II program officer (with the project for three years) spent significant time in assessing and addressing needs in an ongoing manner. This led to very tailored capacity building interventions, which included assisting partners with guided report-writing, reviewing their accounts with them, inspecting documentation in field offices and giving guidance on how to improve it. It also led to specific trainings that groups requested following their own reflection and assessment processes. The trainings and/or guidance and assistance included subjects such as land law, forest rights, IP rights, complaint-writing, local governance processes, commune investment planning, working with the media, and writing press announcements, among others. Importantly, there was also a structure to the assistance and occasional facilitation that EWMI provided to the PLCN, timed to their perceived needs but also guided via reflection that helped the fledgling network become more aware and work through various organizational issues. Hence they rejected a traditional NGO structure, adopting a more representative framework, and focused energies on building network membership. This process was positively reflected on in the assessment of grassroots groups undertaken last year by a respected local organizational consultant. (A general indication that – at some 45 point in the future – PLCN members would like to exist as a formal entity that can independently manage its funds, is not inconsistent with the decisions it has made thus far.) Structured assistance by CORD was also built into a grant for CPN to help it address its organizational issues. This eventually led to CPN registering its secretariat and starting to work through significant leadership and membership issues. It should be noted that EWMI’s documentation programs at the MoJ were built precisely on the model described in the evaluation report: "needs assessment" (on file), "development of learner-centered activities offered by a skilled facilitator who is knowledgeable about the country" (three-day retreat with all MoJ department managers to design Intranet, five-week initial trainings on basic documentation and computer skills for 100 MoJ staff, offered by Sam Sophal --- a five-year veteran of CCC, and close day￾to-day colleague of MoJ management and staff). Working with government The evaluation report overstates the effect of the 2011-2012 design modification related to engagement with the government, and suggests that PRAJ II ignored opportunities to work with government while its subgrantees did so successfully. Following the elimination of Objective 3 based on USAID’s determination that there was a general lack of shared objectives supporting work in the justice sector, PRAJ II personnel have consistently looked to work in cooperation with Cambodian officials in sectors and situations where opportunities were presented. This includes the development of the criminal case database at the PPMC, the new program enhancing government capacity to prosecute cases associated with illegal pharmacies and counterfeit pharmaceuticals, and a number of smaller collaborations noted below. As noted above, PRAJ II has trained more than 1,000 government justice officials and continues to do so at present. Such efforts on EWMI’s part also included several instances where the PRAJ COP and Bar Liaison/Grants Manager accompanied legal aid partners to meetings with the presidents of provincial courts to request improvements in court administration (such as notice to the defense lawyers of the closure of the investigative stage of proceedings, timely notice of hearings, etc.), which generally resulted in the courts’ commitment to improve the specific matters raised. PRAJ also supported training of court clerks form the various provinces in preparing administrative reports on cases using computers (this was also aimed at encouraging other courts to adopt the PPMC’s criminal case database). PRAJ II’s provision of training to its lawyers working groups (which has regularly targeted training on issues arising from changes in law and procedure) has enlisted judges as trainers and the legal education unit has included judges and prosecutors as judges in its annual competitions. As noted above with relation to legal aid lawyers meeting with court presidents, grant partners have been supported in engaging with government officials (CCHR’s trial monitoring program is supported by a grant designed, for example, to include the courts monitored as partners, and we supported CCHR in taking the time necessary --- despite consequent delays – to get the judges on board). It is worth noting that in February 2013, the Cambodian Minister of Justice requested that EWMI’s MoU with the Ministry of Foreign Affairs be extended for three years, to support EWMI’s collaboration with officials in Cambodia’s justice sector. “Shifting Priorities” -- LANGO While EWMI’s substantial contribution in pushing back the LANGO in 2010-11 could be partially reversed if the draft returns for parliamentary consideration in 2014 and is adopted, there are several 46 impacts that will not be undone even if that occurs. The space for civil society that was preserved through 2013 contributed to the political shift surrounding the July 2013 general election, which appears to be a game-changer (the opposition platform and stump speeches made heavy use of themes that PRAJ subpartners and other CSOs had been emphasizing between 2010-2013, among these, the lack of a fair and independent judiciary and rampant land-grabbing that was destructive to the rights of occupants and the environment). Moreover, the effort EWMI supported in pushing back against the LANGO brought CSO solidarity to a new level. With direct facilitation by EWMI, the informal groups opposing the LANGO agreed to join in a statement that had been developed by the NGO community. So the LANGO effort was not so much a shift in priority. While it responded to an exigency, it accomplished this by deepening cross-CSO collaboration. It promoted and elevated the use of evidence-based advocacy (by providing a legal expert who, in collaboration with ICNL lawyers, deconstructed each of the four LANGO drafts and exposed their inconsistencies with international standards), which USAID had expressed as a priority in early 2010, well before the LANGO’s appearance. And the aim was the maintenance, if not expansion, of social space for civil society. Open Development Cambodia Although the evaluation team recognized the effectiveness of the ODC platform, it nonetheless expressed concerned “with the locus of the development of this database as well as the funding that launched the initiative.” Apparently this conclusion is based on an interview with an NGO leader who felt that EWMI should have engaged an existing NGO to develop it and the comment of a CCHR representative that its sithi.org database was similar to ODC. This conclusion reflects a misunderstanding of the circumstances surrounding the creation of ODC and the premise underlying the platform. USAID requested that PRAJ try to facilitate start-up of a land platform, tracking both land concessions and related land cases; this was supposed to be the NGO Forum Land Center. PRAJ funded that for about a year, but it never took off and NGO Forum declined continued funding for it. They hobbled along for some time, not producing anything, and afraid to publish. Rights NGOs would not cooperate with it (a similar problem faced by the subsequent sithi.org platform). ODC is a very different kind of database. It does not track violations in the same way that sithi does. Moreover, ODC did not come at USAID’s specific request (though it responded to USAID’s request that PRAJ develop programming that would also constrain corruption by promoting transparency and accountability). ODC represents a new strategy that was born out of EWMI’s work with grassroots groups, demonstrating the need to: 1) Get more information into the public; 2) Aggregate information across various sectors (as EWMI did this for its Prey Lang work, issues that had been hitherto hidden started to become evident, convincing EWMI of the need to share it.) Before this information was siloed and rights groups were generally unaware of other implications beyond the current issues; 3) Transfer the aggregation and sharing of information away from an advocacy or even specific development agenda, given the government’s -- and even the publics -- propensity for rejecting what they saw as biased data. 4) While remaining able to relate this data independently to advocacy. As its name suggests, ODC is driven by an open data approach. Given that open data is new in Cambodia, EWMI did initially attempt to shop the project out to an appropriate group. Even though NGO Forum is an advocacy group, its position suggested that it might be able to do the project with firewalling. However, NGO Forum refused to entertain the idea, saying it was too dangerous. CCHR, LICADHO, and ADHOC were aggregating certain kinds of data, but were insistent on linking this to 47 advocacy. They also refused to cooperate with one another. Other groups including think tanks and research groups also felt the work was too sensitive. Finally, EWMI also tried to interest start-up TI in the project but the organization was still too young, and the director did not demonstrate an understanding of an open data approach. This led EWMI to look for a way to start up a group that would represent a new capacity and approach. ODC was born out of that effort. Recently, a Columbia graduate student group working with ODC to build out a donor and development assistance page, heard EWMI’s strategy vindicated in discussions with donors, who noted that ODC’s practice of presenting data objectively and outside an advocacy framework is proving a valuable way to get data into the public and government eye. PLCN EWMI believes that the time and funding constraints under which the evaluation team visited Prey Lang may have resulted in skewed perceptions about the attitudes of PLCN core group members. By interviewing only two of the 20 core group members, both from the same province, the team was unable to get a broader sense of the group vision (as the evaluation notes, the two interviewed members apparently were unaware that the network encompassed hundreds of villages in four provinces). Regarding the registration issue, registering the group earlier on would have been -- and even now is -- premature for several reasons, including capacity (which has increased significantly but which still remains low), continuing leadership challenges in a country where strong-man models dominate but are not appropriate for networks, and the fact that at this stage PLCN would have to register as an advocacy group (if it would be even allowed to register at all) almost certainly making it ineligible for the forest management role that it is pursuing. EWMI believes that the PLCN is currently in a stronger position remaining unregistered but increasing its membership and improving its leadership. It should also be said that for a long time the PLCN has expressed the view that it did not want to register, and it is still not clear that the leadership group as a whole would agree to do so (and they do have periodic discussions about it.) Arguably, in the Cambodian context, registering the group might be the fastest way to destroy it. EWMI also takes issue with the suggestion that PRAJ did not provide adequate attention and support to PLCN. Early in PRAJ II, Pyrou Chung, PRAJ II’s field coordinator and a Khmer Australian, did visit PL villages regularly and even spent days at a time with groups in the forest. When her position was discontinued after the first year, for funding reasons, our program officer Horn Kim Hong assumed a similar role. EWMI viewed this this as an appropriate transfer of responsibility to competent local staff. It was not practical for the GNA advisor to do this level of field work. Both because of a work load that included more than the PLCN, but also because the work was best done in fluent Khmer by local staff. PLCN never requested reflection meetings – run by NGOs, including EWMI – to be held at village level. Nor was EWMI funded or staffed to do this level of work. Earlier PRAJ II partners (OPKC, EHE, and CED) were funded to do this and did to the best of their ability. They were sometimes assisted and advised by program officer Kim Hong. As those partnerships were phased out, PLCN members were encouraged to take on this role themselves, as the most appropriate way for them to build their own membership. In fact, for at least the past year, security issues also affected how EWMI and other NGO workers could operate at village level. Village level work was most appropriately undertaken by network members themselves. This also contributed to their own ownership of processes, discouraging them from becoming NGO dependent. Except for a period when managers of the Supporting Forest and Biodiversity project asked EWMI to defer overt activities with the PLCN because of the July 2013 general election and issues associated with sensitivities at the Forest Administration (and some periods between staffing changes), EWMI has not 48 been at a loss for how to help the network. It has remained committed to supporting, facilitating when possible, and advising processes that have helped the PLCN build both an internal network constituency and an external constituency for protecting and sustainably managing the forest. This has included the incremental growth of the network from a loose affiliation of advocates to a more genuine network represented by an elected core leadership group (of their design), an assembly of representatives, and affiliating villages. This is an immense achievement in the Cambodian context where organizational issues have led repeatedly to the demise of such groups. The fact that EWMI has not rushed or forced the process but has attempted to match pace with the network’s organic development is one of the reasons that the network has not dissolved into the kinds of conflicts which have led similar groups to collapse. 49 ANNEX III: USAID STATEMENT OF DIFFERENCES Statement of Differences with the Final Evaluation of the Program on Rights and Justice II (PRAJ II) USAID/Cambodia USAID welcomes the final evaluation of the Program on Rights and Justice (PRAJ) II, and USAID/Cambodia is pleased to see that the evaluation team identified many positive outcomes of the project. Indeed, one of the report’s main conclusions was that there were likely positive results and outcomes that were not adequately identified through the indicators. While USAID acknowledges some weaknesses in the project, particularly with respect to monitoring and evaluating (M&E), USAID holds the view that the project has on the whole made a significant contribution to the protection of legal and human rights in Cambodia, and provided crucial support to civil society in Cambodia. In particular, USAID/Cambodia agrees with the team that there could be improvements to the M&E framework, and appreciates the team’s much elaborated emphasis on measuring incremental changes instead of numbers. USAID also appreciates the evaluation’s recommendations at the end of the report and will consider incorporating them into future programming. On the other hand, USAID would like to express differences with some aspects of the evaluation. USAID/Cambodia notes that time limitations may have affected the quality of the evaluation team’s output. USAID also recognizes that the project is far-reaching and difficult to evaluate, which may have led to some misunderstandings. A finding indicating a misunderstanding of the Cambodian context concerns the decision to reduce support for collaboration with the Ministry of Justice on judicial reform. The evaluation team wrote that “The basis for the design modification in 2011-12 (i.e., the unwillingness of the RGC to engage in reform) appears erroneous as every key informant interviewed for this evaluation identified productive working relationships with RGC entities”. The evaluation team seems unaware that national level entities, such as the Ministry of Justice, are completely different than local level officials. Local level officials in Cambodia do not make policy, nor are they able to have a significant influence on reform. At best they can ensure that existing policies are implemented in practice. USAID/Cambodia encourages its partners to engage with sub-national officials if it leads to improved outcomes in specific cases or groups of cases. However, according to long-term observation of Cambodia, it is not realistic to expect that changes in action by local authorities will necessarily lead to a national level policy change. Finally, USAID is willing to engage with the Cambodian government on judicial sector reform when it appears that there is genuine political will to enact reform. However, in the context of limited funding for human rights and support to the judicial sector in Cambodia, it did not make sense to continue this group of activities in 2011 or 2012. USAID/Cambodia notes that other donors have made the same assessment and conclusion. Indeed, recently the Cambodian government introduced three judicial laws – without any public consultation – that local and international experts fear will further consolidate the power of the judiciary. One of the team’s main recommendations includes “forming an association/ coalition of actors seeking to create a more democratic society”. Without a more precise definition of who they are referring to, it appears that the recommendation is unrealistic, as there are a multitude of organizations and donors involved in “creating a more democratic society”. The evaluation then states that the Mission should “assume a leadership role in the Technical Working Group (TWG)”. While there is a Technical Working Group on Legal and Judicial Reform, which both EWMI and USAID play a central role in, there is no TWG for human rights or democratic processes. A TWG in these sectors might be desirable, but 50 its existence depends on the RGC’s initiative in addressing these issues in the National Strategic Development Plan. The evaluation then proceeds to suggest pooling of donor funds and harmonizing M&E policies across donors. While one donor is potentially interested in pooling some funds with USAID, others have indicated that this would be difficult for them. M&E harmonization would also be challenging due to donors adhering to policies made at headquarters, not at the country mission level. Claims that donor pooling would reduce time-consuming competition between NGOs, or that CSOs could agree to divide up their work into different informational niches also seems unrealistic given the competitive nature of CSOs in Cambodia. On the issue of sustainability, USAID/Cambodia notes that the team was asked to answer the question “Which individual Non-Governmental Organizations (NGOs) or strategic partnerships between NGOs show evidence of sustainability, in terms of a) capacity to function effectively, b) having a constituency which is interested enough to maintain the organization, or c) financially sustainable (or moving in a direction of sustainability outside of sole-USAID grants)?” However, the team chose to focus entirely on the issue of c), financial sustainability, ignoring the first two types of sustainability, about which USAID/Cambodia was very interested to gain deeper insight. In focusing on financial sustainability, the evaluation team repeatedly writes that none of the sub-grantees, with the exception of IDEA, have developed strategies to be more financially sustainable. USAID questions the practicality of finding sources of funding outside of bilateral donors for these human rights and legal aid organizations. In an extremely poor country, domestic charity is not well developed and individuals tend to give to humanitarian causes (if they give at all). The private sector in Cambodia, while occasionally engaging in charity, typically supports apolitical and non-controversial activities that would not endanger their business interests in a national context in which their right to conduct business depends on patronage relationships. Human rights monitoring, advocacy campaigns, and judicial reform activities are unlikely to receive private sector support in the near future. While USAID/Cambodia acknowledges that in theory local organizations could conduct fundraising targeted to the public, these activities would be more labor intensive with limited returns, and could draw human resources away from these organizations’ valuable mission. On the issue of “shifting priorities”, USAID/Cambodia also acknowledges that the priorities of the project changed slightly over the five years of its implementation. In fact, this is a donor best practice as projects should be adjusted to pursue successful interventions and ensure efficient use of resources. In particular, supporting a response to the Law on Associations and NGOs (LANGO) was a strategic decision taken to defend the role of human rights, advocacy and legal aid organizations whose space to operate could have been curtailed by the proposed legislation. On the request to set up a data sharing website that tracked land concessions and land cases (eventually leading to the creation of Open Development Cambodia), USAID/Cambodia asserts that this is within the broad objective of the award to create information that supports evidence-based advocacy, including on rights-related issues. Further, ODC has been very successful, incorporating ever expanding data sets and attracting support from other donors. Due to its neutrality, it is used as a resource by the private sector, civil society, and government officials to inform decision making. With regards to the creation of ODC, the evaluation team raises a number of questions without making any specific findings. In its statement of difference, EWMI responded with several points explaining its decision to develop ODC itself, including that 1) other organizations were unwilling to create a website which would not have an advocacy component, 2) several organizations were not interested due to their perception that it was too sensitive, and 3) several organizations did not show a strong grasp of the advantages of an ‘open data’ approach. USAID/Cambodia is satisfied and proud of the success of ODC as an information sharing platform which can be used for evidence based advocacy. 51 USAID/Cambodia also questions the finding that there was a “limited breadth of constituencies that were provided with grants to work on reform”. Throughout the implementation of PRAJ II, the project has provided sub-grants to a large number of organizations- between 15 and 20 at any given time. In addition, the project provided small grants to a number of activities such as International Women’s Day and Pride events, supported legal education and law student competitions, and implemented the ODC website as already noted. USAID/Cambodia questions the efficacy of attempting to fund an extremely large number of organizations and questions whether other donors would pursue this strategy either. EWMI broadly advertised its grants program and provided sub-grants within the constraints of funding that was available. The evaluation also suggested that more awards could have been issued if additional avenues to building constituencies for reform had been considered. Yet within the Cambodian context, it would be difficult to build constituencies for reform in a way that the RGC would not consider to be overtly related to human rights or “politics”. With regards to the issue of Prey Lang Community Network (PLCN) members desiring to form an NGO, rather than remain as a network, USAID shares EWMI’s perspective that there may be risks associated with becoming a formal NGO. While becoming an NGO may enable PLCN to access donor resources, it also risks control of PLCN being seized by the leadership of the NGO and losing its participatory, associational character. That said, EWMI should not prevent PLCN from becoming an NGO if that is their wish, but EWMI should not and did not encourage that either. On a related note, USAID/Cambodia agrees with the recommendation to supplement work with grassroots networks by providing training or resources in livelihood development to generate income. With regards to capacity building for partners, PRAJ II did provide financial skills training for partners, which enabled them to better comply with USAID requirements. EWMI has already detailed how PRAJ II provided capacity building on report writing, accounts maintenance, and field documentation, as well as training on advocacy techniques and law, such as media liaison, complaint writing, land law, and commune processes, among others. USAID/Cambodia also finds a contradiction between the evaluation’s “concern… that more of a patron-client relationship emerged rather than a true partnership”, and its emphasis on the need for capacity building. An excessive emphasis on capacity building (also often referred to as training) would establish or reinforce a patron-client type relationship by emphasizing the perceived needs and deficits of the sub-grant recipients. Nevertheless, USAID recognizes the need for continued capacity building for CSOs and intends to build this into future programming. USAID also agrees with EWMI that the evaluation team may have misunderstood the reporting process. Sub-grantees were not required to submit weekly reports, but could submit material reflecting the week’s activities, which would help them to accumulate material for quarterly reports. As EWMI stated, much of the material is gathered from the press or by telephone calls, which should not place much burden on the recipients. This information is useful to Mission leadership, particularly for a far reaching project that involves a large range of partners involved in diverse activities. Further, maintaining regular contact with and oversight of the sub-grantees was EWMI’s responsibility to ensure that U.S. government funds were being used to fund activities appropriately. Overall, USAID appreciates many of the evaluations findings, particularly its emphasis on measuring incremental change as a monitoring and evaluation tool. USAID also notes that the evaluation recognized many positive results from the project. At the same time, some of the findings were inaccurate, unrealistic, or failed to sufficiently consider the Cambodian context. USAID appreciates the recommendations made by the evaluation team and will consider many of them for similar programming in the future. 52 ANNEX IV: CHANGES IN PMP AND M&E INDICATORS Explanatory Note on Evolution of MEP and List of Historical Indicators3F3F 34 Evolving from PRAJ I, PRAJ II was designed to support development of improved justice and human rights policy through direct technical assistance to government officials, as well as assistance to civil society. At the beginning of the program, PRAJ II had productive working relationships with the Royal Academy of Judicial Professionals (RAJP), the Ministry of Justice (MoJ) and the Ministry of Interior (MoI). A significant component of PRAJ II’s programming was premised on the successful rollout of the Model Court Project, a primarily Danida-funded effort driven jointly by the MoJ and the Commission on Legal and Judicial Reform (CLJR). That Project faltered when, in 2010, Cambodia failed to match funds offered by Danida (and for other reasons). In his November 2010 submission supporting USAID’s portfolio review, the PRAJ COP proposed several changes in indicators. These were formally resubmitted in early March 2011, and proposed dropping/adjusting RF indicators and related PMP (MEP) indicators that were intertwined with the Model Courts. Under Objective 2, this included dropping “new generation judges” (after EWMI’s training partner, the RAJP, stopped training new judges, and whose performance could not be correlated with key statistics because of rotation/reassignment, among other things). A number of indicators were dropped from Objective 3, because the activities had depended on the now moribund Model Court Project. Two were salvaged by changing the term “model courts” to “partner courts” with the expectation that EWMI would leverage its own relationships with individual courts. The remaining indicator removed was a casualty of government restructuring, when the portfolio that included investigation and mediation of land disputes was transferred out of the MoI office EWMI had partnered with. Indicators were also added; one under Obj. 2 relating to courts’ performance in reporting CTIP case data because, with USAID/State encouragement, PRAJ engaged in providing training and database development to support this effort. An indicator added to Obj. 4 captured the development of an online information platform (now Open Development Cambodia). USAID had provided a soft earmark of funds in 2011 for counter-corruption programming; it was to be applied in developing an online platform that would increase government accountability for allocation/exploitation of natural resources, which linked closely to Obj. 4. An indicator was also added (or rather moved from Obj. 1) to Obj. 5, tracking excessive pre-trial detention. With the collapse of the model court program, the monitoring was seen as more appropriately conducted by legal aid partners. The definition was changed from “6 months” to “beyond legal limit” to make it applicable across felony and misdemeanor cases. The strategic underpinnings of PRAJ II shifted substantially in August, 2011, when PRAJ received instructions from USAID that, in preparing the FY12 Workplan, work with ministries and agencies of the Cambodian government should cease because there was lack of “shared objective” on the government side. This view was well-supported, and accepted by EWMI staff managing PRAJ II. The Model Court Project, which was stymied the year before, had shown no signs of moving forward, and the CLJR’s entire program of legal and judicial reform appeared to lack political will for advancement. EWMI’s partnership with the MoI had come to a fork in the road in December 2010, when the Ministry produced a draft Law on Associations (LANGO) presenting a significant threat to civil society. USAID’s instruction was not unreasonable. 34 Excerpted from Boname, Andrew. 2012. PRAJ II One –Year Extension Proposal. Phnom Penh: USAID. 53 Exceptionally, EWMI requested and received permission from USAID to provide basic maintenance support to EWMI’s TIPs/GBV database (because of its support to civil society organizations advancing women’s rights that benefited from the data generated). Likewise, EWMI was permitted to advance a pilot criminal case database, already in development, at the Phnom Penh Municipal Court, because it would help check excessive pretrial detention and provide access to case information for partner legal aid NGOs. The impact on indicators was immediate at the beginning of FY12 in that the workplan eliminated activities supporting the government. The indicators themselves were changed in January 2012, in association with identification of end-of-project targets, as recommended by an OIG performance audit. The indicator changes included, most obviously, elimination of those that had then been tracking Obj. 3, which had directly supported judicial administration. Other changes made at that point included, as to Obj. 2, dropping the CTIPs case reporting indicator, because EWMI’s role in this area had declined (in part due to EWMI’s diminished relationship with the MoJ). Indicators for Obj. 4 were all retained but were modified to better capture activities bearing on civil society organizations, rather than just community-based organizations, and to more closely align with OP indicators. One indicator in the set, which had referenced application of the NGO advocacy index, was modified to apply a set of criteria different from the indexes because, when EWMI was initially preparing the TOR for a pre-assessment of the networks’ strength, it determined that some elements of the index could not apply meaningfully to Cambodian Grassroots Networks. An indicator under Objective 5, which captured data on the number of criminal defendants receiving representation, was seen to completely overlap with another “The No. of indigent clients represented,” and discarded. Others related to pretrial detention, were also modified. One relating to juvenile detention was dropped because the legal aid partner (Legal Aid of Cambodia) that was engaged on this issue indicated that the problem was largely eliminated (they indicated that there was no point in funding them for this program any further). The remaining pretrial detention indicator was transferred back to Obj. 1 (where it had begun) and framed in terms of systematic monitoring against the problem. This was for two reasons. First, it linked the indicator to expected impact from the pilot criminal case database. Second, and more importantly, because excessive pretrial detention has substantially diminished generally, future effort should focus on establishing a monitoring mechanism that keeps it in check. 54 EWMI M&E Plan and PMP To illustrate the types of changes made in the M&E Plan and PMP that led to a number of project disconnects, we present the objectives and indicators developed as a result of the mid-term evaluation and program audit. Objective 1 1. Percent of specific objectives met for each partner organization for establishing an effective and comprehensive information system (partners here are defined to be those Human Rights Non-governmental Organizations (HR NGOs) provided grants by EWMI and The Asia Foundation (TAF) to record and investigate HR violations in the Phnom Penh Municipal Court (PPMC) and in other regional locations. The organizations include LICADHO (which itself keeps its own database), CCHR (the keeper of the HR database, keeper of the sithi.org website, and publisher of annual thematic research), ADHOC, CHRAC, CLEC, the NGO Forum. 2. Number of organizations that are using information systems and data effectively (determined through a rapid appraisal data collection methodology; no summary report has been received by the team). 3. Percent of criminal cases systematically monitored to determine excessive detention through establishment of database and reporting requirements and protocols to determine the number of individuals held in excessive pre-trial detention or beyond their sentence. Results to be achieved: 1) Independent judicial performance monitoring system established 2) Quality Control and Advisory Commission established 3) Court performance system established Objective 4 1. Number of advocacy initiatives supported by target CSOs (grantees) (definition of “initiatives” problematic and so became all inclusive, e.g., meetings or speaking on radio, rather than focus just on community activities) 2. Number of citizens engaged by target CSO to build and broaden the constituency for reform (disaggregated by Grassroots CSOs and National CSOs) (numbers problematic as those trained were far fewer than those “engaged.” Grantee organizations include ICSO, CED, DKA, CCD, AEC, IRAM, Prey Lang Community, Open Development Cambodia (ODC), and others to be identified.) 3. Improved quality of networks defined as affiliations among citizen’s groups around common objectives. Criteria for improvement include: recognizes and identifies opportunities; able to mobilize funding; has a clear set of objectives; establishes values and a longer term vision to guide its work; establishes a mechanism to elicit input by constituents; establishes process to engage constituents in advocacy responses; establishes a process to share information; is transparent in terms of financial reporting or other questions that may emerge from constituents; and reports events, issues or other information to constituents. 55 Results to be achieved: 1) Increased number of local land activists/local networks functioning 2) National NGOs more effective at local level constituency building 3) Increase in stakeholders from business/professional associations 4) Increased number of government-constituency group discussions Objective 5 1. Number of indigent clients represented by target organizations disaggregated by NGO and public interest law firm as well as criminal defendants, and gender. Target organizations include IBJ, LAC, VSLG, CDP, SLG, WMC, LSCW, LWG, RISC, IDEA, FIT, Samreth and Vishnu law firms, and others to be identified. 2. Number of cases handled by target organizations disaggregated by NGOs and public interest law firm. 3. Percent of respondents that are aware of key rights and know where to go for legal aid disaggregated by gender; data obtained from survey administered at end of program (and compared to baseline conducted by Benetech). (If this study has been undertaken, it has not been shared with the team.34F34F 35) Results to be achieved: 1) BACK and private bar increases involvement in legal aid 2) Increased percentage of represented criminal defendants 3) Reduction in pre-trial detention time for juveniles 4) More Cambodians are aware of and seek legal aid Although results to be achieved were identified, data were not connected and research not undertaken to comment on whether these results were achieved and whether these results had an impact on the achievement of the project goals. TAP RF and M&E Indicators As a partner with responsibilities for providing small grants and monitoring their use, as well as of providing training and capacity building to grantees, TAF had its own Results Framework and M&E Plan with indicators. They are as follows: 35 According to EWMI’s PMP dated February 2012, a baseline value was provided (80% of respondents of the CLJR Legal Aid Study did not know the term “legal aid.” 56 Results Framework Objectives 1. To strengthen user-friendly web portal infrastructure for open online documentation of human rights violations and dissemination and use as a tool for online advocacy 2. CSOs are empowered through ongoing training 3. The situation of human rights in Cambodia is collaboratively monitored and documented – the CCHR and Project participants, collaboratively and separately, use the Project infrastructure to monitor and document rights violations relevant to their specialism 4. Varied information on the situation of human rights in Cambodia is shared 5. Detailed and collaborative research and analysis is undertaken on human rights 6. CSOs organizational strength is developed M&E Plan Indicators 1. Human rights violation cases documented from media monitor 1.1Cases monitored and documented during this reporting period 1.2Case referral (cases received through hotline and referred to other organizations) 1.3Documentation on Sithi.org 2. Advocacy Initiatives 3. Constituency-building activities 4. Training activities TAF’s QRs also report on outputs and not outcomes. Hence, it is not clear what the results of this small grants and capacity-building program have been and how they are related to the goals of the project. 57 ANNEX V: TAF MONITORING AND EVALUATION REPORT FOR PRAJ II GRANTEES Monitoring and Evaluation Report for PRAJII Grantee Contents 1. M&E info for Actions for Environment (AEC) p.2 2. M&E info for Community Capacities for Development (CCD) p.3 3. M&E info for Community Legal Education Center (CLEC) p.4 4. M&E info for Dai Ku Aphiwat (DKA) p.5 5. M&E info for Fisheries Action Coalition Team (FACT) p.6 6. M&E info for Cambodian Human Rights and Development Association (ADHOC) p.7 7. M&E info for League for the Defense and Promotion of Human Rights (LICADHO) p.9 8. M&E info for Cambodia Human Rights Action Committee (CHRAC) p.11 9. M&E info for Actions for Cambodian Center for Human Rights (CCHR) p.12 10. M&E info for Actions for Children Development Association (CDA) p.15 11. M&E info for Indigenous Community Support Organization (ICSO) p.16 12. M&E info for Community Peace-Building Network (Cord/CDPS/CPN) p.17 13. Definition of certain terms of indicators p.19 14. Technical Assistance Provided by The Asia Foundation to Sub-Grantees p. 24 58 Name of Organization Action for Environment and Communities (AEC) Grant per period January 2009-September 2010 2010 2011 2012 2013 US$49,938 US$45,571 Terminated Composition of Organization Ms. Sor Sat, Executive Director Mr. Heang Sokun, Program Officer Activities Target Area 1. Organize and facilitate women and youth groups so that women and youth can participate and have a voice in advocacy by their communities 4. Two provinces:  Toek Phos and Boribo district, Kampong Chhnang province  Krakor, Pursat province 2. Help build commitment and interest among members of local communities to participate in using their rights to protect their interests, build their skills in analysis, unite people in village-level networks for the ultimate purpose of community advocacy, and promote collective leadership. 3. Facilitate a local level community network that unites communities in three districts for advocacy on land and livelihoods rights 4. Facilitate the local community networks in linking with community networks at the national level and contacting civil society organizations to seek support for their advocacy efforts M&E Indicators Number Indicator 1: Number of communities organized AEC organized 7 forestry communities, 8 women groups and 7 youth groups. Indicator 2: Number of participants trained Total 992, including 663 women Indicator 3 Number of citizens participating in the constituency building 11,030 (6,535 women) Indicator 4 Number of advocacy initiatives 89 actions Overall outcome from grant 59 Name of Organization Community Capacities for Development (CCD) Grant per period January 2009-September 2010 2011 2012 2013 US$330,415.00 Terminated Composition of Organization Ms. Yim Leang-Y, Executive Director Ms. Sum Vicheka, Finance Officer Ms. Seng Kim Sroeu, Project Manager Activities Target Area 1. To organize communities in three districts in Kampong Chhnang province  Toek Phos and Boribo district, Kampong Chhnang province  Krakor, Pursat province 2. Provide support to the Community Peace-Building Network to conduct local and national advocacy, to build community capacity in active non-violence, leadership, and to document community issues M&E Indicators Number Indicator 1: Number of communities organized CCD organized 60 communities included 28 youth groups Indicator 2: Number of participants trained: Total (women) 1,516 (719 women) Indicator 3 Number of citizens participating in the constituency building: Total (women) 5,477 (2,644 women) Indicator 4 Number of advocacy initiatives 8 actions Overall outcome from grant CCD supported CPN and conducted many campaigns. For example, filing the coordinated complaints to preserve the natural resources. The government ordered the company (Pheapimex) to return 1,500 hectares to 150 families in Toek Phos district. 60 Name of Organization Community Legal Education Center (CLEC) Grant per period January 2009-September 2010 2011 2012 2013 US$290,000.00 Terminated Composition of Organization Mr. Yeng Virak, Executive Dirctor Mr. Khov Vannith, Finance Manager Mr. Huon Chundy, Program Manager Activities Target Area 1. To provide legal and skills training to community representatives and grassroots advocates of target communities in land rights and informal dispute resolution  Phnom Penh, Koh Kong, Rattanakiri, Kratie 2. To provide in-depth legal consultations on land and livelihood rights to 3 new communities 3. To provide continued legal representation to 7 old communities 4. To provide legal representation to community representatives and grassroots advocates facing criminal charges … M&E Indicators Number Indicator 1 Number of participants trained: Total (women) 400 (191 women) Indicator 2 Number of cases/clients represented: Total (women) 10 cases/49 people Indicator 3 Number of citizens advised: Total (women) 3,199 (102 women) Overall outcome from grant 61 Name of Organization Dai Ku Aphiwat (DKA) Grant per period January 2009-September 2010 2011 2012 2013 US$86,495 Terminated Composition of Organization Ms. Neang Peng Hor, Executive Director Ms. Pen Sothy, Finance Officer Mr. Chhon Sokha, Program Coordinator Activities Target Area 1. Organize and build capacity of women’s leaders at village and commune level  5 villages in Kampot provinces  3 villages in Sihanouk province  8 villages in Kampong Speu province  8 villages in Siemreap province  8 villages in Oddar Meanchey 2. Organize youths and facilitate meetings of youths at commune level 3. Assist communities to better organize, to conduct dialogs, and to support community development and advocacy initiatives 4. Provide training on natural resources and support community advocacies on natural resource protection at national level … M&E Indicators Number Indicator 1: Number of communities organized 33 groups had been established (14 women groups, 10 youth groups and 9 saving groups) Indicator 2: Number of participants trained: Total (women) 612(267 women) Indicator 3 Number of participants in the constituency building: Total (women) 3,378 (1,485 women) Indicator 4 Number of advocacy initiatives 7 actions Overall outcome from grant 62 Name of Organization Fisheries Action Coalition Team (FACT) Grant per period 2009 2010 2011 2012 2013 None US$40,871 Terminated Composition of Organization Mr. Om Savath, Executive Director Activities Target Area 1. Capacity building to key fishers Three provinces: - Preah Sihanouk - Koh Kong - Kampot 2. Conduct 18 commune dialogues in 12 commune target areas 3. Organize provincial or inter-provincial dialogues 4. Organize advocacy campaigns … M&E Indicators Number Indicator 1: Number of communities organized 11 fishery communities were established Indicator 2: Number of participants trained: Total (women) 111(33 women) Indicator 3 Number of participants in the constituency building: Total (women) 2,096 (1,116 women) Indicator 5 Number of advocacy initiatives 28 actions Overall outcome from grant 63 Name of Organization Cambodian Human Rights and Development Association (ADHOC) Grant per period 2009 2010 2011 2012 2013 2014 $52,598 $74,474 $88,661 $61,398 $59,229 $7,079 Composition of Organization Mr. Thun Saray, President; Mr. In Kea, Secretary General; Ms. Yi Phina, Finance Manager; Mr. Ny Chariya, Head of Investigation; Mr. Latt Ky, Land Program Officer Activities Target Area 1. Conduct investigation and intervention in new cases of land grabbing and land evictions  10 provinces (1st phase 2009-2010): 5. Kampong Thom, Siem Reap, Kratie, Battambang, Ratanakiri, Mondulkiri, Preah Vihear, Banteay Mean Chey, Stung Treng and Kampong Speu 6.  14 provinces (2nd phase 2011-2013): 7. Mondulkiri, Ratanakiri, Kratie, Stung Treng, Kompong Speu, Preah Vihear, Kompong Cham, Kompong Thom, Kampot, Battambang, Siem Reap, Sihanoukville, Kompong Chhnang, and Pursat. 8. 2. Provide victims with legal aid and emergency assistance 3. Convene press conference to exert pressure on government institutions involved for action 4. Conduct field trips to ensure the project implementation progress, and while staying in provinces, ADHOC monitors from Phnom Penh helped field staff to implement the project correctly 5. Organize monthly meeting of volunteer human rights activists or ADHOC local networks 6. Organize monthly meeting of ADHOC provincial monitors for sharing information and experiences 7. Empower the voice of affected communities Additional activities in Phase 2. 8. Conduct advocacy through provincial workshops and dialogue and cooperation with the government M&E Indicators Number Indicator 1: Number of land cases investigated 484 cases Indicator 2: Number of clients implicated in land disputes represented 468 (60 women) Indicator 3: Number of participants trained 3,517 (1,683 women) Indicator 3: Number of participants in constituency building 15,683 (4,826 women) Indicator 4: Number of advocacy initiatives 1,874 actions Overall outcome from grant 2,542 indigenous people (675 women) were empowered by ADHOC through meeting and training them. Around 21,883 hectares have been returned or allowed to continue farming to 5,511 families for FY2009-2010 64 Name of Organization League for the Defense and Promotion of Human Rights (LICADHO) Grant per period 2009 2010 2011 2012 2013 2014 $227,213 $301,961 $285,895 $224,959 $173,750 $22,500 Composition of Organization Ms. Pung Chiv Kek, President Ms. Naly Pilorge, Executive Director Mr. Kong Chanroatha, Deputy Director of Finance Ms. Chanhorm, Program Officer Activities Target Area 1. Investigate all complaints of human rights violations 13 provinces: Battambang, Banteay Meanchey, Siemreap, Pursat, Kampong Thom, Kampong Chhnang, Kampong Cham, Koh Kong, Kampong Speu, Phnom Penh, Svay Rieng, Kampot, Preah Sihanouk 2. Provide assistance and means for seeking remedy to victims and follow up with victims regularly 3. Provide legal representation and advice to victims of human rights violations 4. Monitor prison conditions and prisoners’ rights 5. Maintain database and documentation of cases investigated 6. Conduct advocacy to promote and protect human rights M&E Indicators Number Indicator 1: Number of cases of human rights violations investigated 1,938 Indicator 2: Number of cases/clients represented 386 clients (66 women) Indicator 3: Number of prisoners interviewed 9,878 inmates Indicator 5: Number of advocacy initiatives 24,959 Overall outcome from grant Not yet compiled 65 Name of Organization Cambodia Human Rights Action Committee (CHRAC) Grant per period Jan 2009-Sept 2010 2011 2012 2013 2014 US$61,299 $59,537 $40,611 $38,023 $4,001 Composition of Organization Mr. Thun Saray, Chairman; Mr. Suon Bunsak, Executive Secretary; Mr. Nhean Sam Ath, Finance Officer; Mr. Ieng Pich, Chief Investigator Activities Target Area 9. Phase 1: 2009-2010 1. Identify serious violations of basic human rights 10. Whole Cambodia 2. Investigate cases of serious human rights violations and provide support to the victims of violations 3. Conduct persistent advocacy to authorities responsible for the person who violates rights and those responsible for prosecution of violators 4. Take actions in respect of particular serious violations of human rights Phase 2: 2011-2013 1. Identify, investigate, document serious human rights violation cases and hold dialogs with relevant authorities to seek interventions in rights violations Whole Cambodia 2. Follow up the progress of the investigated and intervened cases so as to verify their progresses and to facilitate resolution 3. Observe court trials on cases intervened by CHRAC 4. Collect data and document media reports on governmental and judicial actions in support of fact￾based dialogues and advocacies 5. Provide victims of human rights violations with legal advice and other appropriate supports for their advocacy actions 6. Organize appropriate advocacy campaigns on high profile human rights cases and other relevant issues in order to call for the improvement of the general human rights situation 7. Organize annual advocacy workshops on general human rights and legal and judicial issues 8. Participate in regional and international human rights advocacy actions and cooperate with relevant actors 9. Support strengthening of Civil Society Organizations and community networks. M&E Indicators Number Indicator 1 Number of cases investigated 124 cases 66 Indicator 2 Number of citizens participating in constituency building 4,449 (1,184 women) Indicator 3 Number of advocacy initiatives 573 actions Overall outcome from grant Universal periodic review reports and reports on torture were submitted to UN UPR and CAT committees 67 Name of Organization Cambodian Center for Human Rights (CCHR) Grant per period January 2009-September 2010 2011 2012 2013 2014 US$199,256 $98,516 $70,112 $59,056 $6,990 Composition of Organization Mr. Ou Virak, Ecutive Director Mr. Khan Kalina, Finance Manager Mr. Lach Vannak, Program Officer Activities Target Area Phase 1: 2009-2010 1. A user-friendly database is created which enables documentation of human rights violations and provides Cambodians with a localized medium of complaining about human rights violations (the “Human Rights Database”)  Whole Cambodia 2. The Human Rights Database is used by NGOs working on human rights issues throughout Cambodia (the “Database Participants”), with information on human rights violations collected (the “Violation Data 3. The Database Participants are able to refer cases to each other, matching violations to NGOs’ specialisms 4. The Database Participants investigate the human rights violation(s) most relevant to their specialisms 5. Raise awareness of these documented human rights violations 6. The Database Participants analyse the Data, to determine the overall Human Rights ‘picture’ (i.e. trends, patterns etc.) 7. The Database Participants make the Violation Data available to the Cambodian public and any other interested parties (e.g. donor governments and organizations, the media, regional and international governmental organizations (i.e. Association of South East Asian Nations (ASEAN) and the United Nations (UN)), and regional and international Human Rights NGOs (e.g. the Asian Commission on Human Rights (ACHR) and Human Rights Watch) 8. Interested Database Participants use the Violation Data (and analysis) as a basis for sophisticated advocacy and campaigning, supported by facts. Phase 2: 2011-2013 1. Further develop and maintain sithi.org web portal to enable open documentation of human rights violations and dissemination of other varied information on human rights situation and providing a tool for online advocacy Whole Cambodia 2. Further develop and perfect project 68 infrastructure and tools, such as directory of organizations participating in the project, human rights hotline and CSO help desk, human rights library, CSO support fund, introductory resources, and training curriculum. 3. Encourage at least 40 prospective CSOs (“Project Participants”) to participate in the project and provide training to at least 20 new partners (on the use of manual of human rights portal and project manual, monitoring and documentation, research and analysis, dialog and advocacy, and organizational development) and engage in collaborative monitoring, documentation and information sharing on human rights and joint actions for change. 4. Provide hands on field assistance to at least 12 project participants in collaborative monitoring and documentation, information sharing, research and analysis of human rights situation, and advocacy initiatives with the government, public, and international community. 5. Undertake high-level monitoring and documentation of violations of fundamental freedoms of expression, association, information and movement 6. Coordinate research and analysis of fundamental freedoms to develop at least 90 recommendations for policy, legislative and structural changes and to improve respect for human rights. 7. Assist project participants to strengthen organizational development such as governance and management, staff capacity building, relationship building, strategy, and sustainability and fundraising. M&E Indicators Number Indicator 1 Number of visits 925,107 visitors Indicator 2 Number of cases referred 1,328 cases Indicator 3 Number of participants trained 580 (160 women) Indicator 4 Number of citizens participating in constituency building 167 (96 women) Indicator Number of advocacy initiatives 3,207 actions Overall outcome from grant Web portal sithi.org receives awards (isif award 2011 and University of Queensland 2013) for promotion and protection of human rights and has become a source of information for media and human rights organizations    69 Name of Organization Children Development Association (CDA) Grant per period 2009 2010 2011 2012 2013 2014 None None $49,267 $28,994 24,432 2,858 Composition of Organization Mr. Chee Boreth, Executive Director; Mr. Brack Sopheap, Finance Officer; Mr. Rith Bo, Program Manager; Mr. Ly Rachana, Project Officer Activities Target Area 1. Provide trainings on active non-violence, illegal logging, community leadership, rights based approach, sub-decree on economic land concessions, legal framework related to natural resources and advocacy skills, community organizing, community research, information management and social analysis, community media skills, commune investment planning (CIP), and commune development plan (CDP) to Community Forest Management Committee (CFMC) and networks 11. Oddar Meanchey province 2. Disseminate the Forestry Law and Land Law, Sub￾decrees, CDPs, CIPs in relation to the impact of economic land and mining concessions 3. Provide funds to CFMC and networks to implement advocacy activities in relation to natural resources and human rights violations (illegal logging and land and mining issues) 4. Facilitate meetings at commune, district and provincial levels on natural resource issues and human rights at national network level 5. Legal services: link with other NGOs to seek legal services for communities, network and youth who are charged with criminal offenses as a result of their advocacy to protect their land rights and natural resource M&E Indicators Number Indicator 1 Number of participants trained 2,013 (902 women) Indicator 2 Number of citizens participating in constituency building 15,968 (7,215 women) Indicator 3 Number of advocacy initiatives 130 actions Overall outcome from grant 70 Name of Organization Indigenous Community Support Organization (ICSO) Grant per period January 2009- September 2010 2011 2012 2013 2014 US$205,000 $105,921 $63,079 $41,000 $7,000 Composition of Organization Mr. Sao Vansey, Executive Director Ms. Chang Mouy, Finance Coordinator Mr. Chin Thavro, Program Coordinator Mr. Svay Sochea, Program Officer Activities Target Area 1. Establish indigenous peoples working group (IPWG) through community election 12. 15 provinces: 13. 14. Mondolkiri, Rattanakiri, Stung Treng, Kratie, Kampong Cham, Kampong Thom, Preah Vihear, Siemreap, Oddar Meanchey, Banteay Meanchey, Battambang, Pursat, Kampong Speu, Preah Sihanouk, and Koh Kong 2. Provide practical skills and experience required for community awareness raising, community mobilizing, developing networks, and advocating at national and international level for indigenous rights 3. IPWG to provide support and training on community networking, indigenous rights, human rights, and advocacy skills to indigenous grassroots community 4. Provide support to IPWG and community representatives to conduct advocacy initiatives on indigenous issues in cooperation with non-indigenous and international networks … M&E Indicators Number Indicator 1 Number of participants trained 2,938 (958 women) Indicator 2 Number of citizens participating in constituency building 11,415 (10,420 women) Indicator 3 Number of advocacy initiatives 254 actions Overall outcome from grant 71 Name of Organization Community Peace-Building Network (Cord/CDPS/CPN) Grant per period January 2009-September 2010 2011 2012 2013 2014 Included in CCD support $142,066 $107,822 $67,769 $7,000 Composition of Organization (CPN is a community network and receives funds from host organizations successively from Cord and CDPS) Mr. Keng Bunchhoeut, Country Director, Cord Mr. Tep Sokhoeun, Financial Manager, Cord Mr. Heng Sokheng, Executive Director, CDPS Ms. Bin Thyda, Finance Officer, CDPS Mr. Ek Yothin, Advisor Activities Target Area 1. To facilitate the selection and engagement of NGOs and other NGO networks that CPN will build coordinating relationships with 15. Whole country 2. To involve CPN in co-facilitating regular meetings between CPN and its stakeholders to review and coordinate their support. This process will be gradually handed over and continue independently through CPN 3. To facilitate capacity development trainings on area of need, including but not limited to governance and leadership, networking, non-violent communication, cooperation with local authorities, training on relevant policies and laws 4. To facilitate reflection and learning activities working with the CPN in an action-learning approach, helping them to make their own decisions, implement and then reflect on the outcomes and progress toward an independent, well￾governed, community-accountable people’s network 5. To provide capacity development support to CPN on governance and leadership within the framework identified by CPN at their national congress 6. To work with CPN support programme to develop a leadership framework capacity of CPN and work with them and their member groups to establish a leadership framework allowing for an independent community￾accountable people’s network and constituency to be developed 7. To work with CPN to engage and learn from existing constituency based regional networks and emerging area networks such as Prey Lang, Aural 8.To develop a planning framework that the CPN will be able to utilize as a tool to plan advocacy activities with room for creativity 9.To co-facilitate regular reflection and learning sessions for CPN through the national congress, leadership group meetings and through general CPN meetings 10.To work with CPN to develop monitoring and evaluation protocols to ensure that work towards achieving CPN objectives 72 11. To provide capacity development support to CPN through trainings/workshops followed by individual or group mentoring and coaching/advising based on the key areas identified during the strategic review and recent assembly in response to the needs of CPN project 12. To additionally provide oversight and capacity development support in the following areas:  Facilitation and coordination skills;  Administrative and financial management and a comprehensive system development including policies for ensuring accountability and transparency;  Fund raising, proposal development, report writing and donor liaison  Capacity development around long-term strategic direction and planning  Capacity development in conflict resolution, governance and leadership, community organizing and mobilization, mentoring and coaching skills  Capacity retention plan development for the support project  Bank account setting for channelling funding with appropriate decision making right to the support project and CPN 13. CPN supports national advocacy 14. CPN supports capacity building for active nonviolence and leadership and people’s rights 15. CPN strengthens regional and national level networks 16. CPN supports community media related to advocacy 17. CPN monitors target communities and continues to prepare strategy planning M&E Indicators Number Indicator 1 Number of participants trained 2,719 (1,478 women) Indicator 2 Number of citizens participating in constituency building 16,303 (6,971 women) Indicator 3 Number of advocacy initiatives 333 actions Overall outcome from grant Communities under CPN were able to conduct several campaigns against land grabbing, illegal economic land concessions, and destruction of natural resources. 73 ANNEX VI: SUMMARY OF GRANTS UNDER PRAJ II Grants Made by EWMI Grantee/Type Purpose No. of Years Total Amount Comment CCHR/ HR NGO Trial Monitoring; database development; periodic thematic reports 5 $162,357 See TAF ($433,930)35F35F 36 LAC/ PILF Provide legal aid services for in all cases, including land 6 $918,813 IBJ/ PILF Provide legal aid services in criminal cases 4 $298,860 LSCW/ PILF Provide legal aid services to victims of gender-based violence 5 $518,646 SLG/ PILF Provide legal aid services in all cases 4 $340,086 VLG/ PILF Provide legal aid services in all cases 2+ $80,468+ CDP/ Legal Aid Provide legal aid services in all cases 2 $167,325 BAKC/ Bar Assoc. Provide legal aid services in all cases; lawyer support 2 $95,176 RISC/HR NGO Provide support services for returning deportees from US 4 $127,761 WMC/Media Support production of HR films 3 $143,148 IDEA/GO Support development of HR for various income-generating constituency groups 4 $102,215 CHRAC/ HR NGO HR and advocacy 3 $23,350 See TAF ($203,471)36F36F 37 BCV Community Media Forums 2 ? NGO Forum National Consultation Workshops 2 ? CVS Youth Forum on Dam 1 ? See TAF CCC Public Comments on LANGO 1 ? TOTAL $2,978,325+ 36 CCHR – TAF’s grant supporting Sithi.org is the primary grant to CCHR and was in place first. When it was determined that CCHR would also be funded to conduct court monitoring, it appeared that, because of the relationship between court monitoring and EWMI’s efforts to reform judicial administration and provide training to our legal aid lawyers working groups (and related technical expertise that we had in-house), this grant should be managed by EWMI. 37 CHRAC – the only long term grant for ongoing activities is that provided through TAF. EWMI’s grants have been small grants for single events (IHRD) or short-term events (UPR support) that were funded under unsolicited applications specific to those activities. (CHRAC’s were detailed in EWMI’s grant summary because of the significance of the UPR outcomes). 74 Grants Made by TAF Grantee/Ty pe Purpose No. of Years Total Amount Comment AEC/GO Advocacy and constituency building 2 $95,509 GRC did not recommend its further funding. The project supported activity only as to Pheapimex. Due to fund limitations it was determined to cease funding. (Grant was awarded twice for interval of two years, but the second grant ended after one year. ) CCD/GO Community organizing 1 $330,415 See comment below under Cord/CDPS/CPN CLEC/GO Legal and skills training 1 $290,000 DKA/GO Women and youth leadership development 1 $86,495 FACT/GO Constituency building 1 $40,871 After one-year grant ended, GRC did not recommend for the next phase funding. FACT appeared to have substantial conflicts within its management. ADHOC/ HR NGO Trial investigation in land cases 6 $343,439 LICADHO/ HR NGO Investigation of all activities related to government infringement on people’s rights 6 $1,236,278 CHRAC/ HR NGO Investigation of all HR violations 5 $203,471 See EWMI ($23,350) CCHR/ HR NGO Creation of database; use of DB in generating thematic reports 5 $433,930 See EWMI ($162,357) CDA/ HR NGO HR and land violation awareness raising 4 $105,551 ICSO Establish and equip Indigenous People’s Working Group 5 $422,000 IRAM used funds (provided through grant to ICSO) to support training of and meeting with local indigenous leaders, and documentation of indigenous rights violations. Information about the specific amounts ICSO spent through IRAM is not immediately available. If that information is required, we will need more time. Cord/CDPS/ CPN GO Constituency building 4+ CCD $324,657+ CPN used funds to support their members in: 1) travel, food , and accommodation for hotspot visits and trainings; 2) travel for thumbprint collection and community organizing and advocacy campaigns; 3) payment of meeting 75 Grantee/Ty pe Purpose No. of Years Total Amount Comment and training expenses ; 4) payment to Prey Lang Community Network for costs of its meetings, training, and advocacy campaigns. From 2008-2010, CPN grant went through CCD. Amount allocated for CPN activities was $149,502 (excluding CCD grant). From 2011-2012, CPN grant went through Cord. Amount allocated for CPN activities was $189,001 From 2013-2014, CPN grant went through CDPS. Amount allocated for CPN activities was $67,769 TOTAL $3,912,756 + APPROXIMATE TOTAL FOR GRANTS OVER 2009-2013 PROJECT PERIOD: $7,000,000 76 ANNEX VII: DATA COLLECTION INSTRUMENTS 1. QUESTIONS FOR USAID USAID/Cambodia – D&G Team and Program Team 1. What was the context out of which PRAJ I and II emerged? What was the importance of this project to USAID’s D&G portfolio? 2. How did USAID view the theory of change upon with PRAJ I and II were developed? Was this as useful during PRAJ II as it was in PRAJ I? How did this change as a result of the PRAJ II mid-term evaluation? How did this affect the way EWMI managed the project? 3. Please explain why there were so many changes to the PMP; why Objective 3 was dropped; why Objective 2 was not important to the current evaluation? 4. Please describe the relationship you have with EWMI and its partners? On a scale of 1-4, with 1 being very satisfied and 4 being very dissatisfied, please rank your relationship. Why this number? Please explain. How could these relationships be improved? 5. How did USAID provide oversight to the project? How useful (and USED) were the QRs or other documentation submitted to the AOR or the DG team by project staff? How did you discern what was the most salient information for managing this project? What has been your means of transmitting information from one AOR/office member to the next? How did 70+ pages in each QR serve your purpose? What are your suggestions for improving this reporting? 6. Please explain the absence of annual reports and summary documentation produced by EWMI? Please also explain the absence of project activity outcomes on the effectiveness of NGOs/CBOs/CSOs that received grant funding. What issues arose in the management/oversight of the project? How were these resolved? 7. Over the course of PRAJ I and II, how many changes have there been in D&G leadership? Program Office leadership? What is the hand-over process when new personnel come in? Tell us about the shift in responsibilities for monitoring and evaluation from the technical to the program team. What challenges have you faced in making this transition? Thusfar, how have they been resolved? 8. How would you characterize the evaluation skill level of your staff? What are their strengths? Weaknesses? What do you think is needed to improve skills? Beyond this learning evaluation, in what other evaluation capacity building activities have you/your staff participated ? In what other activities do you plan on participating in the future? 9. The last few QRs indicated an increase in violence resulting in the death of several leaders. What was USAID’s official response to these killings? How did you work with EWMI on addressing this violence? What policy covers these types of eventualities? 10. What has been USAID’s official response to government failure to deliver on its MOU? Please provide us with a copy of the MOU. 11. Was there a close-out report written for PRAJ I? If so, please provide us with a copy (we have the final evaluation). Did PRAJ II implement the recommendations for improvement noted in the final evaluation? Why/why not? How did you see PRAJ II changing focus from that of PRAJ I? What was different? 12. What is your assessment of the efficiency and effectiveness of EWMI as an implementing organization? What are their strengths? Weaknesses? What are your suggestions for their improvement? 77 13. Please explain how the five original objectives of the project were coordinated. What did EWMI do? What did you do? How did this coordination change when Objective 3 was dropped? Did this affect the outcomes of the project? 14. What was your rationale for extending PRAJ II another year? How do you see the project changing over this final year? How do you think this will affect overall outcomes and reaching the project goal? 15. Given the status of government seeming disinterest, what types of D&G objectives would be appropriate to pursue in the future? How do you envision the participation of HR NGOs, CSOs/CBOs/Grassroots Organizations, if at all? 16. What types of recommendations emerging from this evaluation will be most helpful to you in determining what the next D&G project might be? Other SOW-related questions for USAID and EWMI: 1. Why are there no documents listed as to what the team would review in preparation for the evaluation? 2. Obj. 1 – What type of training manuals were developed and what type of training was delivered to HR NGOs on the construction and maintenance of databases? 3. Obj.4 – 4.1 addresses land and livelihood rights and advocacy training to rural communities, but IDEA is urban? Did this objective shift? 4.2 and 4.3 it seems there is an overlap in this sub-objective – and reports suggest that CPBN might not be the grantee. Why were grants only for a year? What happened to the funding to the HR Forum? What was the national level reform agenda? 4.4 encourage dialogue with government – was training provided on conflict prevention to grantees? To whom did FIT provide financial training/capacity building? What manuals were used to train organizations and networks on management and advocacy? Why are the areas covered by TAF presented in a summary document received and not transmitted by EWMI. 4. Obj.5 – Only poor children is the target of this objective; when did it change to represent HR defenders and those charged with criminal offenses? 5.1 Why was a new public interest law firm supported when LAC was already there? How was Samreth chosen as the “new” PILF? Why did funding stop? 5. Question 2 – Clarify “original” project design vs. all the changes. 6. Question 3 – EWMI was to have conducted an organizational assessment on the strength of grantees. While one was conducted, it was on the basis of IT, not on the ability of each organization to do its work. There was an attachment A referred to in the 2011 civil society update, but it was not included. Does this document include an organizational assessment? Other PMP-related Questions (based on 2012 new PMP) for USAID and EWMI: 1. Why have the indicators in the 2012 PMP changed? How did this affect how project performance was actually measured? 2. Obj. 1 – Each partner organization that established a database had objectives to satisfy to obtain funding. What were these? How do we know that a certain percentage of these were achieved? An appraisal of the number of users of each database was to have been developed. Where is this? How is “effective” use to be measured? “Criminal cases” are followed for the criminal courts database, but there is no indicator for the HR violation cases? How are these monitored? What is the indicator for them? 78 3. On results to be achieved for Obj. 1, how is the independent judicial performance monitoring system related to the achievement of the project goal? What is the Quality Control and Advisory Commission? What are the details? 4. Obj. 4 – how is signing a petition an “initiative”? How did the minutia of activities become initiatives to be documented extensively in the quarterly reports? What defines “engagement” of citizens in CSOs? How is this information tracked? What numbers have emerged? How did ODC become an important part of this objective? The networks were to have been improved, but networks are often loosely organized citizens around an issue; there is no “formal” organization, so how is data collected on objectives values, funding, etc. collected? How is “improvement” measured? How is feedback reported to the constituencies of these informal networks? 5. On results to be achieved for Obj. 4, increased number of land activists and local networks functioning, only grantee data is collected? How many proposals are received, accepted and rejected each year? On national NGOs more effective at constituency building, what training was provided to achieve this? How different was the training provided by TAF and that of EWMI? On increased number of government-constituency group discussions, does this mean between networks, or formal organizations? What were the outcomes of these discussions? 6. Obj. 5 – Here, the reference is to number of indigent clients, but in SOW it is youth? Which is it? Where is the report on survey respondents that know where to go for legal aid? 7. On results to be achieved for Obj. 5, why was funding to BAKC stopped? Why should there only be a decrease in pre-trial detention for juveniles? Why not all clients represented or changes in system? 79 2. QUESTIONS FOR EWMI 2a. Mark West of EWMI was interviewed in the US on, Tuesday 11/19/2013, as he left Cambodia 1-1/2 years ago. 1. What is your specific role in the project? 2. Can you tell us more about the theory of change on which the information sharing activities were built? 3. Which databases did the project work with? 4. What is the purpose of each database? Do the organizations have written statements of purpose for each database? 5. What was target audience of each database? How was each one used? 6. Who had oversight of each database and who within the project had follow-up responsibilities? 7. What were your expectations for the use of each database? Why were those expectations fulfilled or not? 8. Why do you think there was lack of collaboration among NGOs in sharing data? 9. What did the project do to address this lack of collaboration? (in addition to the NGO information￾sharing workshops that is noted in the quarterly reports). A notable outcome of one such workshop was a joint publication. What did that publication lead to? And, were there any other outcomes of the NGO information sharing workshops? 10. PPMC’s database used to collect information from two courts, but the second was dropped. Why was that? 11. What was the thinking behind establishing multiple databases among EWMI’s legal aid partners? (the 1-year extension proposal recognizes that a shared database would allow for better identification of systemic issues) 12. Why do you think the MOJ dropped use of GBV and TIP databases in spite of international accolades? 13. What are your lessons learned from the M&E system and the work around databases? 14. Do you think the investment in databases was worthwhile? 15. What do you see as overall impact of this work? What would you recommend in future vis a vis the theory of change, in particular the role of information sharing and database management? 16. Which database users in Cambodia would you recommend we interview? 2b. Questions for EWMI staff collectively and/or individually On Management 1. Please tell us briefly about the original theory of change under which PRAJ II was launched and how and why it changed over the period of implementation. 2. What areas of PRAJ II were most challenging to implement? Why? 3. How is EWMI/Cambodia organized to conduct its work? From the reporting and activities, it appears that each objective is a separate program within the project. Could you explain how this organizational strategy emerged? How did you identify ABA and TAF to be your partners? Under what arrangement were these partnerships entered? Why did the partnership with ABA end? 4. In the EWMI grants manual, you state (p. 18) that you will conduct an evaluation of the overall effectiveness of the program, review financial reports for cost effectiveness, and evaluate organizational capacity-building efforts. And that you might perform a second risk assessment 80 similar to the one conducted by Benetech originally. Have you performed all this work? What were your findings (is there a report)? On Objective 1 5. How has data collected on HR abuses been used to improve the justice system in Cambodia? What have been the overall results of this objective, e.g., number and types of databases constructed, MOJ and NGO use of the databases, outcomes of the use of the databases? What has changed since the release and localization of Firefox in Khmer? Why was the use of the TIP and GBV databases discontinued? What are the ramifications of these databases not being managed any more? What is your impression of the ability of the HR NGOs and the MOJ to continue with the work after the project ends? Which NGOs do you believe can carry on without project assistance? What would be the source of their funds? How have the reports been used to advocate for addressing human rights violations in Cambodia? What has changed in the human rights situation since the release of those reports? What have been the outcomes of the cases against legal aid defenders? How have the courts influenced the policies and practices of government in terms of supporting human rights (name all areas, e.g., land, criminal accusations, etc.). What strides in human rights would you say have been made as a result of the implementation of this objective? What do you see as “unfinished business” in terms of EWMI fulfilling this objective? What would you do differently if you were to rewrite this or a similar objective on improving human rights? What is your exit plan for funding these organizations to continue their work? On Objective 4 6. How to you make known throughout the country that grants are available to support the work of judicial reform? How was TAF exercised oversight over grants? What kinds of training are provided to CBOs/CSOs and grassroots organizations? What type of training did FIT provide and to whom? What has been the outcome of all this training? Among these grantees, which three would you say are the best performing (i.e., meeting their goals and being effective among their constituencies)? How would you characterize the leaders of more local organizations? How do they identify a particular concern to pursue under the grants they receive? How do organizations build their constituencies? How will these organizations obtain funding once the project is over? What does EWMI/TAF do when violence breaks out after an organization does what it proposed to do? How are these outbreaks of violence addressed by the government? What in the project guides you in addressing violence? What strides has EWMI made in creating judicial reform in terms of group advocacy efforts against all sorts of violations? If you were to rewrite the activities of this objective, what would you drop? Include? What is your exit plan for the end of funding for these organizations? 7. Prey Lang and other land issues, how do you see the process of defense of land rights moving? How do local constituencies organize themselves to protest the land grabbing and government’s granting of concessions? What has been the reaction of government? How are these violations being addressed through the project? What types of investigations are EWMI facilitating to understand the basis of land disputes? To hold dialogues with government over land concessions to rubber and timber companies? How will local people benefit, if at all, from these concessions? In terms of protecting the land and other rights of indigenous people, what 81 government policy protects them? How is this implemented and enforced? What else do you believe a project like EWMI can do to further enhance land and livelihood protection? On Objective 5 8. On Objective 5, what have been the reactions of the government and court officials to the fair trial monitoring and other reports? What has been changed in court proceedings in response to those findings? How much have the legal aid services helped protect the rights of the poor? What progress has been made in establishing a legal aid “culture” in Cambodia so that the poor know there is a place to go for defense? What have been the main challenges in establishing legal aid organizations and law firms? EWMI requested to take a lead role in working with the government on making legal aid a priority, what led to that decision? What were the results of EWMI taking the lead role? What type of informational campaigns has EWMI conducted to increase the number of legal aid providers and to let detainees know that they can be defended without charge? What have been the main challenges presented to legal aid organizations in defending their clients, either from the courts or elsewhere? How are legal aid programs becoming sustainable? What exit plan does EWMI have with legal aid programs for when EWMI is finished? Please tell me about the split between Samreth and Vishnu. Why did this occur? Did lawyers working for Samreth join Vishnu? Why? How has it altered any of the original objectives of the project? What is the difference in the work Samreth does and what Vishnu does? How have clients viewed the split? What role did EWMI play in the decision to split? Recommendations for Future 9. If you had this program to do all over again, what would you do differently – beyond what has already been asked about each objective? 3. QUESTIONS FOR TAF 1. Please tell us how you became involved with PRAJ II. 2. Please tell us briefly about your work with Objective 4 of PRAJ II – how do you determine who your grant recipients should be? What is the financial range of grants you have provided? How do your organize yourselves to do your work/what is your management structure? Could you please provide a list of grantees, the amount they received, the duration of the grant, and the focus of their activities? 3. What training/capacity building have you provided to grantees? How did you identify the training or capacity building need of the grantees? Is it different for CBOs/CSOs/Grassroots networks? If so, how? What other training do grantees still need? What has been the outcome of this training, i.e., what results have the grantees achieved in terms of organizational and financial operations? 4. How would you describe the overall capability of grantees to fulfill their proposal objectives? Why? What challenges have they faced in meeting the objectives? What other support has TAF provided in helping them move along in meeting their objectives? How do they financially manage their grants? How are the organizations/groups structured to achieve their objectives and financially manage their work? How do you oversee the grants used by grantees? What have been the challenges in grant management by those grantees? What have you done to address those challenges? On a scale of 1-4 with one being very satisfied, and 4 being very dissatisfied, how satisfied would you say grantees are with your management of the grants? What do you think they would change? 82 5. What kind of guidance does TAF provide to organizations seeking a grant? How does TAF organize itself to review grant proposals? What criteria are applied to awarding a grant? How are these criteria linked to the overall goal of PRAJ II? On a scale of 1-4, with one being very successful and 4 not very successful, how would you rank – overall – the performance of grantees in terms of being able to effect change? Why? 6. What have been the significant achievements of grantees in terms of government judicial reform? Please explain. What have been the main stumbling blocks to achieving these reforms? 7. It appears from reports that grassroots advocacy networks largely address problems in land and livelihood. How have these networks, and their constituency groups, addressed these issues; what is the range of strategies they have employed? How successful have these advocacy networks been in protecting land, livelihood and resource rights? What issues have arisen owing to government activities? How have these been addressed? 8. In providing capacity building in how do conduct advocacy campaigns, there has been a steady increase in violence. How does TAF address the rising incidence of violence? How do grantees address the rising incidence of violence and the killing of their leaders? What recourse does TAF/a grantee have when violence emerges and people are severely injured or killed? In light of this outcome, how would you address advocacy training differently, if at all? 9. In providing capacity building, how does TAF address substantive knowledge needs of how things work in Cambodia (beyond the face-to-face interactions)? How are grantees equipped to negotiate and resolve conflict with government agencies? What have been the results of these efforts? 10. How would you characterize your relationship with the different types of grantees? Are some more problematic than others? Why? What do you do when problems in relationships arise? How do these issues affect the way you provide grants? 11. How and how often do you monitor the activities of grantees? Could you please provide us with your M&E plan template with indicators monitored? How do you know when to trouble-shoot any challenges? Or visit when the grantee seems to be going off course? 12. Please characterize your relationship with EWMI in terms of the work of PRAJ II? How responsive is the COP to any suggestions made or issues raised in administering the grant program? What misunderstandings have arisen? How were they resolved? 13. If you had to do this program all over again, what would you do differently? 14. What would be your recommendations on how USAID could best support civil society and human rights? 4. QUESTIONS FOR CCHR (HR NGO with Database) 1. Please tell us about the mission, core business and activities of your organization. 2. How did you become involved in PRAJ II? How did you learn of the grant opportunity? What was the focus activity of the grant? Were you provided a sufficient amount and time to complete the projected activities of the grant? How would you recommend that the grant process could be improved? On a scale of 1-4, with 1 being very satisfied and 4 being very dissatisfied, how would you rank your relationship with EWMI? What can you suggest to improve this relationship? 3. What were the specific activities you pursued under PRAJ II? How did these activities fit into your organization’s strategic plan? How did you organize yourselves to perform these activities? What was your management and work structure before the grant and after the grant? How many new 83 employees were hired after you received the grant? In what categories were they? How did the new organizational structure and employees affect your budgeting and sources of funding? 4. In terms of your database, how was it constructed? How did it change under the PRAJ grant? From where do you collect information to be entered? What information do you collect to be entered? What challenges did you face in collecting data? How did you overcome these challenges? Over the course of the project, what changes did you have to make in the creation and maintenance of your database? Why? How did the translation of different aspects of the database into Khmer affect your work? What were your significant achievements? 5. How has the creation of databases, publishing reports, monitoring HR violations, etc., contributed to change in HR in Cambodia? What are some of the strategies you employed to link information to institutional reform? What have been the main challenges in linking information to institutional reform? 6. In general, who do you see as the primary audiences for your work? Is there a different audience for your work under PRAJ II? If so, how does it differ? What has been the outcome of this work with these audiences? 7. How do you interact with other HR organizations (international, other national, provincial, informal)? In what ways do other organizations or individuals contribute to the work of your organization? How do you share information? How often? How do you formulate advocacy campaigns or policy positions? 8. How would you describe the awareness level of the Cambodian public of issues affecting the enjoyment of their political, economic and social rights? How would you characterize their involvement in advocating for change? What do you suggest as a good strategy to increase their awareness and to motivate them to act? 9. What do you see as your organization’s greatest achievement under PRAJ II? What has been the biggest contributor to that achievement? What was the impact of this achievement on HR overall? What have been the main hindrances to the work of your organization? How did you overcome these? 10. What strategies do you think are most effective for creating constituencies for reform? For actually achieving reform? What suggestions for change would you have for the next generation of such a program? 5. QUESTIONS FOR LICADHO and ADHOC (HR NGOs with Databases and Legal Representation) 1. Please tell us about the mission, core business and activities of your organization. 2. How did you become involved in PRAJ II? Please provide us with the details of the grant you received – its purpose, amount, target audience, etc. 3. How did you learn of the grant opportunity? What kind of training did you receive on administering the grant from TAF? How would you recommend that the grant process could be improved? 4. On a scale of 1-4, with 1 being very satisfied and 4 being very dissatisfied, how would you rank your relationship with TAF as a grants manager? What can you suggest to improve this relationship? 5. What was your management and work structure before the grant and after the grant? How many new employees were hired after you received the grant? How did the new organizational structure and employees affect your budgeting and sources of funding? 84 6. In terms of your database, how was it constructed? What data do you collect to be entered? How do you collect data? How often? What challenges did you face in collecting data? How did you overcome these challenges? What were your significant achievements? 7. How has the creation of databases, publishing reports, monitoring HR violations, etc., contributed to change in HR in Cambodia? What have been the main challenges in linking information sharing to institutional reform? 8. In general, who do you see as the primary audiences for your work? What has been the outcome of this work with these audiences? 9. Please tell us about how you represent HR defenders? How do you identify clients, how many, etc. How do you determine which cases should be referred to PILFs? What has been the result of the HR cases you have defended? What have been the greatest challenges? 10. How do you investigate the cases you represent? How much time do you have to prepare? How often do you meet with the clients before court appearance? 11. How do you interact with other HR organizations (international, national, provincial, informal)? In what ways do other organizations or individuals contribute to the work of your organization? How do you share information? How often? 12. How would you describe the awareness level of the Cambodian public of their human rights? How would you characterize their involvement in advocating for change? 13. What do you see as your organization’s greatest achievement under PRAJ II? What has been the biggest contributor to that achievement? What have been the main hindrances to the work of your organization? How did you overcome these? 14. What strategies do you think are most effective for creating constituencies for reform? For actually achieving reform? 15. What other strategies would you recommend to pursue in a future project to create judicial reform? 7. QUESTIONS FOR Phnom Penh Municipal Court (posed in Khmer) 1. Please tell us about the job of the PPMC – types of cases, personnel, judges, court process, etc. 2. What is your specific role in the PPMC? 3. How did the PPMC become involved with PRAJ II? 4. How are data collected in the PPMC for the criminal case database? 5. How does the PPMC as a whole use these data? 6. How has the database facilitated the work of the PPMC? How has it hindered the work of PPMC? 7. What do judges think about the database? How do they use it? 8. What do lawyers think about the database? How do they use it? 9. If you could advise EWMI about other information to be included in this database, what would it be? 8. QUESTIONS FOR PUBLIC INTEREST LAW FIRMS (PILF) – Samreth and Vishnu 1. Please tell me about how your law firm became interested in doing legal aid work? Please give me details on how large your firm is and how many lawyers take on legal aid cases. 2. How did you become involved with PRAJ II? Please provide us the details of the grant you received – its purpose, amount, target audience, etc. 3. What kind of legal aid cases do you generally take on and in which locations? Which are the most challenging to defend and why? Overall, what are the major challenges in defending legal aid cases? 85 How has the firm resolved these challenges? What are the results of legal aid cases defended? What changes have been made in the court system as a result of the activities of legal aid organizations? 4. What kind of training did the lawyers in your firm have to conduct legal aid defenses? What other training would you say is needed to prepare a firm to do this? 5. About how many private cases does the firm take on vs. the number of legal aid cases in a year? How does the firm manage this financially? 6. How does your firm let those indigent who have been accused/detained that you are available to defend them? How do they get in touch with you? 7. How do you investigate their case? Generally, how far in advance do you meet with a legal aid client before court appearance? What types of work do you have to do to prepare to defend a client? What would be the ideal amount of time you have to prepare for a case? And how much time do you actually have? 8. Was there any collaboration or coordination between your firm and Legal Aid Organizations? In what way and why? If there is none, should there be a relationship established and how? 9. How would you characterize the relationships you have with EWMI? On a scale of 1-4 with 1 being very satisfied and 4 being very dissatisfied, please rank these relationships. Why this number? Please explain. 10. How would you characterize the relationships you have with your clients? On a scale of 1-4 with 1 being very satisfied and 4 being very dissatisfied, please rank these relationships. Why this number? Please explain. 11. What would you recommend for law firms to strengthen legal aid work? What would you recommend to improve legal aid services in Cambodia? 12. Please explain the split between Samreth and Vishnu. 9. QUESTIONS FOR CBO/CSO/GO LEADERS (IDEA AND PLCN) 1. Please tell me about your organization, the major work it does, the number of staff involved, your constituencies, and where they are located (geographic scope). 2. Please tell me about how you heard about EWMI and how you became involved with them? How did the grant you received from EWMI fulfill your organization’s goals? 3. Please explain how you identify/form constituency groups and their composition, and how you have expanded them. Please name the groups you work with and the substance of their work. How many of your constituency groups are classified as youth groups? How have you expanded and built your constituencies? 4. Before you received the grant from EWMI, what were your funding sources? What other types of support did you receive? From whom? 5. When you wrote your grant proposal, what was the focus of your activity? Did anybody help you write the proposal? For how long was the grant? What was the amount? Was the amount and time covered under the grant sufficient for you to complete the work you proposed? 6. What training did EWMI provide you on managing your grant? What other training did EWMI provide you? What did you learn that you did not know before? How did the training change the way you do your work? What type of training have you had from FIT? Was it adequate? What other training do you need to be able to manage your organization financially? What other training do you think you need to be more efficient in your work? 86 7. What kind of training did you provide your constituency groups? What were your constituency groups able to do after they were trained? 8. How did EWMI monitor the use of your grant? How did you administer the grant, i.e., what portion of the grant went to pay for training and advocacy activities, and what portion went to constituents and other costs? 9. How would you characterize your relationship with EWMI? On a scale of 1-4, with 1 being very satisfied and 4 being very dissatisfied, please rank this relationship. Why this number? Please explain. What have been the challenges in working with EWMI? How were these resolved? What recommendations do you have for improving this relationship? 10. How would you characterize the relationships you have with your constituency groups? On a scale of 1-4 with 1 being very satisfied and 4 being very dissatisfied, please rank these relationships. Why this number? Please explain. 11. What were challenges you and your constituent groups faced to work with local/regional/national governments? What has been their response to your activities? 12. What have been the biggest challenges in working with your constituency groups? How have they been resolved? Overall, how can these relationships be improved? What are the biggest achievements you have experienced in working with your constituency groups? 13. In terms of advocacy, what kinds of campaigns did your constituencies hold? What was the subject matter? What results were achieved through these campaigns? What other strategies do constituency groups employ to expand their outreach? 14. Please tell me about the security issues you have encountered in you or your constituency organizations? As the umbrella organization working with this constituency, how did you address the violence? 15. How do members of your constituency groups share information? Are there some better at sharing than others? If so, which ones, and why/why not share? 16. How will you and your constituency groups continue working when EWMI’s fund ends? 17. Over the period of your involvement with EWMI, how has your organization changed in terms of structure and the way you do business? To what do you attribute these changes? 18. If another small grants program is part of a future project, what differences would you put in the new design? Why? 87 ANNEX VIII: SCHEDULE OF DATA COLLECTION INTERVIEWS AND LEARNING ACTIVITIES Date Time Person Interviewed/Activity Undertaken Position Organization Location Purpose 12/2 8:30-4:30 Team Learning Day on SOW Sofitel How to generate a SOW 12/3 8:30- 10:00 1) April O’Neill 2) Patricia Orlowitz 3) Janice McGuire Dhulce 1) Acting Dir., D&G 2)Project Development Officer, Program Office; 3) Democracy Officer (USAID) USAID Mission Overall discussion of PRAJ II 10:00- 11:00 1) April O’Neill 2) Rebecca Black 3)Sean Callahan 4) Karen Exel-Stone 1) Acting Dir., D&G 2) Mission Director 3) Mission Deputy Dir. 4) Program Office Dir. (USAID) USAID Mission Discussion of SOW; pose questions on project management 11:00- 4:30 Team Debrief and Planning for interviews with EWMI on 12/4 USAID Mission Examination of what learned at USAID interviews 12/4 8:30 – 5:30 1) Andrew Boname 2) Terry Parnell 3) Sin Kimsean 4) Kristen O’Connell 5) Sao Sotheary 6) Sang Tedh Kundhy 7) Silas Everett 8) Koy Neam EvaluationTeam (5) 1) EWMI COP 2) Obj. 4 Team Leader & ODC Proj. Mgr. 3) Obj. 5 Team Leader 4) Editor 5) Co’rdnat’r 6) Legal Office 7) TAF Country Director 8) TAF Program Officer EWMI Office Pose questions on all aspects of PRAJ II 88 Date Time Person Interviewed/Activity Undertaken Position Organization Location Purpose 12/5 9:00- 12:00 1) Silas Everett 2) Koy Neam 3) Chin Lowe 4) Ke Bunthoeurn 5) Moul Samneang Evaluation Team (5) TAF 1) TAF Country Director 2) TAF Program Officer 3) TAF Program Officer 4) TAF Ass’t Program Officer 5) TAF Senior Program Officer TAF Office Pose questions on all aspects of PRAJ II 2:00-4:15 1) Virak Ou 2) Duch Piseth CCHR 1) President, 2) Project Coordinator, Trial Monitoring CCHR Office Pose questions on implement￾tation of grant received 12/6 9:00- 12:00 1) Osung Sieng 2) Sok Roeun 3) Hoy Serey 1) Vice President of PPMC 2) Deputy Prosecutor 3) Assistant/ Data-base Manager PPMC Office Pose questions on Municipal Court changes 1:00-4:30 Team Meeting/ Debriefing on week’s interviews; begin identification of emergent themes USAID Mission Reflection and Feedback Learning 12/7 8:30- 12:30 Team Meeting – Training on PRA data collection techniques; rewriting data collection instruments for following week; planning and finalizing appointments Sofitel Reflection and Feedback Learning; PRA techniques in data collection 12/9 9:00- 12:00 1) Sao Kagna 2) Tep Neth 3) Sok Phanna Vishnu 1) Manager 2) Deputy Manager 3) Senior Legal Officer Vishnu Law Office Pose questions on legal aid activities 2:00 – 4:30 1) Ith Mathoura 2) So Sorphon Samreth 1) Manager/ founding partner 2) Financial Officer Samreth Law Office Pose questions on legal aid activities 89 Date Time Person Interviewed/Activity Undertaken Position Organization Location Purpose 12/10 8:00 – 5:00 Teams working in respective hotel/homes as difficulties in scheduling interviews on UN International Human Rights Day Hotel and Homes 12/11 9:00- 12:00 Ny Charya Head of Investigations, ADHOC ADHOC Pose questions on database data collection and legal representation 2:00-4:30 Naly Pilorge (f) Director, LICADHO LICADHO Pose questions on database data collection and legal representation 12/12 9:00- 12:00 1) Heng Sam Orm 2) Khuth Samony (f) 1) Secretary General 2) Project Assistant IDEA IDEA Pose questions on grassroots urban organization 2:00-4:30 Four female representatives of street vendors; four male representatives of tuk￾tuk and bike/taxis n/a IDEA Conduct PRA exercise with members of two IDEA constituent groups 12/13 8:00- 12:00 Travel to Kampong Thom 2:00 – 4:30 1) Core Leaders from Kampong Thom Prey Lang group 2) Sokor and Satheiri 1) Kampong Thom Prey Lang community 2) EWMI field director and HQ team leader Hotel in Kampong Thom Pose questions of Prey Lang leaders; conduct PRA with field and HQ staff 12/14 8:00- 10:30 Two females and two males from distant Prey Lang villages (two of these were core members of PLCN) Hotel in Kampong Thom Conduct PRA exercise with Prey Lang leaders and villagers 10:30- 2:30 Return travel to Phnom Penh 12/16 8:30- 12:00 1) Run Saray 2) Diep Kulam 3) Ny Chandy LAC 1) Executive Director 2) Lawyer 3) Legal Director LAC Pose Questions of Legal Aid of Cambodia 90 Date Time Person Interviewed/Activity Undertaken Position Organization Location Purpose 1:00 – 4:30 Learning Time on themes for data analysis. USAID Mission 12/17 8:30-5:00 Conduct thematic data analysis Hotel and respective homes/ offices 12/18 8:30- 12:30 Mutual debriefing of findings according to themes Himawari Hotel 1:30-4:30 Continue conducting thematic data analysis; begin to identify points for PPT presentation at USAID Respective hotel and homes 12/19 8:30- 10:30 Continue thematic data analysis Respective hotel and homes 10:30- 12:30 Prepare PPT for USAID Debrief Himawari Hotel 1:30-5:00 USAID debrief and team debrief thereafter; Melissa depart p.m. USAID Mission 12/20 8:30-5:00 Team Work on incorporating USAID comments into analysis; begin work on draft report Himawari Hotel 12/21 8:30-5:30 Continue writing draft report; Yoke depart a.m. Respective hotel and home 5:30 Nancy depart p.m. 91 ANNEX IX: BIBLIOGRAPHY Blue, Richard. 2008. Evaluation of the Program on Rights and Justice (PRAJ). FINAL REPORT: Phnom Penh, Cambodia: USAID. Blue, Richard and Panhavut Long . 2011. Program on Rights and Justice (PRAJ) II, USAID/Cambodia. Interim Evaluation Report. Phnom Penh, Cambodia. USAID. Cambodian Center for Human Rights (CCHR). 2013a Appeal Hearing Monitoring Checklist. Phnom Penh Cambodian Center for Human Rights (CCHR). 2013b. Information Sharing for Evidence-Based Research Dialogue and Advocacy to Improve Respect for Human Rights in Cambodi – “Sithi Project.” Quarterly Progress Report. Phnom Phen Cambodian Human Rights Action Committee (CHRAC). 2013. Compilation of the Reports Submitted by Civil Society Organizations (CSOs) to the United National Human Rights Council During the 18th Session of Universal Periodic Review (UPR) of the Kingdom of Cambodia. Phnom Penh: CHRAC Secretariat. East-West Management Institute. 2010. PRAJ II Workplan for October 1, 2010 to September 30, 2011. New York: EWMI. East-West Management Institute. 2011. PRAJ II Workplan for October 1, 2011 to September 30, 2012. New York: EWMI. East-West Management Institute. 2012a. Facing the Facts: How Evidence-Based Programs Improve Gender￾Based Violence Prosecutions in Cambodia. Phnom Penh: USAID Program on Rights and Justice Implemented by the East-West Management Institute. East-West Management Institute. 2012b. Grants Manual. Program on Rights and Justice. New York: EWMI. East-West Management Institute. 2012c. PRAJ II Workplan for October 1, 2012 to September 30, 2013. New York: EWMI. East-West Management Institute. 2013. Proposed PRAJ II Summary Workplan for October 1, 2013 to September 30, 2014 (submitted in support of one-year extension to Praj II). New York: EWMI. Office of Inspector General, USAID. 2012. Audit of USAID/Cambodia’s Program on Rights and Justice II. Audit Report No. 5-442-12-003-P. Manila: USAID. Phnom Penh Post, December 11 and December 13. Program on Rights and Justice II (PRAJ II) Quarterly Reports: 1. October 1-December 31, 2008 2. January 1-March 31, 2009 3. April 1-June 30, 2009 4. July 1-September 30, 2009 5. October 1-December 31, 2009 6. January 1-March 31, 2010 92 7. April 1-June 30, 2010 8. July 1-September 30, 2010 9. October 1-Decejber 31, 2010 10. January 1-March 31, 2011 11. April 1-June 30, 2011 12. July1-September 30, 2011 13. October 1-December 31, 2011 14. January 1-March 31, 2012 15. April1-June 30, 2012 16. July-September 2012 17. October-December 2012 18. January-March 2013 19. April-June 2013 Program on Rights and Justice II (PRAJ II). 2013. One-Year Funded Technical Application. New York: EWMI. Reports produced by EWMI in response to Evaluation Team request:  Website traffic by month for ADHOC, Sithi.org.  PRAJ Partners Social Media Presence Reports produced by The Asia Foundation in response to Evaluation Team request:  n.d. Number of Indicators for FY 2009-2013  Land Size Affected FY 09-2013  Land Restitution  Result of legal representation (FY09-13)  LICADHO prisoners interviewed from FY 2009-FY 2013  Sample Output Tracking Sheet, Quarterly Outputs, January – March 2011  Monitoring and Evaluation Report for PRAJ II Grantees  Sample Annual Work Plan 2012 The Asia Foundation. 2013. Sub-grant Management Manual. Cambodia Office. Phnom Penh. Tripathi, Vijaya and Patrick Ball. n.d. Initial Benetech Assessment of Cambodian NGOs’ Information Management Strategy and Sustainability of the Martus Tool. Palo Alto, CA: The Benetech Initiative. USAID/Cambodia. 2008. Cooperative Agreement No. 442-A-00-08-00011-00 Titled “Rule of Law and Human Rights in Cambodia. Bangkok: USAID RDMA. Vannuk, Lim. 2012. Assessment Report on PRAJ-Supported Constituency-based Groups. Phnom Penh. Vishnu Law Group. n.d. Case Evaluation form and Case Intake Policy. Phnom Penh: Vishnu Law Group. ANNEX X: DISCLOSURE OF ANY CON 93 FLICTS OF INTEREST For more information, contact: Program Cycle Service Center 1201 Pennsylvania Ave, N.W. Suite 315 Washington, D.C 20004 Tel: 202-347-1420 DevTech Systems, Inc. 1700 North Moore St. Suite 1720 Arlington, Virginia 22209 703-312-6038 www.devtechsys.com