SEPTEMBER 2020 DISCLAIMER: The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. RESILIENCYCAMBODIA BASELINE FINDINGS REPORT: Baseline Findings from a Civil Society RCT in Cambodia— Local Organizations -Movement Toward Self Reliance Activity (LO-MTSR) SANDRA COBURN / THE CLOUDBURST GROUP SEPTEMBER 2020 DISCLAIMER: The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Written and prepared by Aleta Starosta (Cloudburst), Jeremy Springman (DevLab@Duke University), Ryan Hatano (Cloudburst), and Erik Wibbel (DevLab@Duke University), with support from Lucy Right (DevLab@Duke University). Prepared for the United States Agency for International Development Cambodia, USAID Contract Number 72044219C00004, Local Organizations – Movement Towards Self-Reliance (LO-MTSR) Activity Implemented by: The Cloudburst Group 8400 Corporate Drive, Suite 550 Landover, MD 20785-2238 Tel: 301-918-4400 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | i TABLE OF CONTENTS TABLE OF CONTENTS ................................................................................................................................ I TABLE OF FIGURES.....................................................................................................................................II TABLE OF TABLES.......................................................................................................................................II ACRONYMS ..................................................................................................................................................III EXECUTIVE SUMMARY .............................................................................................................................. 1 ORGANIZATIONAL CAPACITY ....................................................................................................................................................... 2 RELIANCE ON USAID AND OTHER INTERNATIONAL DONORS.................................................................................................. 2 NETWORKS...................................................................................................................................................................................... 2 1.0. EVALUATION PURPOSE AND QUESTIONS................................................................................... 4 EVALUATION PURPOSE.................................................................................................................................................................... 4 EVALUATION QUESTIONS................................................................................................................................................................ 4 DATA SOURCES ................................................................................................................................................................................ 5 PROJECT BACKGROUND ................................................................................................................................................................. 6 OVERVIEW OF RESILIENCYCAMBODIA ........................................................................................................................................... 7 2.0 EVALUATION METHODS AND LIMITATIONS ............................................................................. 10 METHODS .......................................................................................................................................................................................10 SAMPLING METHODOLOGY...........................................................................................................................................................11 BASELINE DATA COLLECTION....................................................................................................................................................... 16 DATA QUALITY..............................................................................................................................................................................16 CHALLENGES ENCOUNTERED .......................................................................................................................................................17 AMPLE CHARACTERISTICS..............................................................................................................................................................17 3.0 FINDINGS – ORGANIZATIONAL CAPACITY ................................................................................ 20 4.0 FINDINGS – RELIANCE ON USAID AND OTHER DONORS....................................................... 23 5.0 FINDINGS – NETWORKS.................................................................................................................... 27 6.0 BALANCE AND STATISTICAL POWER.......................................................................................... 30 BALANCE........................................................................................................................................................................................ 30 STATISTICAL POWER.....................................................................................................................................................................30 APPENDIX A. DESIGN REPORT.............................................................................................................. 32 APPENDIX B. INDEX VARIABLE COMPOSITION............................................................................... 33 ANNEX C. RESILIENCYCAMBODIA PARTICIPATING ORGANIZATIONS ................................... 34 APPENDIX D. BALANCE TABLES .......................................................................................................... 40 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | ii TABLE OF FIGURES FIGURE 1. CAUSAL MODEL ......................................................................................................................................... 7 FIGURE 2. CSO SAMPLING METHODOLOGY ....................................................................................................11 FIGURE 3. MATCHING VARIABLES .........................................................................................................................14 FIGURE 4. PROVINCES REPRESENTED ..................................................................................................................19 FIGURE 5. AGE OF ORGANIZATIONS ..................................................................................................................20 FIGURE 6. INTERNAL CHALLENGES FACING CSOS........................................................................................22 FIGURE 7. EXTERNAL CHALLENGES FACING CSOS .......................................................................................23 FIGURE 8. PERCENT OF TOTAL NUMBER OF GRANTS BY SOURCE........................................................24 FIGURE 9. VALUE OF GRANTS BY SOURCE........................................................................................................24 FIGURE 10. GRANT FUNDING SOURCES ............................................................................................................26 FIGURE 11. PERCENTAGE OF BUDGET MADE OF USAID FUNDS .............................................................27 FIGURE 12. NUMBER OF PARTNERS, BY TYPE AND SECTOR......................................................................28 FIGURE 13. SOCIAL MEDIA PRESENCE BY SECTOR.........................................................................................29 FIGURE 14. FACEBOOK POST ACTIVITY BY SECTOR ....................................................................................29 FIGURE 15. STATISTICAL POWER ACROSS EFFECT SIZE AND OUTCOME ...........................................31 TABLE OF TABLES TABLE 1. SAMPLE TRAINING TOPICS...................................................................................................................... 9 TABLE 2. RESILIENCY CAMBODIA COHORTS...................................................................................................11 TABLE 3. SAMPLE SIZE BY TREATMENT GROUP...............................................................................................12 TABLE 4. SAMPLE SIZE BY USAID SECTOR ..........................................................................................................18 TABLE 5. ORGANIZATIONAL SECTORS ..............................................................................................................18 TABLE 6. BASELINE CHARACTERISTICS OF ORGANIZATIONS..................................................................21 TABLE 7. CSO ENGAGEMENT IN ADVOCACY BY SECTOR.........................................................................21 TABLE 8. ALTERNATIVE REVENUE SOURCES.....................................................................................................25 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | iii ACRONYMS CDCS Country Development Cooperation Strategy CPP Cambodia People’s Party CSO Civil Society Organization DRG Democracy, Human Rights, and Governance HI High Intensity ICW Inverse Covariance Weighting IE Impact Evaluation IRB Institutional Review Board LANGO Law on Associations and NGO LI Low Intensity LO-MTSR Local Organizations – Movement Towards Self-Reliance NBM Non-Bipartite Matching RCT Randomized Control Trial ROC Resilient Organizations in Cambodia RGC Royal Government of Cambodia SBAR Small Business Applied Research USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 1 EXECUTIVE SUMMARY This baseline report analyzes baseline data from an Impact Evaluation (IE) of the ResiliencyCambodia program implemented under the USAID/Cambodia’s Local Organizations – Movement Towards Self￾Reliance (LO-MTSR) Activity. The baseline survey analysis has two primary objectives: 1) improve understanding of the project context, particularly the characteristics of Civil Society Organizations (CSOs) in Cambodia and 2) provide a plenary assessment of baseline differences across the IE comparison (intervention) groups that will be used to measure ResiliencyCambodia’s impacts. LO-MTSR and the ResiliencyCambodia program are designed to increase the organizational resiliency of Cambodian civil society organizations across the health, education, food security, agriculture, and democracy, human rights, and governance sectors by increasing their capacity to expand their networks and tap into new markets and revenue streams, thereby decreasing their reliance on aid from foreign donors and empowering them on their journey to self-reliance. The LO-MTSR IE seeks to assess the outcomes and impacts of interventions that occurred as part of the ResiliencyCambodia program, primarily the implementation of an innovative Resiliency Cambodia Framework, which uses adaptive strategies, more effective narratives, alternative organizational and funding models, stronger mechanisms of transparency and accountability, and wider networks across the public, private, and non-profit sector to increase organizational resiliency. Interventions include trainings, toolkits, networking events, and financial resources. The IE is a first-of-its-kind experimental evaluation, using a Randomized Control Trial (RCT) design. RCTs estimate the impact of an intervention by comparing outcomes for treated units with outcomes for a “counterfactual” group that was randomly selected to not receive the treatment. This technique gives implementers and donors a straightforward way to understand program effects and how outcomes would be different if the intervention had not taken place. The random assignment of treatment and control units is the most scientifically rigorous way to establish a causal relationship between an intervention and outcome; it is considered the “gold standard” in policy evaluation. The IE is designed as a tiered intervention with two treatment cohorts, one receiving a low intensity treatment and the other receiving a high intensity treatment. A third group serves as a control. The overarching policy question that underlies this evaluation is: Does building organizational capacity increase the ability of CSOs to strategically plan for and expand cross-sectoral connections, increase their financial diversity, and ultimately increase organizational resiliency? The IE will test three research hypotheses that follow from the evaluation objectives and LO-MTSR Theory of Change at the organizational level. Specific hypotheses in this IE include: Organizations participating in the ResiliencyCambodia program will: • H1. Increase organizational, administrative, and financial capacity • H2. Decrease the organization’s reliance on funding from USAID and other interventional donors, the RGC, and large international donors at odds with USAID’s CDCS objectives • H3. Increase the density of the organization’s CSO network within and across sectors USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 2 This report presents a summary of several aspects of the baseline data collected from treatment and control organizations, including details on sample characteristics, organizational capacity, reliance on USAID and other international donor funding, and organizational networks. The baseline data consists of three sources of primary data, a CSO survey -- a budget survey, and social networking data -- collected from 105 organizations from April – July 2020. This report also assesses the statistical ability of the study to detect changes in important indicators, as well as achieve balance across treatment and control organizations. Our assessment of likely indicator variables, including sector, size, government connections, management capacity, and funding sources indicates the sample is well-positioned to detect moderate, but not small, effects . All of the likely indicator variables were checked for sufficient impact, and the results are listed in Section 6 of the report. Treatment and control organizations are also balanced across all key indicators. Several key summary findings are described below. ORGANIZATIONAL CAPACITY Organizations in the sample are diverse, ranging from small, relatively recently founded organizations to large, well-established ones. On average, organizations have existed for 17 years, have 33 employees and conduct work in four program locations, most commonly Phnom Penh, Siem Reap, and Battambang. Just over 40 percent of organizations engage in advocacy work, and that number rises to 65 percent when only considering organizations in the DRG Sector. Eighty-four percent of organizations conduct an external audit, and just over a quarter of the organizations were able to submit a budget. Twelve percent of an organization’s time is spent engaging in advocacy work. The greatest challenges facing organizations are financial constraints and lack of staff, followed by overworked staff, lack of engagement with businesses, and adapting to changing environments. RELIANCE ON USAID AND OTHER INTERNATIONAL DONORS Annual revenue in 2019 for CSOs in the sample ranged from $10,000 to over $11 million, with an average of just over $500,000. This funding is derived from an average of 2.1 sources of revenue. USAID/Cambodia funding for grants and subgrants is the leading source of revenue. Thirty-eight percent of organizations receive funding from USAID/Cambodia, either as a primary or subcontractor. Of those organizations, USAID funding made up nearly 40 percent of their revenue in 2019. Other common sources of revenue included other international governments, foreign individuals, and local NGOs. Approximately 10 percent of the organizations seek alternative revenue sources, such as income from fundraising, social enterprises, or fees for services or dues, though the amount of revenue in that case is smaller than that received from grants. The Cambodian government funds only six organizations in the sample but provides large amounts of funding to those organizations with established partnerships. No organizations receive any funding from malign actors. NETWORKS Establishing mutually beneficial partnerships enables CSOs to coordinate their activities, share resources, and build solidarity and increase their resilience. At baseline, organizations maintained an average of seven USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 3 active partnerships in 2019. These partnerships were primarily with other Cambodian NGOs, though foreign NGOs were also frequent partners. Their partnerships with the private sector were rare, and in many sectors, non-existent. Social media is a valuable tool for organizations to engage both their community and with potential donors, particularly for organizations in the education and DRG sectors. Eighty-five organizations have a Facebook page, with an average of over 9,000 followers. Since 2016, organizations active on Facebook posted an average of 265 posts per year, which were liked an average of 6,278 times, shared 787 times, and commented on nearly 200 times. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 4 1.0. EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE This report presents findings from the baseline data collected as part of an IE of USAID/Cambodia’s Local Organizations – Movement Towards Self-Reliance (LO-MTSR) Activity, conducted by The Cloudburst Group. The LO-MTSR is a three-year, $2,230,510 USD project funded by USAID/Cambodia as part of the Small Business Applied Research (SBAR) pilot mechanism. The LO-MTSR Activity includes both the implementation of an innovative civil society capacity building program aimed at increasing organizational resiliency, and an evaluation of the effectiveness of the program through a Randomized Control Trial (RTC). The Cloudburst Group is conducting both the ResiencyCambodia program and the IE, in partnership with PartnersGlobal and the DevLab@Duke. The primary development hypothesis driving the ResilencyCambodia program is that by increasing the organizational resiliency of Cambodian civil society organizations across the health, education, food security, agriculture, and democracy, human rights, and governance sectors by increasing their capacity to expand their networks and tap into new markets and revenue streams, it decreases their reliance on aid from foreign donors and empowering them on their journey to self-reliance. The IE proposes an innovative, first-of-its-kind experimental evaluation to measure the key development impacts of the ResilencyCambodia program under the LO-MTSR Activity; it represents the first attempt by USAID to design and evaluate CSO programming using the highest scientific standards. The evaluation will provide an evidence base for improved policy making and programming by testing the development hypotheses and enabling the ResilencyCambodia theory of change to be validated and adjusted as needed to develop a program that could be implemented on a larger scale, both in Cambodia and around the globe. EVALUATION QUESTIONS The LO-MTSR IE is designed to measure the key development impacts of the ResilencyCambodia activities in Cambodia that aim to strengthen a local organization’s resiliency and capacity. This includes increasing network connections, funding diversity, abilities to respond to shocks and protect against external closing space threats. The overarching policy question underlying the evaluation is: Does building organizational capacity increase the ability of CSOs to strategically plan for and expand cross-sectoral connections, increase their financial diversity, and ultimately increase organizational resilience? Based on this overarching policy question, the Cloudburst team developed several research objectives to focus the evaluation activities. Specifically, the evaluation will investigate the extent to which the activities under the LO-MTSR Activity generate the following outcomes and impacts: 1. Increased organizational capacity to protect against external closing space threats and organizational vulnerabilities that impact resiliency; 2. Decreased CSO reliance on funding from USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 5 a. USAID and other international donors b. Cambodian government and large international donors who are at odds with USAID’s CDCS objectives; and 3. Increased CSO network connections within and across sectors. These three evaluation objectives form the basis for a series of testable development hypotheses and indicators on the impact of the LO-MTSR Activity, as well as for measuring the magnitude of that impact. Specific hypotheses to be tested by this IE include: That organizations participating in the ResilencyCambodia program will: • H1. Increase organizational, administrative, and financial capacity • H2. Decrease an organization’s reliance on funding from USAID and other interventional donors, the RGC, and large international donors at odds with USAID’s CDCS objectives • H3. Increase the density of an organization’s CSO network within and across sectors Each of these three primary hypotheses will be modified and expanded into multiple sub-hypothesis in the pre-analysis plan, submitted in Year 2. DATA SOURCES To test these hypotheses, the evaluation will utilize three primary sources of organizational level data to investigate outcomes of interest, including organizational characteristics, networks, revenue-generating activities, employee perspectives of political and financial independence, funding sources, and revenue. The Cloudburst team collected the baseline data for the evaluation from April–July 2020, which included: 1. CSO Baseline Survey (N=173)— Organization-level data was collected from 105 organizations. As called for in the IE design, in 68 organizations, two representatives from the organization submitted the survey. In 37 organizations, only one representative from the organization submitted the survey. Respondents self-administered this close-ended survey through Qualtrics. 2. Budget Survey (N=94)— One representative of each organization collected organization-level data. In 11 organizations, budget data was not submitted before the deadline for randomization but will be captured by the end of Year 1. This close-ended survey was self-administered by respondents through Qualtrics. 3. Social Media Data Scraping (N=105)- Cloudburst collected organization-level data on each post made to a CSO’s Facebook page, including the date of the post, its content (text and images), and the number of likes, comments, and shares that the post received. This information will provide multiple measures for each organization’s Facebook activity and the extent of its online following. This information was collected through an automated Python script. In addition to the automated data scraping, Cloudburst hand-collected information about the organizations’ mission statements, number page likes, number of page followers, number of events (in 2019 and in the future), number of top fans, and if the page is running ads. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 6 PROJECT BACKGROUND In Cambodia, nearly 5,000 local and international CSOs play a critical role as service providers in the health, education, democracy, governance, and agriculture sectors, especially in remote areas and communities. For USAID/Cambodia to achieve its Development Objectives, it is critical for CSOs to be resilient and self-reliant. As donor funding becomes more limited or tied to specific governmental objectives, growing competition for these resources threatens to divide local organizations working toward development goals. While Cambodian organizations have “graduated” to become direct and compliant USAID grantees, they have failed to thrive as independent, mission-oriented organizations with diverse funding sources to insulate them from external shocks. Furthermore, while governments around the world strive to shrink civic space and restrict the operations of organizations promoting accountability and good governance, reduced access to international funding will decrease the independence of CSOs and erode their ability to provide checks on governments. Encouraging CSOs to shift to more sustainable financing models is important for them to continue fulfilling this purpose. CURRENT CHALLENGES FACING CIVIL SOCIETY IN CAMBODIA In recent years, interference in CSO activities by the Royal Government of Cambodia (RGC) has intensified. While this increased scrutiny has resulted in burdensome registration requirements and invasive monitoring practices that affect the entire sector, the brunt of this interference has been targeted toward CSOs receiving foreign support for rights-based advocacy and democracy promotion, as well as local organizations focused on land rights and environmental protection. Much of this increased interference has found a legal basis in the Law on Associations and NGOs (referred to as LANGO) enacted in August 2015 (Curley, 2018), which was met with widespread criticism from civil society and the international community. Chief concerns about the LANGO are mandatory registration for all domestic and international associations, unfettered discretion by the Ministry of Interior over registration, and the requirement that all associations and organizations be “politically neutral”. Legal restrictions on civil society are ever-present, including LANGO, restrictions on media, freedom of expression and assembly, and social media usage. Legal channels have been used to limit political competition, expel foreign NGOs, and eliminate independent media. These moves come despite increased pressure from the European Union and other international actors to encourage a more open civic space. While these actions represent a purposeful closure, Cambodian CSOs’ reliance on international funding, lack of social embeddedness, divisions within civil society, and insufficient CSO capacity make CSOs in Cambodia especially vulnerable to this type of restrictive legislation. For more details about the current circumstances of CSOs in Cambodia, see Annex A, the IE Design Report. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 7 OVERVIEW OF RESILIENCYCAMBODIA The goal of the Local Organizations – Movement Towards Self-Reliance (LO-MTSR) Activity is to build the organizational resiliency of targeted CSOs so they are better able to strategically plan for and expand network connections outside of primary donors. In doing so, organizations will be less dependent on government or large international donor funding that may be at odds with USAID’s Country Development Cooperation Strategy (CDCS) objectives. The primary development hypothesis driving the LO￾MTSR Activity is that by increasing the organizational resiliency of targeted CSOs through the Resiliency Cambodia suite of activities, including the Resilient Organizations in Cambodia (ROC) workshop and assessment, one-on-one coaching, resources for leadership development, and training on entrepreneurship, communications, and social media, organizations will be better positioned to seek out and gain access to funding streams outside traditional international donors such as USAID, or other funding that may be at odds with USAID’s CDCS objectives. This increased financial diversity will empower local organizations to be increasingly independent of donor funds, further advancing USAID’s Journey to Self-Reliance goals. Figure 1, to the right, outlines the causal model approach to implementing performance metrics of the LO-MTSR Activity. The primary activity of the LO-MTSR Activity is the implementation of an innovative ResiliencyCambodia Framework, which uses adaptive strategies, more effective narratives, alternative organizational and funding models, stronger mechanisms of transparency and accountability, and wider networks across the public, private, and non-profit sector to increase organizational resiliency. The framework was adapted from PartnersGlobal’s Resiliency + Framework, which was developed through a rigorous, evidence-based learning process to increase financial resilience and decrease reliance on variable donor funding for local organizations operating in challenging environments. ResiliencyCambodia provides local organizations with the strategies, tactics, tools, and peer-to-peer assistance needed for addressing obstacles to self-reliance through four main steps: 1. Conduct an environmental assessment of the operational landscape to understand the threat, challenges, and opportunities facing local organizations. FIGURE 1. CAUSAL MODEL USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 8 2. Organize and prioritize identified threats and challenges, and chose tactics, tools, and strategies that can be implemented to address, mitigate, or respond to them. 3. Conduct an organizational model and life cycle self-assessment to determine internal strengths, weaknesses, threats, and opportunities related to the organizations’ existing business model and life stage. 4. Devise an intervention plan based on the outcomes from the three previous steps. ResiliencyCambodia utilizes trained facilitators, called Coaches, to help organizations create customized interventions that mitigate the major threats identified in the framework, all of which exist to some degree in Cambodia. The core innovation of ResiliencyCambodia is its contextual adaptability. By building in activities designed to customize the framework locally, ResiliencyCambodia programming is uniquely equipped to address the challenges and needs of organizations operating in diverse contexts. INTERVENTIONS The focus of the impact evaluation is the bundle of ResiliencyCambodia interventions. These interventions are offered to two cohorts at two different intensities: High Intensity (HI) and Low Intensity (LI). One of the largest differences between these two levels of the intervention will be the accompaniment of the program activities with active coaching and mentoring. Because assignment to HI or LI will be random, the IE enables a careful cost-benefit analysis of the additional value created by the resource-intensive coaching component. The relatively small sample size in this case may result in an analysis that is statistically underpowered, but a more speculative evaluation of the relative impact of these two levels of the intervention will still be informative. Resiliency Orientation All organizations will send two staff representing various levels of seniority to participate in a Resiliency Orientation. The one-day small group workshop will introduce organizations to the ResiliencyCambodia Framework, including discussions of the most prevalent civic space threats and common vulnerabilities that make organizations susceptible to civic space shifts. During the workshops, CSOs will implement the Resilient Organizations in Cambodia (ROC) Self-Assessment Tool. By taking this self-assessment, CSOs gain insights into which areas of their organizations present potential vulnerabilities that could either be affected by external civic space threats or prevent an organization from capitalizing on opportunities that may arise in changing civic space. ROC is grounded in both research and insights from the latest in academic and practitioner thinking within the fields of organization-level crisis management, disaster management, organizational resilience across various industries, complex operating environments, and civil society organizational models and capacities. ROC reveals an organization’s capacity and ability to plan for, respond to, and recover from impacts of changing civic space. Resiliency Roadmap Following the Resiliency Orientation, each of the 52 CSOs in the treatment group will create a Resiliency Roadmap with assistance from PartnersGlobal and each organization’s Resiliency Coach, which will be comprised of strategies, tools, tactics, and approaches that correlate to the challenges identified through ROC. Organizations will work with their Coach to develop personalized targets and action plans. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 9 Coaching and mentoring Resiliency Coaches will work one-on-one with each organization to provide on-going mentoring support for interventions detailed in each organization’s respective resiliency plan. Resiliency Coaches will schedule regular check-ins and will be available on-call to assist with specific ad hoc challenges or difficulties organizations may face during the implementation process, for an average of eight hours of support per month for each organization. Resiliency Toolkit LO-MTSR will curate a virtual collection of tools and resources to help CSOs implement their Resiliency Roadmaps. These tools will include information on financial diversification, funding sources in Cambodia, and information on various trainings offered by our partner organization, PartnersGlobal, all translated into Khmer and adapted for the Cambodian context. This toolkit will be available to both the HI and LI cohorts. Group trainings and workshops CSOs will be invited to participate in group trainings and workshops that address organizational needs identified in their respective ROC assessment. Up to six group trainings or workshops will be conducted during the invention timeframe. These workshops will include multi-week virtual trainings led by PartnersGlobal and the Resiliency Coaches, as well as one-day workshops hosted by established capacity building organizations in Phnom Penh, such as Impact Hub. Partners Global will also produce one stand￾alone training guide. An illustrative sample of topics that may be covered at workshops is highlighted in Table 1 below. HI organizations will participate in the full suite of trainings, including in-person discussions to augment the virtual trainings. LI organizations will participate in the virtual components of the trainings, and receive the stand-alone guide, but will not participate in additional trainings offered by local organizations. Training topics will be finalized in Year 2, based on the needs identified in the ROC assessment and Resiliency Roadmaps. TABLE 1. SAMPLE TRAINING TOPICS TOPIC DESCRIPTION / PURPOSE Civil Society Coalition Building Participants would do a civil society mapping of relevant actors in their operating space, conduct awareness raising on the importance of collective action and how to build coalitions based on common interests, and undertake an intensive coalition building activity to create and leverage new partnerships. Knowledge Building For CSO Sustainability To build understanding of the legal issues impacting CSO financial health, we would hold a workshop on regulations that affect CSOs’ ability to access funding. This may include raising awareness of anti-money laundering efforts and bank de-risking practices that unnecessarily target CSOs, restrictions on foreign funding of civil society, and challenges in obtaining tax-exempt status. Strategic Communications and Social Media Raising awareness across organizations around shared issues and causes in addition to raising the profiles of individual organizations can be essential to building lasting change and increasing investments in CSO funding. Organizations and their staff would receive a training on developing a social media strategy, including how to use photography, video, and hashtags that align with best practices to increase their ability to use social media as a resource to expand their audience and amplify their message. The workshop would help organizations to identify influencers, citizen journalists, and cross-promote each other to further build these organizations influence and capacity in line with fundraising goals. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 10 TOPIC DESCRIPTION / PURPOSE Digital Security Knowledge and Expertise Effectively operating in today’s civic space requires the effective use of online tools and resources. Digital security is a critical yet often overlooked underlying factor to any organization as they transfer money, maintain an online presence and identity, store information on staff or stakeholders, and plan project-related activities. A digital security workshop would instill the importance of digital security in the minds of participants, present common security threats, and introduce various tools that are available. Resources for leadership development and institutional support The HI Cohort will receive $3,000 over 18 months for leadership development and institutional support needed to implement their Resiliency Roadmaps. Resiliency Coaches will help organizations identify appropriate leadership and organizational coaching opportunities, including professional conferences/training, network events, courses, and/or trainings by specialized resource partners or consultants. Interventions will be focused on building organizational capacity or helping implement identified improvements. 2.0 EVALUATION METHODS AND LIMITATIONS METHODS The ResilencyCambodia Impact Evaluation (IE), under the LO-MTSR Activity, is a randomized control trial (RCT). RCTs estimate the impact of an intervention by comparing outcomes for treated units against outcomes for a “counterfactual” group that was randomly selected to not receive the treatment. This technique gives implementers and donors a straightforward way to understand program effects and how outcomes would be different if the intervention had not taken place. The random assignment of treatment and control units is the most scientifically rigorous way to establish a causal relationship between an intervention and outcome; it is considered the “gold standard” in policy evaluation. Random assignment is also a normatively fair method for assigning CSOs to programming given that a limited budget necessarily implies that the programming can only be provided to a modest number of CSOs. There have been very few rigorous evaluations of programming on CSOs, so this approach provides an enormous opportunity for USAID to learn crucial, rigorous lessons that might help civil society programming writ large. The IE is designed as a tiered intervention with two treatments cohorts, one receiving a low intensity treatment and the other receiving a high intensity treatment. A third group serves as a control. Originally, the design called for 30 CSOs to be assigned to the low intensity cohort, 30 CSOs to be assigned to the high intensity cohort, and 60 to be assigned to the control group. A description of the intervention for each cohort is described in Table 2, below. This design allows for the pooling of cohorts to evaluate the impact of receiving any R+ intervention against outcomes in the control group and for the comparison of outcomes for each cohort-specific intervention against outcomes in the control group. In other words, the distinct impact of each tier of the Resiliency+ Framework will be measured. This tiered approach serves two purposes: First, it will provide insight into whether, and to what extent, CSO resilience can be improved with a light touch, or if it requires larger, more sustained programming and coaching; second, it will allow for a careful cost-benefit analysis of low- vs. high-intensity programming. To the extent USAID promotes CSOs across the developing world, this careful cost-benefit analysis will USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 11 provide crucial insight for technical teams and help to identify how much of the effects detected by the evaluation were the result of the most resource-intensive components of the intervention. TABLE 2. RESILIENCY CAMBODIA COHORTS INTERVENTIONS COHORT Resiliency Orientatio n & ROC assessment Resiliency Roadmap Coaching and Mentoring Group Trainings Resiliency Resources Resiliency Toolkit Resources for institutional development Cohort 1: Low- Intensity Resiliency Cambodia Intervention (N=20) x x 1 day a month 2 trainings x x Cohort 2: High-Intensity Resiliency Cambodia Intervention (N=32) x x 1 day a month 4 trainings plus interactive discussions x x x Cohort 3: Control (N=53) X (after endline) X (after endline) SAMPLING METHODOLOGY To secure a sufficiently large sample of local organizations, Cloudburst attempted to recruit at least 120 organizations through a call for applications distributed by our partner organizations in Cambodia. The organizations would then be filtered to include only those CSOs with at least three full-time staff members. Those existing for at least three years are registered with the RGC as NGOs, and have been approved by USAID/Cambodia. If more than 120 CSOs applied, organizations would be randomly selected to take part in the baseline survey before being randomly selected using a statistical matching process, detailed below, to be part of the HI cohort (N=30), the LI cohort (N=30), or the control group (N=60). The proposed sampling process is illustrated in Figure 2. FIGURE 2. CSO SAMPLING METHODOLOGY USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 12 However, challenges during recruitment (detailed below in the Challenges section) ultimately led to 105 organizations applying to and being selected for the program. All organizations were matched using matched-quadruplet randomization, described in detail below. The final randomization resulted in 32 High Intensity (HI) organizations, 20 Low Intensity (LI) organizations, and 53 Control organizations. Each of the 105 organizations was asked to complete two CSO surveys and one budget survey. The original intent was to analyze the variance in responses to learn about information sharing within organizations. However, not all of them had two respondents complete the CSO survey. The final sample size by treatment group, including the number of surveys collected, is shown in Table 3 below. TABLE 3. SAMPLE SIZE BY TREATMENT GROUP MATCHED-QUADRUPLET RANDOMIZATION Before assigning CSOs to intervention cohorts, the team uses baseline data to assign organizations to groups, or blocks, of similar organizations. To identify similar organizations, we focus on characteristics that are related to either the outcomes of interest or how CSOs might respond to the intervention. Matching organizations on relevant characteristics and randomizing within these matched groups allows us to compare outcomes across very similar organizations. This technique can dramatically reduce variance and increase statistical power. In short, this matching exercise ensures that our R+ cohorts and control group are as similar as possible, thereby improving our capacity to detect differences at endline 1 All 105 participating surveys will complete a baseline budget survey. The final missing 11 surveys were gathered over the next 3 months, prior to the beginning of organization training. PLANNED ACTUAL ORGS THAT SUBMITTED 2 SURVEYS ORGS THAT SUBMITTED 1 SURVEY TOTAL CSO SURVEYS TOTAL BUDGET SURVEYS High Intensity Organizations 30 32 17 15 49 27 Low Intensity Organizations 30 20 15 5 35 20 Control Organizations 60 53 36 17 89 47 Total 120 105 68 37 173 941 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 13 between organizations that did and did not receive programming. This permits a more precise measurement of measure program effects. This also helps to ensure that important organizational characteristics that shape how organizations respond to the program are distributed equally between the treatment and control groups. Absent initial matching, randomization among a relatively small group of CSOs could produce very different groups of organizations in the treatment and control arm simply as a result of random chance. This potential problem is particularly stark in the cases of sample sizes below 300 (Bruhn and McKenzie, 2008; Greevy et al., 2004). The teamperforms non-bipartite matching (NBM) using the BlockTools R package to produce matched quadruplets. BlockTools uses optimal NBM to create matched pairs and then implements a greedy￾matching algorithm to select the third and fourth member of each quadruplet. Fully optimal matching is not computationally possible for blocks that contain more than two units, so creating quadruplets increases variance within blocks relative to pairs.2 To further reduce this heterogeneity, we re-randomize within quadruplets 5,000 times and select the randomization with the smallest imbalance between treatment and control units.3 Imbalance occurs when there are statistically significant differences between organizations in the treatment and control groups before programming begins. Imbalances make it more difficult to detect the effect of the program on outcomes of interest. 2 An alternative approach would have been to match the NGOs in our sample into most-similar pairs and randomize one NGO in each pair to be treatment and one to be control. Following treatment assignment, we would then create matched-pairs of treatment NGOs and randomize half of the treatment NGOs to receive the high-intensity intervention and half to receive the low-intensity intervention. However, due to the uneven assignment of treatment organizations to high and low intensity cohorts, this would have required some pairs to contain one high intensity and one low intensity while others contained only high intensity, precluding the use of treatment-pair fixed effects. Matched quadruplets were therefore preferable for this application because they will allow us to fully account for the structure or randomization in our estimation using block fixed effects. 3 Due to budget constraints, only 32 CSOs could receive the high-intensity intervention. To achieve this number, some quadruplets had two High-Intensity CSOs and while some have one High and one Low-Intensity unit. Therefore, from the 5,000 randomizations, we limited our selection to those that produced 32 High-Intensity and 20 Low-Intensity units. Within qudruplets, each unit has a 50% chance of being assigned to the control condition, and 31% chance of being assigned to the High-Intensity Cohort, and a 19% chance of being assigned to the Low-Intensity Cohort. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 14 There are several important characteristics that we anticipate with condition how CSOs respond to the treatment, including whether the CSO is registered as a foreign organization, whether it primarily focuses on advocacy work or service delivery, and whether the organization completed all of its assigned baseline surveys. 4 To encourage balance regarding these important characteristics, we apply weights during the matching process to increase the penalty for within-block differences on these variables (relative to other matching variables). This ensures that the matching algorithm prioritizes these important variables to increase the comparability of our treatment and control CSOs on variables that we believe are the most important. Variables that were matched are shown in the text box on the right. There are two characteristics that have been shown to be correlated with capacity but which we believe are of secondary importance. These variables include whether the organization is based in Phnom Penh and the number of years since its founding, which receives a smaller weight of 0.25. This smaller weight means that the algorithms will place a lower priority on creating balance on these variables relative to those with a higher weight. We give these characteristics a smaller weight because they are indirect measures of capacity and we place a higher weight on more direct measures. Finally, there are six broad categories of variables that are either primary outcomes of interest or important potential mechanisms through which the treatment may affect these outcomes. For example, the team believes that increasing the size of CSOs’ networks or diversifying the composition of those networks may allow them to access new sources of funding. Therefore, we consider the characteristics of CSOs’ network as a mechanism through which the treatment affects financial diversification. To that end, the team created six indices: • Confrontation Sector • Size • Management Practices and Capacity • Networks • Government Connections • Revenue Diversification 4 A total of ten CSOs did not complete a budget survey. While these surveys will be completed prior to the initiation of LO-MTSR programming, matching had to be conducted without these data. Because matching does not tolerate missingness, we replaced the values of variables from the budget survey using data from the LO-MTSR application survey when possible and imputed the mean value prior to standardization for the few variables where an alternative measure was not available. Matching Variables High importance (weight = 1) • Registered as foreign NGO • Whether the CSO’ primary focus is advocacy • Whether the CSO’s primary focus is service delivery • Whether the CSO completed the budget survey • Whether the CSO completed both baseline CSO surveys Low importance (weight = 0.25) • Main office in Phnom Penh • Years since founding Outcome and mechanism index variables (weight = 0.75) • Confrontational sector • Size • Management Practices and Capacity • Network size and composition • Government connections • Revenue diversification FIGURE 3. MATCHING VARIABLES USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 15 The components of each index are detailed in Appendix B. The first index - Confrontational Sector - combines measures of whether a CSO works on issues related to democracy and governance or human rights and whether they primarily conduct advocacy work. These characteristics are important factors that determine the amount and types of funding that CSOs can access, the amount of risk they face, and their resiliency needs. For these reasons, we expect CSOs operating in politically sensitive sectors to be similar in important ways and to respond differently to the intervention compared to organizations that do not operate in confrontational sectors. The second index – Size – combines measures of the number of employees, the number of office and programming locations, and the size of the budget. Large and small CSOs will be able to pursue different types of funding and are likely to face different levels of government scrutiny. We also expect large and small CSOs to face different organizational needs and respond differently to the intervention. The third index – Management Practices and Capacity – combines measures of the extent to which CSOs utilize best practices in management and whether they have the capacity to serve as primary grant recipients for large donors and give subawards to lower capacity organizations. We expect a CSO’s management practices and capacity at baseline to shape its ability to tap into new sources of funding and condition how they respond to the intervention. The fourth index – Networks – measures the number of partners each CSO has across both the non￾profit and for-profit sectors. The fifth index – Government Connections – measures the extent to which organizations partner with or receive funding from the Royal Government of Cambodia. The sixth index – Revenue Diversification – includes measures of the number of funding sources and the types of funding sources each CSO has at baseline. All three of these indexes reflect key outcome variables on which we want to detect differences between CSOs that do and do not receive the intervention at endline. To increase our ability to detect these differences at endline, we maximize the similarity of treatment and control CSOs at baseline. The team uses inverse covariance weighting (ICW) to create index variables that summarize variation on several related variables in a single score. Doing so has several advantages. First, the more variables that are used for matching, the poorer the algorithm can create balance on any given measure. Combining several related variables into a single summary score simplifies this process and improves the overall quality of matches. Second, because we measure CSO characteristics using survey questions, some CSOs may provide inaccurate answers to these questions, creating measurement error. This may occur because respondents fail to understand the question or make up answers when they do not know the correct answer.5 For example, respondents may guess the number of full-time employees. Combining several measures of similar characteristics into a similar summary score reduces the chances that measurement error will produce incorrect matches. Annex B shows the breakdown of each of the six index variables by their components. 5 To reduce measurement error for our most important variables, we asked the same questions to multiple individuals within each CSO. Nearly all organizations completed both an application survey and a baseline survey, and CSOs were given the opportunity to receive additional compensation if a second respondent from their organization completed the baseline survey. We then inspected the responses to these important variables by hand, identifying very large or small values or incorrectly formatted responses and using the responses of other respondents to check and correct these values. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 16 ICW allows researchers to combine measures that are not correlated with each another but that are related to a single underlying concept. For example, the amount of funding a CSO receives, the number of individuals they employ, and the number of provinces they work in are all measures of a CSO’s size. ICW applies an assumption that there is one latent trait of interest and constructs an optimal weighted average on the basis of that assumption. In other words, ICW optimizes information content for an index constructed from items that we believe are related to one another. In contrast to methods based on factor analysis or principal component analysis that isolate variance on one dimension, ICW incorporates information about multiple dimensions by placing a higher weight on variables that are uncorrelated when combining variables into an index. We use code provided by Graeme Blaire and Cyrus Samii provided in the stdidx R package to standardize variables and create an ICW index. BASELINE DATA COLLECTION The team collected data through an online survey hosted by the online survey platform Qualtrics. After completing the Call for Applications and being selected to receive the baseline survey, organizations received an initial Qualtrics contact form requesting the primary point of contact at the organization to nominate two staff members to complete the CSO survey and a third staff member to complete the budget survey. The contact form requested names, positions, and email addresses for the employees who would complete the survey. This information became the sampling list for each survey instrument. Baseline data collection took place between April 20 and July 10, 2020. Respondents entered data directly into the online form, which was then exported into csv files and cleaned and analyzed in R. Once contact information was collected, Qualtrics generated an email to link each survey respondent to its survey. This allowed Cloudburst to track who had opened each email, who had started a survey, and who had finished a survey, and follow up appropriately. Each week, respondents who had not yet completed their survey received a personalized email reminder, and as the survey end date approached, respondents received phone call reminders from a ResiliencyCambodia coach. Respondents were offered a $10 payment, sent through the mobile money app Wing, in exchange for completing the survey. As the survey instruments collected only organizational level data, not personal information, Duke University’s IRB Board waived Institutional Review Board (IRB) approval. However, the survey did include an informed consent section that notified survey takers about the purpose of the assessment, the risks and benefits of the study, and the confidentiality of their responses. Respondents then either self-selected that they consented and continued the survey or did not consent and ended the survey. DATA QUALITY The ResiliencyCambodia IE monitored for data quality through individual review of each survey by the Learning and Evaluation Postdoc, as well as automated high-frequency checks conducted in R and run weekly by the Cloudburst Data Analyst. These checks searched for outliers among the survey answers and for systematic errors that should be corrected, including short survey times, missing responses, outliers, and illogical responses. To reconcile differences, follow up emails were sent to organizations requesting clarification. Results were also compared both to the alternative CSO Baseline Survey if available. Organizations could also clarify any points of confusion by contacting Cloudburst or Duke personal directly through email or WhatsApp. Feedback from the organizational respondents was continuously used to improve survey question design. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 17 CHALLENGES ENCOUNTERED RECRUITMENT AND RETENTION The Design Report sampling design called for a minimum of 120 organizations to participate in the IE. Recruitment began in late January 2020, and while Cloudburst ultimately received 127 completed Call for Applications by the end of May, seven organizations were rejected for not meeting the program requirements, and another 15 organizations did not complete the necessary surveys by July 10, the latest date the ResiliencyCambodia program could delay. Ultimately, only 105 of the 120 desired organizations were recruited, and the number of treatment and control organizations were adjusted accordingly. The implications for this reduction in organizations on the design is discussed in greater detail in Section 6, Balance and Power. SLOW SURVEY RESPONSE RATE The original survey period was intended to run for nearly two weeks, from April 20 to May 1, 2020. However, the rolling recruitment of organizations and the shocks caused by COVID-19 necessitated continuing baseline data collection later into the summer. Each week, organizations received personalized email reminders to complete their surveys, and in the later weeks of data collection, were called by ResiliencyCambodia coaches to answer any questions and encourage them to complete their surveys. When possible, in-person visits were made to the organization’s office. The survey period was extended to the last possible date before it interfered with the program’s ability to implement the first phase of the ResiliencyCambodia program in Year 1. Finally, adjustments were made to the original IE design to allow organizations that only answered one CSO survey to be included in the study. That limited the experiment’s ability to analyze the variance in CSO survey responses as an indicator of organizational capacity. COVID-19 Recruitment and baseline data collection took place in the mist of the COVID-19 pandemic. This serious shock to the CSO sector in Cambodia halted international travel, limited in-person meetings, and led many CSOs in Cambodia to shift to remote work for the first time. Other organizations lost access to funding, or experienced a quick surge in funding related to public health needs during the pandemic. COVID-19 had the following impacts on the ResiliencyCambodia IE. • Survey response time was slower as organizations adjusted to COVID-19 shocks • Two organizations that completed a Call for Applications declined to participate in the program, citing lack of capacity due to COVID-19 • The ResiliencyCambodia program also had to be dramatically re-designed in order to conduct its activities without international travel or large group meetings. For more details about the new design, refer to the “Interventions” section in Section 1, Evaluation Purpose and Questions. AMPLE CHARACTERISTICS USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 18 The sample for baseline data collection included 105 CSOs working in sectors across all three of USAID’s technical areas: Agriculture/Food Security, Health/Education, and Democracy, Human Rights, and Governance (DRG). Table 4 illustrates the disaggregation of the sample across these sectors. TABLE 4. SAMPLE SIZE BY USAID SECTOR AGRICULTURE/FOOD SECURITY HEALTH/EDUCATION DRG High Intensity 13 21 17 Low Intensity 9 15 6 Control 17 27 20 Total 29 73 43 Please note that because organizations frequently work across multiple sectors, some organizations are counted more than once. In addition to identifying each organization’s sector by USAID technical area, the baseline survey also identified organizations by a broader variety of sectors, including children’s rights, gender, and advocacy, as well as disaggregating agriculture, food security, health, and education. Organizations could self-identify that they were conducting programming in multiple sectors and were then asked to select the sector they most identified with. Results are shown in Table 5. TABLE 5. ORGANIZATIONAL SECTORS WORK IN THIS SECTOR (%, N) PRIMARY SECTOR (%, N) Service Delivery NGO Delivering services directly to villages, households, or individuals 88% (92) 64% (67) Advocacy NGO Promoting awareness or conducting advocacy for political issues 41% (43) 7% (7) Intermediary Support Organization or CSO Resource Center; Building the capacity and skills of other NGOs and CSOs. 35% (37) 13% (14) NGO Network, Forum, or Umbrella Organization Supporting and building NGO networks 24% (25) 2% (2) Social Enterprise A for-profit commercial entity aimed at promoting social well-being 22% (23) 10% (11) Professional Association Advocating for organizations and individuals engaged in a particular profession 13% (14) 0% (0) Think Tank or Policy Research Organization producing original research to inform public policy 10% (11) 3% (3) Micro-Finance Institution providing loans or savings schemes for individuals or small and medium enterprises 6% (6) 1% (1) Sample locations USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 19 Participants in the ResiliencyCambodia program were recruited nationally. In total, 16 of Cambodia’s 25 provinces were represented in the program, representing every major region of the country, as shown in the map to the right. Just under two-thirds of CSOs (60%, N=63) in the ResilencyCambodia program were based in Phnom Penh, which is not surprising given the clustering of CSOs nationally in the capital. The distant second most common province was Battambang, a major commercial hub and agriculture center located in the northwest of the country. The third most common province, Siemreap, is also located in the northwest of the country. Battambang and Siemreap and have the second and third most populous cities in Cambodia, after Phnom Penh. These provinces contained eight and seven organizations that participated in the program, respectively. Other provinces represented included, but were not limited to, Kampong Cham, Kampong Thom, Ratana Kiri, and Kratie. Figure 4 shows each province and the number of participating organizations from each. Provinces shaded a darker color, such as in Phnom Penh, Battambang, and Siemreap, had more participating organizations. Those with lighter colors, such as Kampot and Takeo, had only a few organizations. There were no participating organizations from provinces in grey. Organizational maturity In order to apply for the ResiliencyCambodia program, organizations hadto have been established at least three years beforehand. This requirement, determined by the Cloudburst team and USAID/Cambodia, was intended to ensure organizations were established enough to fully benefit from the program, and filter out fraudulent organizations. FIGURE 4. PROVINCES REPRESENTED USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 20 F Figure 5, to the left, shows the average age of each participating organization. On average, organizations were 17 years old (founded in 2003). The oldest organizations were Cambodian Women for Peace and Development, a Phnom Penh based women empowerment organization founded in 1978. The youngest organization, Our Future Organization, was an environmental and education focused organization founded only one year earlier. Figure 5 shows a distribution of the organization’s year of founding, with a mean value of 2003 and a standard deviation of eight years. Participating organizations both inside and outside the capital of Phnom Penh were the same age. Organizations that work primarily in the gender and environmental sectors tended to be younger than organizations in the governance sector, although the difference was small and not statistically significant. 3.0 FINDINGS – ORGANIZATIONAL CAPACITY This section presents key findings about the organizational and financial capacity of organizations at baseline. One of the primary goals of the ResiliencyCambodia program is to increase that capacity of CSOs in the treatment group, measured through increased staffing, reduced turnover, increased geographical reach, increased engagement in advocacy, and increased reporting accuracy. Organizations in the sample were diverse, ranging from small, relatively recently founded organizations to large, well-established organizations. Table 6 below shows the total number of employees, number of offices, and number of programming locations for sample organizations at baseline. There was a wide range in organization staff size; while some averaged 33 employees, most had between 10 and 50 employees. This included full (defined as being paid for over 30 hours a week) and part-time. Two much larger organizations participated,, with 472 and 220 employees respectively. With the exception of a few much larger organizations, organizations typically do not have many part-time employees (an average of 3). The average number of volunteers at an organization is 11, although the larger organizations can employ over 100. Organizations are the same size on average across geographic locations, but CSOs working in the environmental and gender sectors tend to be smaller (N=20, sd=33) compared to other organizations. At 93 percent of organizations, the Director has earned a bachelor’s degree or higher. FIGURE 5. AGE OF ORGANIZATIONS USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 21 TABLE 6. BASELINE CHARACTERISTICS OF ORGANIZATIONS MEAN (SD) MIN-MAX Total Employees 33 (55) 4-472 Full Time6 31 (51) 1-459 Part-time 3 (9) 0-89 Volunteers 11 (36.) 0-289 Number of Offices 2.5 (2.1) 1-15 Number of Programming Locations 4.2 (3.3) 1-14 Seventy-two percent of organizations conduct programming in multiple locations, averaging four per organization. Outside the capital region of Phenom Penh, the most common provinces where organizations conduct programming are the western provinces Siem Reap and Battambang (40% of organizations) and the provinces of Kampong Cham and Prey Vang (30% of organizations) surrounding the capital. One of the objectives of a vibrant civil society is to strengthen a democratic political culture. To that end, organizations were asked if they engaged in advocacy activities as part of their mission, and if so, what percentage of their time was spent engaging in advocacy. Table 7, below, shows CSO engagement by sector. In total, 41 percent (N=43) of organizations engage in advocacy, but as expected, this percentage varies substantially by sector. Organizations in the DRG sector are most likely to engage in advocacy (65%, N=28), and devote the highest percentage of time to such activities (12%, sd=10), followed by environmental protection organizations (41%, N=43), who spend 11% (sd=8) of their time engaging in advocacy work. Agriculture and food security CSOs are least likely to engage in advocacy (33%, N=13), and those that do spend only 7% (sd=6) of their time on such activities. TABLE 7. CSO ENGAGEMENT IN ADVOCACY BY SECTOR Finally, to capture technical capacity, the survey asked organizations if they perform external auditing, and how often. Eighty-four percent of organizations (N=89) conduct external auditing, and 60 percent (N=63) do so at least once a year. Organizations were also asked to attach a budget report with their survey to 6 Full time is defined as 30 hour a week or more ENGAGE IN ADVOCACY (%, N) PERCENT OF TIME ENGAGING IN ADVOCACY (%, N) Democracy and Governance 65% (28) 12% (10) Health and Education 37% (27) 8% (7) Agriculture and Food Security 33% (13) 7% (6) Environmental Protection 40% (17) 11% (8) Total 41% (43) 10% (8) USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 22 allow researchers to determine the accuracy of their budget reporting. Just over a quarter (27%, N=26) complied with this request. In subsequent analysis, Cloudburst will assess the accuracy of the budget data using Benford law as a proxy for financial capacity. The IE design report also called for analysis of the variance of answers to the CSO survey by the two representatives of the organization. Challenges that the organizations faced were divided into internal and external categories. The internal challenges were structural problems that organizations faced.. These included staffing, communication, and organizational problems. The most common internal challenge was lack of staff (n=65) and its related problem, overworked staff (n-49). The second most common problem was struggling to adapt practices and procedures to changing environments (n=48). Communication challenges, either with the government (n=30), beneficiaries (n=19) or across the entirety of their networks (n-30) were the third mostly common internal problem. All three of these problems are consistent both across location and sector. FIGURE 6. INTERNAL CHALLENGES FACING CSOS External challenges, shown in Figure 7 below, included problems related to the financial, legal, and social environments in which the organizations operate. Nearly every organization cited financial constraints as a challenge (94%, N=99). Lack of engagement with businesses was the second most common challenge, cited by 50% of organizations (N=52). Challenges in the political environment, such as restrictions in the legal system (45%, N=48), restrictions to free speech (34%, N=36), and harassment (14%, N=15) were also common. Challenges in the political sector were more acutely felt by organizations in the democracy and human rights sector, with 70 percent (N=30) facing legal restrictions, 45 percent (N=19) facing restrictions on freedom of speech, and 25 percent (N=11) facing direct harassment. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 23 FIGURE 7. EXTERNAL CHALLENGES FACING CSOS 4.0 FINDINGS – RELIANCE ON USAID AND OTHER DONORS This section describes important findings related to CSO’s reliance on funding from USAID and other international donors, as well as reliance on funding from the Royal Government of Cambodia (RGC) and other international donors that are at odds with USAID/Cambodia’s CDCS objectives, particularly as they relate to malign actors. Annual revenue for CSOs in the sample ranges from under $10,000 to over $11 million, with an average of $535,202 (sd=$1,218). On average, organizations received funding from 2.1 sources (sd=1.8), including 1.4 grant sources (1.2) and only .73 alternative sources of revenue (sd=1.1). The most common sources of funding are USAID subgrants (33%, N=30) and USAID grants (11%, N=10), other foreign grants (26%, N=24), and local NGO grants (23%, N=21). Grants from the RGC are the least common (6%, N-6), followed by funding from local businesses (9%, N=8) and local individuals (11%, N=10). The majority of grants are provided by foreign sources, mostly foreign NGOS or governments. USAID, through either direct awards or subawards accounts for about 20 percent of the grants and 33 percent of the total value of the grants. Local sources (either the Cambodian Government, local businesses, or local individuals) account for about 23 percent of grants, but are much smaller in value and account for only 6 percent of the total value of the grants. Figures 8 and 9 below show the number and value of grants by source. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 24 FIGURE 8. PERCENT OF TOTAL NUMBER OF GRANTS BY SOURCE FIGURE 9. VALUE OF GRANTS BY SOURCE A little less than half (41%, N=39) of organizations have non-grant sources of revenue, as shown in Table 8 below. 19% (N=18) have more than one type of non-grant source of revenue. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 25 TABLE 8. ALTERNATIVE REVENUE SOURCES ALTERNATIVE REVENUE SOURCE RECEIVE FUNDING FROM SOURCE (%, N) AVERAGE AMOUNT OF REVENUE RECEIVED (MEAN, SD) Income from fundraisers or other special events 13% (12) $85,048 (113605) Income from the sale of goods and other commercial activities (e.g. craft shops, publications, restaurants, etc.) 12% (11) $18,682 (27254) Fees paid by recipients of services rendered by organization (e.g. health clinics, education fees) 9% (8) $25,728 (40296.24) Membership fees and dues 8% (7) $14,832 (30400) Income from services rendered to another NGO/Community-based organization (CBO) 8% (7) $49,779 (73618) Income from services rendered to the government 2% (2) $808,063 (1119967) USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 26 Figure 10 uses data from the budget survey to show the number of organizations who received grants from each of the various grant-making sources, and the amount of funding received. Larger dots represent larger amounts of funding. FIGURE 10. GRANT FUNDING SOURCES USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 27 Funding from USAID As shown above, 38 percent (N=34) of organizations receive a portion of their 2019 funding from USAID/Cambodia. The large percent of organizations receiving funding from USAID is likely related to the LO-MTSR team recruiting CSOs directly from USAID’s pool of local implementing partners. The sample likely overestimates the magnitude of USAID funding in Cambodia’s civil society sector overall and may not represent the sector overall. For organizations that receive USAID funding, USAID funding makes up over a third of their total budgets (39%), as shown in Figure 11 on the right. Funding from RGC and Malign Actors One of the objectives of the ResiliencyCambodia project is to reduce an organization’s reliance on governments that are at odds with USAID’s CDCS strategy, primarily the RGC, China, and Russia, in order to promote CSO independence and ability to engage in advocacy work. At baseline, no organizations in the sample receive any funding from Russia or China, which is consistent with other research on funding sources for organizations in Cambodia. As referenced above, only six organizations (6%) currently receive grants from the RGC, totaling seven awards. Of those six, two are in the democracy and governance sectors, four in the education sector, and three in the environmental sector. Each has between one and three current projects, and partners with between one and three ministries. The Ministry of Agriculture (five of the six organizations) and the Ministry of Commerce (two of the six) were the most commonly partnered ministries. The average award value from the RGC is $22,644 (sd=171,108) in 2019. 5.0 FINDINGS – NETWORKS The final hypothesis of the LO-MTSR Impact Evaluation is that the ResiliencyCambodia program will increase CSOs’ networks within and across their sectors. Establishing mutually beneficial partnerships enables CSOs to coordinate, share resources, and build solidarity and increase their resilience. Partnerships also enable organizations to pursue new funding streams that they may be better positioned to win as a team rather than in a solo bid. At baseline, organizations partnered with an average of 7.1 nonprofit partners (sd=8.61) and a little less than one business partner (.89, sd=2.12). Figure 12 shows a further breakdown. CSOs primarily partnered with Cambodians DRG, education, and public health NGOS. There is roughly an equal amount of foreign NGOS partnered with, most commonly in the economic development, DRG, and environmental sectors. Figure 12 below shows the partnering organization, by type of partnering organization and sector in which the partnering organization operates. FIGURE 11. PERCENTAGE OF BUDGET MADE OF USAID FUNDS USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 28 The graph above shows the total number of partnering organizations broken down by type and sector operated in. The three column groups represent three types of partnering organization, Cambodian NGOs, Development Firms, and Foreign NGOs. There are similar numbers of domestic and foreign partnering NGOs and both are significantly more than local development firms. Shaded in the darkest hue of blue, democracy and governance represent the largest sector of Cambodian NGO partners and the third largest sector among foreign NGOs. Other common partnering sectors are education and public health which represent the second and third largest among Cambodian NGOs, respectively. Food security, while less common among Cambodian NGOs, represents the second largest sector of Foreign NGOs. Social media is a powerful tool for CSOs in Cambodia to connect with their community, constituents, and potential donors. To measure organizations’ networks, as well as their effectiveness at engaging constituents through social media, the LO-MTSR team, led by the DevLab at Duke, tracked the social media activity of each CSO using a Python script that scrapes the Facebook pages of all CSOs that provided the team with their account information, or that the DevlLab@Duke could locate by hand. Since only 20% of organizations use Twitter, this baseline data scraping focused only on Facebook activity. Eighty-five percent of organizations had a Facebook page, with an average of 9,278 followers (sd=33,358). Organizations in the DRG and education sectors were the most likely to have a Facebook page, while public health and food security organizations were least likely to have one. Education CSOs had the highest number of followers, averaging 10,956 (sd=41,628), nearly double that of the second-highest sector, DRG, which averaged 5,256 followers (sd=8,818). Education organizations also created the highest number of Facebook events in 2019 (1.23, sd-2.7). The full breakdown of Facebook activity by sector can be found in Table 9, below. FIGURE 12. NUMBER OF PARTNERS, BY TYPE AND SECTOR USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 29 FIGURE 13. SOCIAL MEDIA PRESENCE BY SECTOR HAVE A FACEBOOK PAGE (%, N) NUMBER OF FACEBOOK PAGE LIKES (MEAN, SD) NUMBER OF FACEBOOK FOLLOWERS (MEAN, SD) NUMBER OF EVENTS IN 2019 (MEAN, SD) Total 84% (89) 9299 (32325) 9278 (32358) 1.16 (2.79) DRG 86% (37) 5372 (8798) 5256 (8818) 0.94 (2.26) Education 85% (51) 11085 (41767) 10956 (41628) 1.23 (2.7) Environment 81% (34) 3812 (5433) 3862 (5526) 0.75 (2.07) Food Security 77% (30) 4359 (6923) 4245 (6926) 0.37 (0.883) Public Health 76% (28) 5440 (8208) 5339 (8189) 0.88 (2.29) The aim of the social media component of the ResiliencyCambodia program is to increase both how often organizations create content for social media and how people engage with that content. Since 2016, 21 organizations have regularly posted on Facebook. Twelve of these organizations worked in the education sector, 9 in public health, 8 in democracy and governance, 8 in environmental protection, and 4 in food security. The democracy and governance organizations were the most active on social media, posting an average of 331 posts over the four-year span, and had the most engagement, with an average of 8,642 total post likes, 259 post comments, and 1,602 post shares. Environmental and food security organizations were the least active on social media, with the fewest organizations, the fewest posts, and the least number of likes, comments and shares. Engagement with social media posts is shown in Table 10. FIGURE 14. FACEBOOK POST ACTIVITY BY SECTOR NUMBER OF FACEBOOK POSTS (%, N) NUMBER OF POST LIKES (MEAN, SD) NUMBER OF POST COMMENTS (MEAN, SD) NUMBER OF POST SHARES (MEAN, SD) Total (N=21) 265 (142) 6278 (8174) 196 (303) 787 (2385) DRG (N=8) 331 (113) 8642 (10200) 259 (373) 1602 (3729) Education (N=12) 276 (136) 5369 (7739) 106 (111) 208 (238) Environment (N=8) 222 (134) 1156 (834) 24 (20) 120 (225) Food Security (N=4) 206 (101) 5316 (8388) 102.75 (168) 198 (278) Public Health (N=9) 245 (148) 5206 (7354) 171 (292) 333 (478) USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 30 6.0 BALANCE AND STATISTICAL POWER BALANCE To assess balance across the three treatment arms (control, low-intensity, and high-intensity), we implemented omnibus balance tests (also known as joint orthogonality tests) using the RItools R package. The tables below present results of that testing for imbalances on important characteristics between CSOs in the pooled treatment and control arm and between CSOs in the High and Low-Intensity treatment arms. Omnibus tests regress the binary treatment variable on specified variables to assess whether the value of the treatment variable ‘predicts’ values of individual variables (one-by-one comparisons) and produces a joint Chi-square test of independence across the joint distribution of these variables. Omnibus tests are preferable for designs that rely on blocking, as they can be taken into account when estimating differences between groups by including block fixed effects in linear regression models. Columns 1 and 2 in the tables in Appendix C present adjusted means between CSOs in different intervention arms. Adjusted means are predicted values from the regression model described above. Column 5 presents the p-value (I.e., statistical significance) for the difference between arms. Rows in tables 1 and 2 capture balance on each of the variables included in the matching procedure. Rows in tables 3 and 4 capture the same balance; however, index variables are replaced with each component variable that is summarized by the index. The final row of each table captures the Chi-Sq joint test for orthogonality across all variables. If matching and randomization have produced a sample that is balanced across intervention arms, the team should not expect to see large differences in the adjusted means and p-values should not reach conventional levels of significance (p < 0.5). As the tables below show, matching prior to randomization has produced a sample that is well-balanced across all three treatment arms. Overall measures of balance across the full sample do not approach statistical significance, and only a handful of the index variable components indicate statistically significant differences between intervention arms. STATISTICAL POWER This section presents power calculations for the LO-MTSR IE. Here, power refers to the capacity to detect an impact if one does exist; the associated power calculations indicate the sample size required for an evaluation to detect a given effect size. The teamevaluates the ability of our IE to detect differences between CSOs in the pooled treatment and control arms using baseline data and the DeclareDesign R package (Blaire et al., 2018). Rather than calculating the Minimum Detectable Effect Size (MDES), we conduct our power analysis using simulations. These provide a more flexible way to evaluate the power of a design by accounting for blocking and the inclusion of covariates. Our baseline and endline data will contain multiple measures of our outcome of interest. However, testing each outcome variable individually (in other words, testing the same hypothesis using multiple outcomes) increases the chances that a significant value for at least one of the variables is determined by random chance. To avoid corrections for multiple hypothesis testing (MHT), we instead focus on index variables that summarize variation on related variables into a single score (see the section on Matching and Randomization for a more detailed discussion of these index variables). In this application, we create 1,000 simulations of the value of the outcome variable at endline by adding random noise to the baseline values. Because the outcome variables will be measured less than two years USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 31 apart and are unlikely to change by a large amount in the absence of the treatment, we expect the value of outcome variables at endline will be highly correlated with their baseline values. For this reason, we structure our simulations to ensure that the simulated values of the endline outcome are highly correlated with observed baseline values at 0.8. We then repeat these simulations across 10 values of the average treatment effect (ATE) defined as between 0.3 and 0.4 standard deviations of the outcome variable. We choose this range of effect sizes because they are considered moderate effects. Power is derived by estimating ordinary least squares with robust standard errors and block fixed effects and conditioning on the baseline outcome value (including it as a covariate on the right-hand side of the equation). For the financial management outcome variable, Figure 15 indicates that for an effect size of ~.32 standard deviations of the endline value, 80% of the simulations returned a statistically significant (p < 0.05) estimate of the treatment effect. The design is slightly less powered for alternative outcomes of interest, with effect sizes of ~.34 and ~0.38 yielding 80% power for the management and capacity and network size outcomes. FIGURE 15. STATISTICAL POWER ACROSS EFFECT SIZE AND OUTCOME The figure above shows how statistical power varies according to the size of the average treatment effect and the outcome under consideration. The red line indicates statistical power for the financial diversification index variable, the green line indicates power for the management and capacity index, and the blue line for the network size index. The shaded regions represent confidence intervals based on 1000 simulations of the data, across 52 treatment organizations and 53 control organizations and a correlation of 0.8 between baseline and endline outcome values, and a significance level of 0.05. In summary, the IE is powered to detect moderate, but not small, effects of the programming. USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 32 APPENDIX A. DESIGN REPORT The LO-MTSR ResiliencyCambodia IE Design Report can be found at the following URL: https://dec.usaid.gov/dec/content/Detail.aspx?vID=47&ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjkt ZTcxMjM2NDBmY2Uy&rID=NTU3NDAw USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 33 APPENDIX B. INDEX VARIABLE COMPOSITION This Annex lists the composition of each of the six indices that capture primary outcomes of interest or important potential mechanisms through which the treatment may affect these outcomes. Index 1: Confrontational Sector • Human rights focus • Advocacy as primary activity • Democracy and governance focus Index 2: Size • The number of full and part-time employees • The number of office locations • The number of provinces in which they have programming • The total revenues in their last budget cycle Index 3: Management Practices and Capacity • Whether they undergo an annual external financial audit • The Executive Director's highest degree • Whether the organization could attach a formal budget report from the last budget cycle • Whether the organization gives subawards Index 4: Networks (Note: These variables are highly correlated; advocacy NGOs have fewer connections) • Number of NGO partners • Number of private sector partners • Number of network organization memberships Index 5: Government Connections • Number of government ministries partnered with • Number of projects involving partnerships with government ministries • Whether they received government funding Index 6: Revenue Diversification • Count of funder types (USAID, other governments, RGC, int'l/local NGOs, foreign/local businesses/individuals) • Count of earned income types (membership fees, services rendered, fundraisers) • Whether the organization engages in commercial activities (selling food, handicrafts) USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 34 ANNEX C. RESILIENCYCAMBODIA PARTICIPATING ORGANIZATIONS NAME SECTOR TREATMENT STATUS Action For Development Democracy and Human Rights, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Advanced Centre for Empower ment Education High Intensity Banteay Srei Orgnaization Democracy and Human Rights, Women's Rights High Intensity Buddhism for Health Public Health High Intensity Chhouk Sar Association Public Health High Intensity Community Development for P eace and Sustainability Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Conserve Indigenous Peoples La nguages Organization Education, Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Cooperation For Alleviation of Poverty Democracy and Human Rights, Education, Food Security High Intensity Damnok Toek Organization Education, Public Health High Intensity Family Health Development Democracy and Human Rights, Public Health High Intensity Farmer and Nature Net Associa tion Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Forests and Livelihood Organiza tion Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Good Neighbors Cambodia Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution High Intensity Habitat form huminty internatio nal Cambodia High Intensity he Cambodian Human Rights an d Development Association Democracy and Human Rights High Intensity Kampuchea Women's Welfare Action Democracy and Human Rights, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution High Intensity Khmer Association For Develop ment of Countryside Democracy and Human Rights, Education, Food Security High Intensity USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 35 NAME SECTOR TREATMENT STATUS Khmer Youth and Social Develo pment Democracy and Human Rights, Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Life With Dignity Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Live & Learn Cambodia Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Marist Solidarity Cambodia Education High Intensity Minority Rights Organization Democracy and Human Rights High Intensity NatureLife Cambodia Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Non-Timber Forest Products Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Our Future Organization Education, Environmental Protection, Women's Rights, Peace and Conflict Resolution High Intensity Partners for Rural Development Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution High Intensity Phnom Srey Organization for D evelopment Democracy and Human Rights, Education, Food Security, Public Health High Intensity Rachna Satrei Democracy and Human Rights, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution High Intensity Sunflower Film Organization Democracy and Human Rights, Education High Intensity The Affiliated Network for Soci al Accountability Democracy and Human Rights, Education, Environmental Protection, Women's Rights, Public Health, Peace and Conflict Resolution High Intensity Wathnakpheap Democracy and Human Rights, Education, Food Security, Public Health High Intensity Youth For Peace Organization Education, Peace and Conflict Resolution High Intensity Agriculture Services Program fo r Innovation, Resilience and Ext ension Education, Food Security, Public Health Low Intensity Cambodia Anti￾Tuberculosis Association Public Health Low Intensity Cambodian Health Committee Public Health Low Intensity Cambodian People living with H IV Network Education, Public Health Low Intensity USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 36 NAME SECTOR TREATMENT STATUS Cambodian Volunteers for Com munity Development Education, Food Security Low Intensity Centre d’Etude et de Développ ement Agricole Cambodgien / C ambodian Center fo Study and Developmen Food Security Low Intensity Children's Future International Low Intensity Community Empowerment and Development Team Environmental Protection, Women's Rights, Peace and Conflict Resolution Low Intensity Indochina Starfish Foundation Education Low Intensity Kdei Karuna Organization Education, Peace and Conflict Resolution Low Intensity Organization to Develop Our V illages Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Low Intensity Partners in Compassion PC Education Low Intensity Sansom Mlup Prey Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Low Intensity Save Vulnerable Cambodians Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Low Intensity Solidarity Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Low Intensity STAR Kampuchea Democracy and Human Rights Low Intensity Urban Poor Women Developm ent Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Low Intensity Village Support Group Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Low Intensity Young Eco Ambassador Cambo dia Education, Environmental Protection, Women's Rights, Peace and Conflict Resolution Low Intensity Youth Council of Cambodia Democracy and Human Rights, Education Low Intensity 3S Rivers Protection Network Children's Rights, Environmental Protection, Women's Rights, Peace and Conflict Resolution Control Action for Heath Development Public Health Control USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 37 NAME SECTOR TREATMENT STATUS Action for Rural Economic Dev elopment of Cambodia Education Control Advocacy and Policy Institute Democracy and Human Rights Control Akphivath Neary Khmer Organ ization Democracy and Human Rights, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Control Alliance for Conflict Transforma tion Peace and Conflict Resolution Control AMARA Democracy and Human Rights, Women's Rights Control Angkor Buddhist Organization Education Control AusCam Freedom Project Inc Education Control AusRelief Limited Democracy and Human Rights, Education, Food Security, Public Health Control Building Community Voices Democracy and Human Rights Control Cambodia Global Fund Democracy and Human Rights, Education Control Cambodian Children Against Sta rvation and Violence Democracy and Human Rights, Food Security Control Cambodian Children's Advocacy Foundation Organization Education Control Cambodian Children's Fund Education Control Cambodian Children's Trust Control Cambodian Civil Society Partne rship Democracy and Human Rights, Women's Rights Control Cambodian Health and Educatio n for Community Education, Women's Rights, Public Health Control Cambodian Institute for Resear ch and Rural Development Food Security Control Cambodian Rural Development Team Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control Cambodian Women for Peace a nd Development Education, Public Health Control Children and Women Develop ment Center in Cambodia Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control Coalition for Integrity and Social Accountability Democracy and Human Rights, Education, Environmental Protection, Women's Rights, Public Health, Peace and Conflict Resolution Control Community Developers Democracy and Human Rights, Environmental Protection, Food Security, Control USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 38 NAME SECTOR TREATMENT STATUS Women's Rights, Public Health, Peace and Conflict Resolution Cooperation Committee for Ca mbodia Democracy and Human Rights Control Environment and Society Organi sation Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control Facilitation Association of Econ omy for Cooperatives Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control Health and Social Development Education, Food Security, Public Health Control Increase Food Security and Dev elopment Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Control Kampuchea Action to Promote Education Education Control Khmer HIV/AIDS NGO Alliance Public Health Control Krousar Yoeung Associaton Education Control Lom Orng Organisation Education, Food Security Control Men's Health Cambodia Public Health Control Men's Health Social Service Education, Public Health Control My Village Democracy and Human Rights, Environmental Protection, Women's Rights, Peace and Conflict Resolution Control Open Development Cambodia Democracy and Human Rights, Education, Environmental Protection, Women's Rights, Public Health, Peace and Conflict Resolution Control Operation ASHA Public Health Control Operations Enfants du Cambod ge Democracy and Human Rights, Education, Public Health Control Partnership for Development in Kampuchea Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Control Rehabilitation and Development for Cambodians Organization Education, Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control Rural Economic and Agriculture Development Agency Democracy and Human Rights, Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Control USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 39 NAME SECTOR TREATMENT STATUS Save Cambodia's Wildlife Environmental Protection, Women's Rights, Peace and Conflict Resolution Control Tekdeysovanphum Organization Environmental Protection, Food Security, Women's Rights, Public Health, Peace and Conflict Resolution Control The Cambodian Center for the Protection of Children's Rights Democracy and Human Rights, Education Control The Children for Better Future Education Control This Life Cambodia Control Trailblazer Cambodia Organizati on Education, Environmental Protection, Food Security, Women's Rights, Peace and Conflict Resolution Control Veterans International Cambodi a Public Health Control Volunteer in Cambodia Education Control Women Organization for Mode rn Economy and Nursing Education, Environmental Protection, Women's Rights, Public Health, Peace and Conflict Resolution Control Women Peace Makers Women's Rights, Peace and Conflict Resolution Control Women's Resource Center Democracy and Human Rights, Education, Women's Rights Control USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 40 APPENDIX D. BALANCE TABLES TABLE 1. INDEX VARIABLES: TREATMENT VS CONTROL CONTROL TREATMENT DIFFERENCE Z-SCORE P-VALUE Foreign 0.08 0.08 0 0 1 Advocacy 0.06 0.08 0.02 1 0.32 Service 0.63 0.63 0 0 1 complete (budget) 0.9 0.9 0 0 1 complete (main) 1.67 1.62 -0.06 -1.04 0.3 Age 16.67 16.91 0.24 0.16 0.87 Capital 0.62 0.58 -0.04 -0.46 0.64 confrontation -0.04 0.06 0.1 1 0.31 Size -0.01 -0.05 -0.05 -0.46 0.65 management -0.01 0 0.01 0.12 0.9 Networks -0.07 0 0.07 0.82 0.41 government -0.07 0.02 0.09 1.07 0.29 Finances -0.03 0.02 0.05 0.42 0.67 Overall 5.3 (Chi-sq) 0.97 TABLE 2. INDEX VARIABLES: HIGH-INTENSITY VS LOW-INTENSITY LOW￾INTENSITY HIGH￾INTENSITY DIFFERENCE Z-SCORE P-VALUE Foreign 0.05 0.05 0 0 1 advocacy 0.1 0.1 0 0 1 Service 0.55 0.6 0.05 0.58 0.56 complete (budget) 1 0.95 -0.05 -1 0.32 complete (main) 1.75 1.7 -0.05 -0.58 0.56 Age 17.65 14.83 -2.83 -1.08 0.28 Capital 0.6 0.55 -0.05 -0.33 0.74 confrontation -0.08 0 0.09 0.51 0.61 Size -0.06 -0.08 -0.02 -0.08 0.94 management -0.04 -0.01 0.02 0.17 0.87 Networks 0.06 -0.03 -0.09 -0.64 0.52 government 0.11 -0.03 -0.14 -0.97 0.33 Finances 0.17 0.02 -0.15 -0.56 0.58 Overall 6.03 (Chi-sq) 0.87 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 41 TABLE 3. COMPONENT VARIABLES: TREATMENT VS CONTROL CONTROL TREATMENT DIFFERENCE Z-SCORE P-VALUE Foreign 0.07 0.09 0.02 0.71 0.48 advocacy 0.05 0.07 0.02 0.71 0.48 Service 0.63 0.65 0.02 0.33 0.74 complete (budget) 1 1 0 0 1 complete (main) 1.72 1.66 -0.06 -0.98 0.33 Age 15.71 16.97 1.26 0.75 0.45 Capital 0.62 0.56 -0.05 -0.54 0.59 human rights 0.66 0.69 0.03 0.39 0.69 advocacy (focus) 0.42 0.41 -0.01 -0.09 0.93 democracy 0.41 0.42 0.01 0.09 0.92 employees 26.02 32.09 6.06 0.82 0.41 Offices 2.34 2.24 -0.1 -0.21 0.83 operations 3.85 3.93 0.08 0.14 0.89 Budget 355506.97 372316.23 16809.26 0.16 0.87 Audit 0.57 0.42 -0.15 -1.55 0.12 subawards 0.15 0.24 0.09 0.97 0.33 ED education 4.31 4.59 0.28 1.6 0.11 Report 0.37 0.18 -0.19 -1.98 0.05 NGO partners 6.47 7.67 1.2 0.5 0.62 bus partners 0.76 1.03 0.27 0.5 0.62 Forums 2.61 2.76 0.15 0.4 0.69 ministry partners 1.47 1.68 0.22 0.69 0.49 ministry projects 1.5 1.53 0.04 0.12 0.91 RGC funding 0.04 0.04 0 0 1 grant sources 1.42 1.38 -0.03 -0.13 0.89 income sources 0.68 0.8 0.12 0.52 0.6 social enterprise 0.15 0.16 0.01 0.11 0.91 Overall 24.17 (Chi-sq) 0.57 USAID.GOV LO-MTSR RESILENCY CAMBODIA IE BASELINE REPORT | 42 TABLE 4. COMPONENT VARIABLES: HIGH-INTENSITY VS LOW-INTENSITY LOW￾INTENSITY HIGH￾INTENSITY DIFFERENCE Z-SCORE P-VALUE Foreign 0 0 0 0 1 advocacy 0 0 0 0 1 Service 0.62 0.77 0.15 1.41 0.16 complete (budget) 1 1 0 0 1 complete (main) 1.85 1.77 -0.08 -1 0.32 Age 18.96 13.73 -5.23 -1.47 0.14 Capital 0.54 0.46 -0.08 -0.38 0.71 human rights 0.85 0.62 -0.23 -1.34 0.18 advocacy (focus) 0.15 0.62 0.46 2.45 0.01 democracy 0.31 0.54 0.23 1.34 0.18 employees 31.54 20.12 -11.42 -0.95 0.34 Offices 1.65 3.23 1.58 1.4 0.16 operations 5.27 2.38 -2.88 -1.97 0.05 Budget 435727.46 302452.92 -133274.54 -0.84 0.4 Audit 0.46 0.31 -0.15 -1 0.32 subawards 0.31 0.23 -0.08 -0.45 0.65 ED education 4.31 4.46 0.15 0.45 0.65 Report 0.15 0.15 0 0 1 ngo partners 11.46 5.5 -5.96 -0.93 0.35 bus partners 0.96 1.46 0.5 0.53 0.59 Forums 2.62 2.54 -0.08 -0.1 0.92 ministry partners 1.54 1.88 0.35 0.6 0.55 ministry projects 1.35 1.5 0.15 0.31 0.75 RGC funding 0.08 0 -0.08 -1 0.32 grant sources 1.62 1.62 0 0 1 income sources 1.15 0.62 -0.54 -1.02 0.31 social enterprise 0.23 0.15 -0.08 -0.45 0.65 Overall 13 (Chi-sq) 0.45