PARTNERSHIP FOR THE DEVELOPMENT OF THE EASTERN CONGO IMPACT EVALUATION BASELINE REPORT August 2023 This report was produced by the Mission Strategic Support Program of the United States Agency for International Development Mission in the Democratic Republic of the Congo (DRC). It was prepared independently by Levison S. Chiwaula – Team Leader (University of Malawi) and Emmanuel Muzigirwa (Innovations & Social Entrepreneurship (IES)) on behalf of SoCha, LLC. Aerial view of Mambasa town in Ituri Province Page | 1 ABSTRACT This report presents the results of the baseline impact evaluation study of the Partnership for the Development of Eastern Congo (P-DEC) project. P-DEC works in the provinces of North Kivu (Beni Territory) and Ituri (Mambasa Territory) with the goal of strengthening the foundations for lasting peace in the region. The program expects to attain its goal through three intermediate results: 1) communities are more resilient to conflicts and crises; 2) institutions are more accountable, trusted, and citizen responsive; and 3) inclusive and diversified economic growth improves stability. As such, the evaluation aims to answer: What is the impact of the P-DEC Project on (a) resilience capacity, (b) social cohesion, (c) livelihood diversification, (d) perceived local government responsiveness, and (e) perceptions on use of violence? The impact evaluation uses quantitative household surveys and a quasi-experimental methodology to measure the impact of P-DEC interventions in households (the treatment group) compared to the counterfactual case of equivalent households where no intervention took place (the control group) over the same period of time. The sample design involved randomly selecting 40 villages each from treatment and control areas. In each village, data collectors systematically sampled a target number of 29 households using the random route technique, and administered a separate community-level questionnaire to village chiefs or their delegates. The achieved sample included 2,435 households and 80 village leaders. The evaluation team analyzed the survey data using the statistical method of propensity score matching (PSM) to create the counterfactual by successfully matching 1,788 households (894 treatment, 894 control) with the necessary statistical balance between the treatment and control groups. Resilience Capacity, measured at the household level using USAID’s published methodology, is the primary impact indicator for the evaluation. The baseline survey found that average household resilience capacity in the study area scored 37.5 out of the possible maximum of 110. We also found low participation in high￾potential value chains (1.93 out of 5) and poor livelihood diversification (4.53 of 15). External and internal social cohesion scores were higher (10.8 and 12.6, respectively, out of 16). Perceptions of local governance responsiveness were 4.7 on a seven-point scale. Finally, 84% of the baseline respondents believe violence should be avoided. Qualitative findings from 36 key informant interviews across nine villages revealed a highly challenging context that differs substantially between the two intervention territories of Beni and Mambasa. Study villages in Beni were the targets of intense attacks by external armed groups. Conflicts in Mambasa were more commonly focused on land disputes and mining. The acute conflict in Beni in particular poses challenges that may demand adapted implementation approaches. Any modified locations or approaches should be documented so the final evaluation can adequately account for these changes. An endline survey of the same households is planned for 2026 to complete the impact evaluation. The evaluation team will then quantify the impact of P-DEC using the difference-in-differences (DID) method to analyze the baseline and endline data from the matched sample. 2 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV PARTNERSHIP FOR THE DEVELOPMENT OF THE EASTERN CONGO IMPACT EVALUATION BASELINE REPORT August 2023 Contract number: 7200AA20D00023/72066022F00001 3 | Page DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. 4 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV CONTENTS Abstract............................................................................................................................................... 1 Acronyms ............................................................................................................................................ 6 Executive Summary ........................................................................................................................... 7 Evaluation Purpose and Evaluation Questions.................................................................................................. 7 Evaluation Design, Methods, And Limitations................................................................................................... 7 Findings, Conclusions, and Recommendations.................................................................................................. 8 Evaluation Purpose And Evaluation Questions ............................................................................. 10 Evaluation Purpose .......................................................................................................................................... 10 Evaluation Questions....................................................................................................................................... 10 Background....................................................................................................................................... 10 Introduction..................................................................................................................................................... 10 The P-Dec Theory Of Change ........................................................................................................................ 11 Impact Indicators............................................................................................................................................. 14 Cost Analysis..................................................................................................................................... 16 Evaluation Methods And Limitations ............................................................................................. 17 Evaluation Design ............................................................................................................................................ 17 Identifying a Counterfactual: Study Sites......................................................................................................... 18 Identifying a Counterfactual: Households ....................................................................................................... 19 Sample Size And Power Calculations.............................................................................................................. 21 Data Collection ............................................................................................................................................... 22 Limitations....................................................................................................................................................... 24 Findings ............................................................................................................................................. 25 Quantitative Findings....................................................................................................................................... 25 Qualitative Findings ......................................................................................................................................... 37 Conclusions ....................................................................................................................................... 40 Recommendations ........................................................................................................................... 41 Works Cited ..................................................................................................................................... 42 Annex I: Timeline............................................................................................................................. 43 Annex II: Evaluation Statement Of Work ..................................................................................... 44 Annex III: Data Collection And Analysis Tools ............................................................................. 54 5 | Page Annex IV: Sources Of Information ................................................................................................. 77 Annex V: Evaluation Team Members ............................................................................................ 78 Annex VI: Power Calculations For The Original Sample............................................................. 79 Annex VII: Distribution Of Sample Between Treatment And Control Regions ....................... 80 Annex VIII: Results Of The Balance Test For Variables Used In Matching Households From Treated And Control Areas ............................................................................................................ 82 Annex IX: Baseline Values Of Components Of The Household Resilience Index..................... 84 Annex X. Characteristics And Outcomes, By Territory And Gender Of Household Head..... 85 6 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ACRONYMS APROHDIV Association for the Promotion of Hygiene and the Integral Development of the Vulnerable CIA Conditional Independence Assumption CORACON Collectif des Radios et Télévisions Communautaires du Nord-Kivu/ North Kivu Community Radio and TV Collective DID Difference in differences DRC Democratic Republic of Congo EWER Early Warning, Early Response FAO The Food and Agriculture Organization of the United Nations IDP Internally displaced people IES Innovations et Entrepreneurial Social IPIS International Peace Information Service (IPIS) LDI Livelihood Diversification Index MSSP Mission Strategic Support Program P-DEC Partnership for the Development of Eastern Congo PSM Propensity Score Matching RCT Randomized Control Trial RIMA Resilience Measurement and Analysis SoCha Social Change SOFEPADI Women’s Solidarity for Peace and Development TOC Theory of Change USAID The United States Agency for International Development WCS Wildlife Conservation Society 7 | Page EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this impact evaluation is to quantify the causal impact of the Partnership for the Development of the Eastern Democratic Republic of the Congo (P-DEC) project on building the foundations for lasting peace in the region. These findings will be used by USAID and P-DEC implementers to identify the extent to which P-DEC interventions led to improved resilience among beneficiaries to inform future programming. P￾DEC expects to build the foundations for lasting peace in the region by achieving three interrelated intermediate outcomes or result areas, namely: communities are more resilient to conflicts and crises; institutions are more accountable, trusted, and citizen responsive; and inclusive and diversified economic growth improves stability. Specifically, this evaluation will answer the following questions: What is the impact of the P-DEC Project on: a) Household resilience capacity (primary indicator) b) Social cohesion c) Livelihood diversification d) Local government responsiveness e) Individual perceptions of the use of violence BACKGROUND P-DEC is a development program in the provinces of North Kivu Province (Beni Territory) and Ituri Province (Mambasa Territory) whose goal is to strengthen the foundations for lasting peace in the region. The program is a cooperative agreement with the United States Agency for International Development (USAID) implemented by a consortium of 12 organizations led by Mercy Corps. P-DEC will work to strengthen household and community resilience by engaging local communities and institutions in diagnosing and identifying solutions to conflict drivers, such as unsustainable natural resource exploitation. The project also aims to build better governance institutions to improve accountability and responsiveness to local population needs. P-DEC will also work to improve the enabling environment for inclusive, sustainable, and ecologically sound rural enterprise development. This involves strengthening value chains that offer economic and social benefits, increasing cooperation among communities and ethnic groups in conflict and providing youth with alternatives to joining armed groups. To quantify the impact of the program on selected indicators, USAID’s Mission Strategic Support Program (MSSP) is conducting an impact evaluation. This report presents the baseline findings of the evaluation. EVALUATION DESIGN, METHODS, AND LIMITATIONS The evaluation's primary impact indicator, household resilience capacity, will be measured through an index developed according to the TANGO "Light" Approach (TANGO International 2018). This method was chosen because its sub-indices also serve as P-DEC impact indicators of interest and because the light approach is less data-intensive and burdensome to respondents in this challenging context. This evaluation design will use difference-in-differences with propensity score matching (DID-PSM) to compare results across areas that will receive P-DEC interventions (treatment) and those that will not (control). Together, DID and PSM help to minimize selection bias to ensure an “apples to apples” comparison between treatment and control beneficiaries. To identify control households, we first employed support from P-DEC implementers to identify non-targeted groupements/communes in the same territories that had similar characteristics to those of their targeted treatment groupements/communes. Then, based on the number of villages in each of these areas, the budget for data collection, and the calculation of statistical power necessary to measure change in the impact indicators between the baseline and endline surveys, the 8 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV evaluation team randomly selected 80 villages (40 from treatment area and 40 from control area) and collected survey data from 2,435 households and 80 community leaders. Thus, the villages and households were randomly selected in both the treatment and control areas to produce representative statistics for the population in each area. Using the baseline survey data, we calculated propensity scores to match treatment and control households across observed characteristics and statistically evaluated the quality of the matching. We then computed the average baseline values of the impact indicators for the matched sample to test the hypotheses that there is no statistically significant difference (95% or greater confidence) between the level of resilience and other impact indicators in the treatment versus the control groups. If P-DEC has impact, then we expect to see a greater positive change in the treatment group relative to the control group between the baseline and endline measurements of the impact indicators. The quantitative evaluation data is complemented by a qualitative study designed to better understand the socio-economic and security context in which P-DEC is implemented and analyze the community-level variation in contextual factors that are likely to affect P-DEC’s performance. At the baseline the evaluation team completed 80 semi-structured interviews across 20 villages and used an Excel matrix to complete qualitative analysis. While the evaluation team found the PSM approach successfully controlled for bias in observed characteristics between treatment and control, a key limitation of this quasi-experimental design, unlike a fully experimental randomized control trial, is that unobservable characteristics that may influence treatment outcomes cannot be controlled. Nonetheless, the evaluation team still believes DID-PSM is the best available approach in this context where randomization is not possible. The security context poses a key risk to this study, as conflict may limit access to study areas for endline data collectors or inhibit the ability to relocate the panel of households at endline due to migration. The sample size was augmented to account for an expected level of attrition due to these issues; however, if attrition is higher than expected, the evaluation team can employ additional sample augmentation approaches to ensure that the study is adequately powered to detect statistically valid impacts. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Following propensity score matching, 1,788 households (894 each from treatment and control) could be suitably matched. Diagnostic indicators for the matched sample, including the region of common support and balance tests, confirmed the matched controls form a valid counterfactual to the households in the treatment area, with no significant differences (at greater than 95% confidence) between treatment and control households at the baseline. The baseline study shows that household resilience capacity in the study area is low at 37.5 out of the possible maximum score of 110. We also found low participation in high-potential value chains (1.93 out of 5) and poor livelihood diversification (4.53 of 15). External and internal social cohesion are above the median value (10.8 and 12.6, respectively, out of 16). Perceptions of local governance responsiveness were 4.7 on a seven-point scale. Finally, 84% of the baseline respondents believed violence should be avoided. Qualitative interviews revealed very different conditions for P-DEC interventions between the two targeted territories. Both Beni and Mambasa are affected by violent conflict, but more intense conditions in Beni render it a more challenging implementation context. Study villages in Beni were the targets of intense attacks by external armed groups, whereas in Mambasa, communal conflicts such as land disputes and mining conflicts were more prominent. Conflict was at the root of many challenges across context factors, especially regarding population movement. While villages in Beni reported experiences with both the displacement of the host population and the influx of IDPs, villages in Mambasa often only experienced a major influx of IDPs. Respondents noted these conditions have led to a drop in agricultural production, as the population no longer 9 | Page has access to fields, the lack of foodstuffs and an increase in food prices, a reduction in trade flows with other regions due to insecurity, an increase in unemployment, and the closing of markets. Another contributor to poor economic conditions was the mining sector, which some accused of reducing available agricultural land and producing few employment opportunities, thereby seeming to increase unemployment and recruitment to armed groups. The baseline is an initial measurement of key indicators, and the calculation of impact requires at least one additional measurement to take place after a suitable interval in which interventions effect change. Therefore, the baseline does not provide for recommendations for programming from the quantitative data. However, the qualitative analysis identified some areas of concern that USAID and P-DEC implementers should take into consideration as the project moves forward. First, the acute crisis in Beni in particular poses substantial contextual challenges that are likely to affect implementation. USAID and P-DEC should reflect on what is necessary for integrated programs like P-DEC to adapt to changing contexts and establish a robust adaptive management plan. Implementers should document key contextual factors and adaptations made in each village throughout the project so that the evaluation team can consider implications for the evaluation. For example, in addition to the main “intention to treat” analysis among all originally targeted communities, the endline might also employ a sub-analysis that removes communities not reached as intended, to test the treatment effect among those that actually received treatment. 10 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The evaluation was commissioned by United States Agency for International Development Mission in the Democratic Republic of Congo (USAID/DRC) as part of the 5-year Mission Strategic Support Program (MSSP). The purpose of the baseline impact evaluation of the P-DEC Project is to provide benchmark values for key indicators before the P-DEC interventions and to provide information about the socio-economic and security context and community-level variations that are likely to affect P-DEC’s performance. The baseline values of the impact indicators will be used together with endline values to quantify changes in the impact indicators caused by P-DEC interventions. The baseline data provide descriptive statistics on the population of the areas where P-DEC plans to implement programming and the selected control area. These data will be useful for the USAID/DRC Peace and Security Office, Mercy Corps and consortium partners, the government of the DRC, and other international donors. The findings will provide a point of reference against which the impact of P-DEC interventions will be evaluated at endline, to be completed in 2026. EVALUATION QUESTIONS The evaluation will answer the following questions: a) What is the impact of the P-DEC Project on household resilience capacity? b) What is the impact of the P-DEC Project on social cohesion at the community level? c) What is the impact of the P-DEC Project on livelihood diversification? d) What is the impact of the P-DEC Project on individual perceptions of local government responsiveness? e) What is the impact of the P-DEC Project on individual perceptions of the use of violence? BACKGROUND INTRODUCTION The Partnership for the Development of Eastern Congo (P-DEC) is a development program in the provinces of North Kivu (Beni) and Ituri (Mambasa) of the Democratic Republic of Congo (DRC) that aims to strengthen the foundations for lasting peace in the region. The P-DEC Project is implemented by a consortium of organizations led by Mercy Corps with financial support from the United States Agency for International Development (USAID). Other implementing partners in the P-DEC Project include the International Alert, the Wildlife Conservation Society (WCS), the Alliance for Responsible Mining (ARM), the International Peace Information Service (IPIS), HIVE, the Fair Congo Foundation, the Association for the Promotion of Hygiene and the Integral Development of the Vulnerable (APROHDIV), Justice Plus, Women’s Solidarity for Peace and Development (SOFEPADI), the North Kivu Community Radio and TV Collective (CORACON), Pole Institute, and other partners (Mercy Corps, 2021). The program was developed after recognizing that despite their economic potential, extensive arable land, and decades of international aid, the provinces of North Kivu and Ituri in the eastern DRC continue to face complex crises, with root causes in conflicts around power, governance, identity, and access to natural resources including minerals, land, and forestry. The P-DEC Project recognizes that the crises and conflicts in eastern DRC are caused by long-standing disputes over the ownership, use, and extraction of natural resources marked by corruption, manipulation by political elites and weak governance systems. The P-DEC Project also identifies that ineffective and unaccountable institutions that fail to address the needs of the 11 | Page people erode citizen trust towards the government, exacerbate ethnic inequalities, undermine civic engagement, perpetuate violent conflicts, and hinder development. The P-DEC Project is thus building long-lasting peace through resilience building to enable communities to better prevent and manage conflicts and their effects while transforming the interrelated social, governance, economic, and environmental dynamics perpetuating crises. The program takes a systems approach, such that peacebuilding is coupled with multi-sectoral reinforcements that mobilize communities and the state around a common framework that is both top-down – enabling the state to deliver on its commitments; and bottom￾up – empowering communities to hold the state accountable, while providing economic opportunities as alternatives to joining armed groups and illegally trafficking natural resources. THE P-DEC THEORY OF CHANGE The theory of change (TOC) for the P-DEC Project states that if communities mobilize to prevent and resolve conflicts peacefully, citizens and government institutions improve accountability and effectiveness of service delivery, and economic growth promotes inclusion and reduces drivers of conflict and support for armed groups; and if these are achieved through anchor institutions and systems actors; then the foundation for durable peace will be strengthened in North Kivu and Ituri provinces (Mercy Corps 2021). Figure 1 details the TOC in the project’s results framework. The overall goal of the P-DEC Project is to strengthen the foundations for durable peace in North Kivu and Ituri provinces. This goal will be achieved through the attainment of three interrelated intermediate outcomes or result areas, namely: a) Communities are more resilient to conflicts and crises; b) Institutions are more accountable, trusted, and citizen responsive; and c) Inclusive and diversified economic growth improves stability. Following these result areas, the P-DEC Project’s strategy starts with building resilience to enable communities to better prevent and manage conflict and its effects, while transforming the interrelated social, governance, economic, and environmental dynamics perpetuating conflicts and crises. Building resilience capacity is the overarching framework of the program. Resilience capacity building in the program involves anticipating and preventing emerging risks. The P-DEC Project is therefore strengthening household and community resilience by engaging local communities and institutions in diagnosing and identifying solutions to conflict drivers, such as unsustainable natural resource exploitation. This approach is important for sustaining the gains from other development interventions in this fragile area. Activities related to this intermediate outcome focus on: 1) community mobilization and dialogues for conflict identification and resolution that support community priorities and action plans; 2) support youth at risk of joining armed groups and ex￾combatants to actively participate in community-based structures that promote equity and cohesion, and prepare youth for governance and economic activities; 3) strengthen conflict-related Early Warning and Early Response (EWER) systems and track community perceptions, rumors, and grievances; and 4) multi-level advocacy efforts to counter misinformation and tackle drivers of conflict. To achieve the second intermediate outcome, the P-DEC Project aims to build governance in institutions, to make them responsive to the needs of the local population. This approach builds on the P-DEC Project’s view that illegitimate, non-inclusive and unfair institutions perpetuate unequal access to essential services, gender and ethnic discrimination, and corruption, leading to grievances that fuel mistrust and violence. Making institutions more accountable, trusted, and responsive to the needs of the population is expected to build trust from the population, thereby making the citizens allow the governance institutions to handle their grievances. This intermediate outcome focuses on four activities—namely: 1) raise citizen voice, including youths, to prioritize and advocate for services; 2) work together with relevant community groups, service providers, ETD and public institutions to develop improvement plans; 3) establish or strengthen social accountability mechanisms used by all parties; and 4) regularly monitor progress with all stakeholders. 12 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV To attain the third intermediate outcome, the P-DEC Project aims to improve the enabling environment for inclusive, sustainable, and ecologically sound rural enterprise development. This involves strengthening of value chains that offer opportunities for mutual economic and social benefits to increase cooperation among communities and ethnic groups in conflict and alternatives to armed group activity. The attainment of this intermediate outcome leverages on four activities: 1) building on existing value chain investments (e.g., cocoa, coffee, vanilla, beans, maize) and collaboration with existing programs; 2) investing in alternative energy value chains to decrease stress on natural resources and illegal logging as conflict sources; 3) developing community forest concession (CFCL) businesses to make the forest-based community model financially viable (through cocoa, vanilla or responsible timber), while reducing harmful natural resource exploitation; and 4) reducing mining-related conflicts and illegality by promoting risk mitigation and increasing transparency of legal and conflict-free minerals chains. Page | 13 Figure 1: Results framework for the P-DEC Project Source: Mercy Corps (2022) Page | 14 IMPACT INDICATORS The evaluation has identified five impact indicators that are measured at household and community levels (Table 1). The primary unit of analysis is a household, where most of the impact indicators are measured, though some are aggregated to the community level through individual-level measurements. Below, we describe the indicators by outlining their relevance to the attainment of the P-DEC TOC, unit of analysis, methods of computation and data needs. The survey items used to measure each indicator are organized into different modules in the survey questionnaire annexed to this report. Table 1: Summary of impact indicators for the P-DEC evaluation Index Respondent Sub-index/Indicator Questionnaire Module Resilience Capacity Head of household Access to cash savings I Access to formal safety nets F Access to humanitarian assistance F Asset ownership: consumer durables D Asset ownership: productive assets D Asset ownership: livestock assets D Bonding social capital E Bridging social capital E Education and training C Woman decision maker in the household Gender equitable decision making J Individual Local government responsiveness M Livelihood Diversification Head of household Livelihood diversification G Participation in high-potential value chains H Social Cohesion Individual Social cohesion: intra-community K Social cohesion: inter-community L Attitude Towards Violence Use Individual Attitude towards the use of violence N HOUSEHOLD RESILIENCE CAPACITY INDEX The household Resilience Capacity index is the primary impact indicator in this evaluation. It will assist in evaluating whether the P-DEC interventions make communities more resilient to conflicts and crises (Intermediate Result 1). According to USAID (2022), resilience is the ability of people, households, communities, countries, and systems to mitigate, adapt to, and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth. Resilient households have the capacity to absorb the impact of a shock, adapt to new ways of living after experiencing a shock, and transform their livelihoods after being impacted by a shock (TANGO International 2018). There are two competing methods of measuring household resilience in the literature: 1) FAO’s Resilience Measurement and Analysis (RIMA) approach (FAO 2016), and 2) the TANGO approach (TANGO International 2018). This evaluation will use the TANGO Approach because it is comprehensive such that some of its sub-indices are used as standalone impact indicators of the P-DEC Project. Sub-indicators of the TANGO Resilience index that are standalone indicators in the P-DEC Project include measures of local 15 | Page government responsiveness and social cohesion. Further to that, a systematic comparison of the TANGO and the RIMA approaches have shown that the two methods lead to similar policy recommendations (Constas, d’Errico and Garbero 2019). The TANGO's household resilience capacity index can be measured using the full or “light” approach. The evaluation team selected the light approach because it is less data intensive than the full approach and uses a shorter household questionnaire. Security concerns in the study area necessitate minimizing time spent collecting data in households. The light approach to household resilience capacity index measurement is constructed from the following elements: bonding social capital, bridging social capital, local government responsiveness, access to cash savings, consumption assets, productive assets livestock ownership, education and training, access to formal safety nets, access to humanitarian assistance, and gender equitable decision making (see Table 1). The calculation of each of the elements is clearly explained in the TANGO Methodological Guidelines (TANGO International 2018). All the calculated individual elements were rescaled to a 0 to 10 scale by using the following formula: 𝑅𝑒𝑠𝑐𝑎𝑙𝑒𝑑 𝑖𝑛𝑑𝑒𝑥 = (𝑂𝑏𝑠𝑒𝑟𝑣𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥−𝑀𝑖𝑛𝑖𝑚𝑢𝑚 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥) (𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥−𝑀𝑖𝑛𝑖𝑚𝑢𝑚 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥) × 10 (1) Where: ● 𝑅𝑒𝑠𝑐𝑎𝑙𝑒𝑑 𝑖𝑛𝑑𝑒𝑥 is the index that is rescaled to a 0 to 10 scale, ● 𝑂𝑏𝑠𝑒𝑟𝑣𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 is the value of the index calculated for a particular household, ● 𝑀𝑖𝑛𝑖𝑚𝑢𝑚 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 the minimum level of calculated index in the sample, and ● 𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 is the maximum value of the index in the sample. The light version of the TANGO Household resilience capacity index was then calculated by summing all the elements, implying that the index will range from 0 to 110. Data for calculating the elements was collected through a household questionnaire that was administered to the household head, primary female decision maker, and one randomly selected individual in the household. SOCIAL COHESION Another impact indicator is social cohesion, which will be measured through a community-level index that measures whether individuals in the village come together either socially or to help others (TANGO International 2018). We measured this index from a set of five dummy variables that capture whether the community came together or assisted each other in the past 12 months with food, labor, and other items, or whether they came together for social gatherings such as weddings. We used two social cohesion indices in this evaluation. The first one is the standard measure as proposed by TANGO International (we called this Intra-Community Social Cohesion), and the second index extended the TANGO Social Cohesion indicator measurement by measuring whether individuals in the village assist or get assisted by individuals of different ethnic group from other villages (Inter-Community Social Cohesion). Two indices of social cohesion were used in this evaluation because the conflicts in the program area can be between individuals within the community or between two communities of different ethnic groups. We collected the data for measuring social cohesion by interviewing a randomly selected individual in the households, although the index was calculated at community level (Table 1). Assessment of social cohesion enables us to evaluate the attainment of the intermediate outcome on communities to be more resilient to crises and shocks. LOCAL GOVERNANCE RESPONSIVENESS 16 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Local governance responsiveness was used as one of the indicators for assessing the attainment of the intermediate result area on accountable, trusted, and citizen responsive institutions. This is a sub-indicator in the measurement of TANGO household resilience capacity index for which individual responses are aggregated to the community level. The indicator measures to what extent the local government responded to community requests for improving community assets or services over five-year period prior to the survey. At the baseline a randomly selected individual in each household was asked whether, in the two years prior to the survey, the respondent or a person in their village requested improvement to roads, schools, health facilities, water service, natural resource conservation, public transportation, and security. If so, they were asked to rate to what extent the local government addressed their request for each issue. Individuals’ ratings across multiple services were averaged, and these summary individual ratings were averaged to create an aggregate village-level local government responsiveness rating. HOUSEHOLD LIVELIHOOD DIVERSIFICATION INDEX The fourth indicator in this evaluation was the household livelihood diversification index (LDI), which will help evaluate whether the P-DEC Project increases economic opportunities for the households. If the program increases economic opportunities for the households, it would imply that the intervention is bringing growth in local economies, thereby attaining its last intermediate outcome on inclusive and diversified economic growth. The index was calculated by counting the number of livelihood activities undertaken by the household in the past 12 months. Inclusivity of the economic growth was assessed by computing the index for young people (households headed by young people), women (households headed by females), and men (households headed by older men). This index was calculated at household level from data that were provided by the head of the household. PARTICIPATION IN HIGH POTENTIAL VALUE CHAINS P-DEC plans to strengthen value chains that offer opportunities for mutual economic and social benefits to increase cooperation among communities and ethnic groups in conflict as alternatives to armed group activity. P-DEC has identified rice, cocoa, coffee, maize, and beans as high potential value chains. We therefore included an indicator for the participation in these sectors, computed at the household level. The index was computed additively from dummy variables of whether a member of the household participated in these value chains, implying that it ranges from zero to 5, depending on how many of these crops were implicated. ATTITUDE TOWARDS USE OF VIOLENCE The P-DEC Project promotes the use of non-violent approaches to conflict resolution as one of the ways that will ensure lasting peace in Ituri and Beni. To measure the attainment of this goal, the evaluation used a binary question that assesses whether the elected individual in the household ever considers use of violence as an appropriate tool for resolving conflicts. COST ANALYSIS P-DEC is a cooperative agreement with a total estimated cost of $39.5 million over five years. At the baseline there is insufficient data for analysis that links activity expenditures to impact indicators because few beneficiary interventions have been undertaken. In preparation for the endline, the evaluation team will work with P-DEC to identify expenditure data that characterizes specific villages and/or households in the treatment group. The evaluation team would then use these econometric variables in the regression analysis of impact indicators to quantify the extent to which different sums and types of P-DEC expenditures determined the presence, absence, and/or magnitude of treatment effects. If feasible, this cost analysis would provide valuable insight into which interventions and levels of expenditure had the greatest effect on activity impacts. 17 | Page EVALUATION METHODS AND LIMITATIONS EVALUATION DESIGN This impact evaluation aims to quantify the causal impact of the P-DEC Project by answering the evaluation questions. Ideally, impact evaluations use experimental designs that involve random assignment study participants into the treatment and control groups. Because this random assignment controls for both observed and unobserved factors that influence impact, randomized control trials (RCTs) are considered the gold standard in impact evaluations (Gertler, et al. 2016). When evaluating development interventions such as the P-DEC Project, random assignment of treatment is not feasible for ethical or logistical reasons (Harris, et al. 2006). Program areas are often selected by the program designers based on some pre-determined criteria. The participants in the program can be selected by the program designers, or they can select themselves to participate in a program. Self-selection occurs when participants are allowed to decide on whether to participate or not. When the choice of the program area and the selection of program participants is non-random, impact evaluations use quasi-experimental methods to quantity treatment effects (Gertler, et al. 2016). One of the common evaluation methods that suits the P-DEC evaluation is the “difference in differences” (DID) approach in which data is collected from beneficiaries and non-beneficiaries before and after the implementation of the program. The DID compares the change in an impact indicator over time in the control group to changes over time in the treatment group before and after a program and attributes the observed “difference-in-differences” to the program. Given data on impact indicators for the P-DEC and non-P-DEC households before and after the implementation of the intervention, we derive the DID estimator of impact as follows (Fredriksson and Oliveira 2019): 𝐷𝐼𝐷 = (𝑌̅ 𝑃𝐷𝐸𝐶, 𝐴𝑓𝑡𝑒𝑟 − 𝑌̅ 𝑃𝐷𝐸𝐶, 𝐵𝑒𝑓𝑜𝑟𝑒) − (𝑌̅ 𝑛𝑜𝑛−𝑃𝐷𝐸𝐶, 𝐴𝑓𝑡𝑒𝑟 − 𝑌̅ 𝑛𝑜𝑛−𝑃𝐷𝐸𝐶, 𝐵𝑒𝑓𝑜𝑟𝑒) (2) In equation 2, 𝑌̅ stands for the average value of the impact indicator for a given group (P-DEC or non-PDEC), while 𝐴𝑓𝑡𝑒𝑟/𝐵𝑒𝑓𝑜𝑟𝑒 refers to periods after and before the interventions, respectively. The first bracket measures the change in the impact indicator for the treatment group while the second bracket measures the change in the impact indicator among the control group. The difference between the changes (difference in differences) is therefore attributed to the treatment. For statistical inference of the DID estimate of the impact indicator in equation 2, an ordinary least squares (OLS) equation (linear regression) of the following specification is used: 𝑦 = 𝛽0 + 𝛽1𝑡𝑖𝑚𝑒 + 𝛽2𝑃𝐷𝐸𝐶 + 𝛽3𝑡𝑖𝑚𝑒 ∗ 𝑃𝐷𝐸𝐶 + 𝜀 (3) Where time is the dummy variable that takes a value 1 for the endline and 0 for the baseline and PDEC is the dummy variable for the program taking the value 1 for the households in treatment group, and 0 for households in the control group. In this specification, the impact indicator is measured by the coefficient, 𝛽3, of the interaction between time and PDEC. That is, if 𝛽3 is statistically significant, then the model will show that the average increase in resilience in the treatment group was greater than the increase in the control group. Finally, 𝜀 is the error term that captures unexplained variance in the regression model. The DID method provides unbiased treatment effect estimates when parallel trends are observed in the impact variable in both the treatment and control groups over the same period (Stuart, et al. 2014). A concern with DID models outside of an RCT context is that the program and treatment groups may differ in ways that would affect their trends over time, or their compositions may change over time. Propensity score methods are commonly used to handle this type of confounding (Stuart, et al. 2014). This evaluation has therefore adopted a difference-in-differences propensity score matching design (DID-PSM). This method has been applied in impact evaluations of similar resilience-building programs (Smith, et al. 2022). PSM uses 18 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV observable characteristics to identify and match households across treatment and control groups that have a similar likelihood of having been targeted for or participated in the intervention. Analysis using PSM compares outcomes across matched pairs. This approach attempts to approximate randomized treatment to control for factors that might influence the trajectories of change between treatment and control groups. In the P-DEC Project, the treatment area was selected by Mercy Corps and its implementing partners. There are no household specific treatments because the P-DEC Project operates at the system and institutional level. All households in the program area were therefore exposed to the treatment and qualify to be treated households. The evaluation identified control areas and selected control households from those areas. The evaluation will also employ a panel design, in which the same households will be surveyed at the baseline and endline (2026). This approach will help constrain random sampling error to increase the chance of measuring statistically significant changes at the endline. IDENTIFYING A COUNTERFACTUAL: STUDY SITES This impact evaluation is implemented in the territories of Beni and Mambasa, respectively in the provinces of North Kivu and Ituri, in eastern DRC, where the P-DEC Project is implemented. All households located in groupements/communes (the smallest administrative divisions) where Mercy Corps is implementing the P￾DEC Project are assumed to be exposed to the P-DEC Project. All groupements/communes that are not targeted by the P-DEC Project could potentially be in the control region. To obtain a good counterfactual from the non-program groupements/communes, the evaluation purposively selected groupements/communes which were deemed to have similar characteristics with those in the treatment areas in terms of level of mix of urban, peri-urban, and rural centers; main economic activities (agriculture, mining, forestry); ethnic groups and customs; and security situation. The selection of control areas also ensured that the control and treatment sites are far apart (at least 25 km) to avoid contamination/spillover effects from the PDEC program. Contamination could be the cause for underestimation or overestimation of the treatment effect of the program (Paul & Small., 2011). The selection of the control groupements/communes was guided by Mercy Corps. After identifying the universe of treatment and control areas, the evaluation team randomly selected sample villages for data collection using Stata software. This clustered sampling approach facilitates logistical implementation of the baseline survey, as random sampling of households across these groupements/communes was not feasible. The summary of the procedure used to select sample villages and households is presented in Figure 2, and the selected study territories, groupements, and villages are presented in Annex VII. 19 | Page Figure 2: “Pipeline” approach in identifying study units IDENTIFYING A COUNTERFACTUAL: HOUSEHOLDS Using household data collected from targeted villages described above, we used propensity score matching (PSM) to pair households from the treatment and control areas that are similar in terms of their observable characteristics (Dehejia and Wahba 2002). Based on the observed characteristics of households and the communities they lived in, PSM computes the probability that a household will be in the treatment area. These “propensity scores” are used to match the households to construct a counterfactual (Dehejia and Wahba 2002, Caliendo and Kopeinig 2008). The evaluation team conducted PSM by first estimating the logit model, where being in the P-DEC area and non-P-DEC area was the dependent variable. Second, propensity scores were computed from the logit model for each of the households. Households were then matched based on the propensity scores. The third step involved the evaluation of the quality of the matching. These steps were performed in Stata software by using the psmatch2 command and its associated post commands such as the psgraph and pstest. The logit model from which the propensity scores were computed was specified as follows: 𝐿𝑖 = 𝑙𝑜𝑔 ( 𝑃𝐷𝐸𝐶𝑖 1−𝑃𝐷𝐸𝐶𝑖 ) = 𝛽0 + 𝛽1𝑋𝑖 Where 𝐿𝑖 is the logit, which is defined as the log of the odds ratio of the household 𝑖 being in the P-DEC area. In the definition of the odds ratio, 𝑃𝐷𝐸𝐶𝑖 = 1 if the household is in the P-DEC area and 𝑃𝐷𝐸𝐶𝑖 = 0 if the household is not in the PDEC area. The variable 𝑋𝑖 stands for observable characteristics of household 𝑖. Since the propensity scores are computed based on the X variables (observable characteristics) that are included in the selection equation, it is advisable to select these variables carefully. The selected observed characteristics are expected to fulfill the conditional independence assumption (CIA). The CIA states that the impact variable(s) must be independent of treatment conditional on the propensity score (Caliendo and Kopeinig 2008). In other words, we should only include observable characteristics that influence simultaneously the inclusion in the treatment and the impact indicators (Caliendo and Kopeinig 2008, Harris and Horst 2016). Simulation studies in medical literature supports the use of observable characteristics that 20 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV are related to both the program and the outcome to obtain the least bias (Harris and Horst 2016). Since the P-DEC Project is allocated to a geographical area that was selected purposively, the matching attempted to equate the communities in the treatment and control areas in terms of characteristics that would influence household resilience and other impact indicators. Since the matching is conducted at the baseline, and the evaluation will be conducted at endline, we will still attain the independence of the impact indicators conditional on the propensity scores. Our choice of observable characteristics was also motivated by variables used in a propensity score matching done in similar context (Smith, et al. 2022) and the model specification that ensured that we obtain good balance between households from the treatment and control areas, while having large proportion of households from the treatment area in the region of common support. The observable characteristics used for PSM are presented in Table 2. Table 2: Definition of variables used to match households from the treatment and control regions Variable Name Description Measurement Level Age head Age of the head of the household measured in years Household Age head squared The square of the age of the household head Household Sex head Sex of the household measured as a dummy variable (male=1) Household Married head Marital status of the household head capture whether the head is married or single Household Household size Number of individuals in the household Household Household size squared The square of household size Household Education adult Existence of an adult member who has some formal education measured as a dummy variable Household Consumption assets Number of consumption assets owned Household Productive assets Number of productive assets owned Household Livestock owned Number of livestock owned Household Crop production Dummy variable for households who obtained income from crop production Household Livestock production Dummy variable for households who obtained income from livestock production Household Agricultural wage Dummy variable for households who obtained income from agricultural wage Household Mining wage Dummy variable for households who obtained income from mining wage Household Formal employment Dummy variable for households who obtained income from formal employment Household Natural resources Dummy variable for households who obtained income from natural resources Household Petty trading Dummy variable for households who obtained income from petty trading Household Rentals Dummy variable for households who obtained income from rentals Household Residence length Number of years a household has existed in the community measured in months Household Religion Dummy variable for region of household head taking 1 for Christianity Household Time town Time from village to the nearest town measured in minutes Community Primary school Access to primary school education measured at community level by time taken to reach the primary school Community Formal safety nets Availability of formal safety nets measured as dummy variable Community 21 | Page Variable Name Description Measurement Level Peace structure Existence of a functional peace structure in the community measured as a dummy variable Community All weather access Availability of all year access road measured as a dummy variable Community Electricity supply Availability of electricity supply by an institution in the village. This is measured as a dummy variable Community Time to market Time taken to reach nearest market measured in minutes Community Time to water point Time taken to portable water point measured in minutes Community Humanitarian support Availability of humanitarian support in the village measured as a dummy variable Community Early warning Availability of conflict related early warning systems Community Bonding social capital Level of bonding social capital in the community Community Bridging social capital Level of bridging social capital in the community Community Local government responsiveness Level of local government responsiveness in the community Community SAMPLE SIZE AND POWER CALCULATIONS Determination of sample size considered the need to attain adequate statistical power 1 to detect the minimum effect of the P-DEC Project on the primary impact indicator, the household resilience capacity index. As a standard, we adopted a statistical power of 80%, and we planned to involve 70 villages (clusters) from the treatment and control areas. The inter-class correlation was assumed at 10%, and the statistical significance level at which we tested the hypothesis was assumed at 5%. We used these assumptions in Optimal Design software2 to determine the average number of households to interview in a village (cluster size) and the minimum detectable treatment effects in the primary impact indicator. The results are presented in Annex VI. In the figure, delta is the detectable effect size, while 𝑛 is the number of households per village. The results therefore show that we needed a minimum of 24 households per village (total 1,680 households) to detect a 25% change in the primary impact indicator, at 80% statistical power. With consideration that the program is implemented in a conflict and crisis region, there is a possibility of high levels of attrition at the village level (villages may be inaccessible at times due to security reasons) and at the individual level (individuals may migrate from the village). Therefore, we adjusted the number of villages and the number of households per village by 15% and 20%, respectively. The upward adjustment of the sample will also take care of the households that will not be matched. As such, the study planned to involve 80 villages (40 villages in the P￾DEC areas, and 40 villages in the non-PDEC areas), and we targeted 29 households per village. At the baseline, data were collected from 80 villages (40 treatment villages and 40 control villages) in 2,435 households, meaning that the average cluster size is 30 households. The evaluation team determined the 30- 1 Statistical power is the probability of rejecting the null hypothesis when a specific alternative hypothesis is true (Spybrook, et al. 2011). It is normally set at 80% 2 A free online tool for conducting power analysis that is found at http://hlmsoft.net/od/. 22 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV household cluster size would be optimal for fieldwork efficiency to reach the total sample size necessary for adequate statistical power and accounting for sample attrition between the baseline and the endline, while achieving enough observations in each village to allow for village-level analysis of intra-cluster variance. A total of 43 villages (18 treatment and 25 control) are from Beni, while 37 villages (22 treatment and 15 control) are from Mambasa. The total number of households in the treatment group was 1,230 (559 households from Beni, while 671 households from Mambasa), and a total of 1,205 households from the control group (757 households from Beni and 448 households from Mambasa). DATA COLLECTION Local data collection partner Innovations & Social Entrepreneurship (IES) completed data collection in April 2023. The baseline impact evaluation used both quantitative and qualitative data. The quantitative data was used to quantify the treatment effects, while the qualitative data aimed to provide explanations for the observed results and establish an understanding of the conditions under which P-DEC could achieve or fail to achieve substantial improvements intended outcomes. The combination of quantitative and qualitative data in conducting impact evaluations is encouraged, as it provides a full picture of the impact of the program. QUANTITATIVE DATA Quantitative data were collected through interviewer-administered questionnaires at the household and community level (Annex III). The household questionnaire collected data from three sources: household head (household questionnaire), primary female decision maker, and a randomly selected individual. • Household questionnaire – this component was administered to the household head (per the household’s own designation) or their representative to capture household level data. Data collected through the household questionnaire include household socio-demographic characteristics, education and training levels of household members, ownership of assets, social capital, availability and access of humanitarian aid, livelihood sources, and access to savings. The modules in the household questionnaire enabled the evaluation team to quantify the baseline values of the household resilience capacity index by using TANGO International’s Light approach, and other impact indicators. • Primary female decision maker questionnaire – the primary female decision maker was interviewed to collect data that enabled us to estimate the gender equitable decision-making component of the household resilience capacity indicator. We did not replace the female decision maker in cases where they were not available during the survey period. In female-headed households, the primary female decision maker was the household head. • Individual member interview - An individual who was at least 18 years old was randomly selected from the household roster to respond to the individual interview questionnaire. The individual was asked questions concerning social cohesion, local governance responsiveness, and their attitude towards the use of violence. These data were used to compute the household resilience capacity, indices of social cohesion, and an indicator of attitude towards the use of violence. • The community questionnaire was administered to the head of the village (Chef de village) or his/her representative in 80 villages. The community questionnaire includes modules and questions to generate community level matching variables as well as some community level components of the household resilience index capacity. In the absence of household lists in the sampled villages, we identified sampled households systematically by selecting every nth household from the village chief's house. We moved in all directions from the village head’s house to ensure that we give households in all directions fairly equal chances of being selected. To randomly 23 | Page select an individual within the household, the Roulette application3 was applied to all household members who met the inclusion criteria. Both the household and community questionnaires were configured online by using the Kobo Toolbox and downloaded to tablets (KoboCollect). QUALITATIVE DATA Qualitative data were collected through semi-structured key informant interviews (KIIs) to complement the impact evaluation with information that enriches the different themes, indicators, and overall objectives of the evaluation. Starting on April 18, 2023, trained field workers interviewed 80 respondents in total (20 women and 60 men) across 20 randomly selected villages (14 P-DEC villages and 6 non-P-DEC villages). In each village, four individual key informants were targeted including the village chief, health workers, head teachers, religious leaders, women’s group leaders, youth group leaders, and local peace committee leader. The village chief and at least one woman were required from each village. The scope of these interviews was focused on understanding the context of violence and other shocks in the community as well as livelihood opportunities and assistance available to the community. For the analysis, the interview recordings were transcribed and translated into French. In an iterative coding process using an Excel matrix, the analysis team identified aspects and determined thresholds that captured important variation between villages for each context factor. QUALITY CONTROL Data collectors received detailed training, and they implemented a pilot among households in Goma to identify challenges to be addressed in the questionnaires before launching full-scale data collection. During data collection, each evening (8:00pm - 8.30pm) and morning, field coordinators held meetings with their teams to provide feedback and advice. At the end of each day, a Data Quality Assurance and Statistics team checked the survey database to review the survey data uploaded. Any observed inconsistencies were shared with the field coordinators for them to share the information with the field enumerators. This allowed Field Supervisors to correct any inconsistencies or errors, to ensure that the data had been collected within the required time (9:00 a.m. to 4:00 p.m.) and within the minimum timing (40 minutes- 1 hour 30 minutes) but also to give feedback to enumerators. A WhatsApp group was created to facilitate discussion and communication with the field team. The WhatsApp group enabled teams to update their peers on progress in data collection at their various villages, share any challenges being encountered so that group members can propose corrective actions based on their experience in their different locations. Quantitative data quality control also involved back-check calls with a 10% sub-sample of the respondents to: (1) confirm whether the respondent actually participated in the study, (2) determine the consistency of data collected by asking the respondent 3 questions to which they had previously responded; 3) receive feedback from the respondents on the behavior and attitudes of the field teams during data collection, and 4) gauge the level of satisfaction of respondents who participated in the study. The back-check calls helped to assess the level of quality of data collection and identify areas for improvements. The average call duration was between 7 to 10 minutes per respondent. We contacted 240 survey participants with back check calls. ETHICAL CONSIDERATIONS 3 Name Roullette Application is a free application used to select names from a given list randomly. 24 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV The evaluation team ensured that no rights of participants were violated in the course of implementing this evaluation through a number of strategies. Firstly, the team obtained a general community consent from the head of the territory to conduct the survey in the region. Once in the village, the team started by obtaining an oral village level consent from the chef de village to conduct the study. Further, the evaluation obtained informed consents from all individual participants before the quantitative or qualitative interviews started. The participation was voluntary and participants were free to withdraw from the interview at any time with no negative repercussions. The study was conducted in a high risk, low security region. To ensure the safety of the research team, the security situation of each village was assessed before entering the villages. In addition, the evaluation ensured adherence to research ethical standards by keeping the identities of the participants confidential. All study participants have been assigned a unique identification number linked to their names, and names of participants will be deleted when sharing the collected information with stakeholders outside the evaluation and the P-DEC Project. DATA MANAGEMENT Data were cleaned using Stata 16 and R software. Data cleaning also involved identifying outliers and inconsistent data. The outliers and inconsistent data were corrected by replacing them with the median values. Other inconsistent values were replaced by physically checking them and detecting the source of the inconsistency. A Stata Do-file and an R script file have been developed to trace the data cleaning and management processes. LIMITATIONS A limitation of this design approach is that propensity score matching can only control for bias in observed characteristics. Unobserved characteristics that may influence outcomes of treatment or control groups and therefore render the groups less comparable cannot be known or controlled. The evaluation team still believes PSM plus difference-in-difference is the best possible approach in this context where randomization is not possible. The positive PSM diagnostic results shared later in this report provide encouraging evidence that it effectively renders comparable treatment and control groups based on observed characteristics. High rates of migration in the study area pose a risk to the evaluation team’s ability to follow up with the same households at endline, and a degree of attrition is expected. The baseline sample size needed to detect a 25% change in resilience capacity was augmented to account for attrition and expected loss of households for which matches could not be identified. As discussed later in this report, the post-matching sample exceeds the required sample size by 108. In this unstable context, it is possible inability to relocate households (attrition) can be higher than expected at endline. In this case, the evaluation team will consider augmenting the sample with household replacement approaches and/or using a one-to-many PSM approach or the alternative approach of coarsened exact matching in which control households could be matched with multiple treatment households to improve precision. Violent conflict and general instability in the study areas, particularly Beni, may limit access to study areas at endline. These issues may also lead to modified or scaled back implementation in high-threat areas such that anticipated treatment effects would be attenuated. 25 | Page FINDINGS In this section we first present quantitative findings detailing the effectiveness of the matching process and the comparative values for key indicators. Following this, we share preliminary qualitative interview findings to identify contextual conditions in the targeted territories. QUANTITATIVE FINDINGS FINDINGS ON MATCHING OF HOUSEHOLDS FROM TREATMENT AND CONTROL AREAS The evaluation team completed surveys in 2,434 households (1,229 in P-DEC treatment areas and 1,205 in control areas) and among 80 village heads (split evenly across P-DEC and control villages). We begin by presenting the findings of the matching process because further analyses depend on the ability to identify valid counterfactuals to the households in the treatment area. The results of the logit model that was used to estimate propensity scores are presented in Table 3 below. The likelihood ratio Chi-square statistic is significant at 1%, implying that the covariates in the model are jointly explaining the probability of observing a household in the treatment area. A number of individual covariates are also significant. This shows that systematic differences exist for households in the treatment area from the households in the control area, hence the need to match the households between these groups. The results from the model were used to estimate the probability that a household will be in the treatment area. The propensity scores were then used to match households from the treatment area with households from the control area by using the one-to-one nearest neighbor matching algorithm with no replacement and a caliper of 0.50. This algorithm was selected because it yields quality matches by having a large common support area and balanced matches. After matching the households, we evaluated the quality of the match by assessing the area of common support and evaluating the balance in the covariates. The common support condition ensures that persons with the same values of the observed characteristics have a positive probability of being both participants and nonparticipants (Heckman et al., 1999). This was assessed by comparing the distribution of propensity scores of households in the treatment and control groups. The graphical presentation of the common support condition is presented in Figure 3. The figure shows that some of the households from the treatment area were outside of the region of common support (i.e. off-support). We found 323 households from the treatment area to be off support (meaning having propensity scores that could not be matched with comparison households), implying that 73% of households in the treatment area were in the region of common support. Since we employed 1-to-1 matching, households off support were discarded, resulting in a final analytical sample of 1,788 matched households (894 in the treatment and 894 in the control group). 26 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Table 3: Results of logit regression for computing propensity scores Variable Coefficient t-statistics Age of HH head -0.0032 -0.16 Age of HH head, squared -0.0000 -0.07 Sex of HH head (male=1) -0.1948 -0.83 Married HH head 0.3581 1.57 Household size -0.0810 -1.18 Household size, squared 0.0065 1.38 Education adult -0.3234** -2.42 Consumer assets 0.0266*** 3.42 Productive assets -0.0593*** -3.31 Livestock owned -0.0147*** -5.25 Crop production dummy -0.0765 -0.74 Livestock production dummy 0.2507** 2.16 Agricultural wage dummy -0.3027*** -3.19 Mining wage dummy -0.2297 -1.54 Formal employment dummy 0.3283** 2.28 Natural resource income dummy 0.7397*** 5.83 Petty trading dummy 0.3978*** 3.96 Rentals income dummy 0.2575* 1.72 Residence length 0.0004 1.61 Religion (Christianity=1) -1.0873*** -5.83 Time to town (minutes) 0.0020*** 6.73 Primary school dummy -1.1917*** -9.80 Formal safety nets -0.0007 -0.01 Peace structure 0.3070*** 2.71 All weather access -0.3184*** -3.02 Electricity supply 0.2190 1.58 Time to market (minutes) -0.0041*** -2.78 Time to water (minutes) 0.0006 0.17 Humanitarian support 0.0614 0.42 Early warning 0.4575*** 4.07 Bonding social capital -0.1099** -2.32 Bridging social capital -0.0744 -1.57 Local government responsiveness 0.0521 0.65 Constant 1.9528*** 3.09 Pseudo R-squared 0.1435 Chi 2 484.49*** N 2435 t statistics in second column * p < 0.10, ** p < 0.05, *** p < 0.01 F i g u r e 27 | Page Figure 3: Common support for propensity scores for being in P-DEC intervention area The balance test was conducted to evaluate whether the control group and the treatment group of the matched samples were similar in all relevant characteristics (Smith et al., 2022; Harris and Horst, 2016). We used the standardized mean bias which is the difference of sample means in the treated and matched control subsamples as a percentage of the square root of the average of sample variances in both groups (Caliendo and Kopeinig 2008). This is the recommended statistic for comparing the means because it does not depend on sample sizes and distribution properties (Austin, 2009). There is no clear cut-off for deciding the level of acceptable standardized bias in the mean, but many studies consider the bias below 5% as sufficient (Caliendo and Kopeinig 2008). Smith et al (2022) indicates that the mean standardized percent biases of 10% is acceptable. The graphical results are presented in Figure 4 and the numerical results are presented in Annex VIII. The results in the table and the figure show that the standardized percent biases for all the 33 but five variables are below 5%. The covariates whose standardized percent bias are above 5% are still lower than 10%. This shows that the matching procedure has balanced the households in the treatment and control areas in terms of observed characteristics. Figure 44: Balance across covariates used in the propensity score matching 28 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV CHARACTERISTICS OF THE SAMPLE In this section, we provide summary statistics on the characteristics of the households and communities to contextualize the study area. We also present the results of t-tests for the differences in means of the variables between the treatment and control groups, which followed a simple regression approach: 𝑋𝑖 = 𝛼 + 𝛿𝑇𝑖 + 𝑢𝑖 Where 𝑋 are the observed characteristics, whose means are being compared, 𝑇 is the treatment status taking the value 1 for households in the area with P-DEC Project and 0 for households in the area without the P￾DEC Project. The standard errors for these tests were robust to clustering. The same procedure was used to test the differences in the means in the impact indicators. While our interest is to test the balance in the characteristics in the matched sample, we present the findings for both the matched and unmatched samples to illustrate the gains from the matching procedure. Descriptive statistics depicting household characteristics are presented in Table 4. In the unmatched sample, marital status of the household head, religion, and ownership of livestock are found to be significantly different between the treated and control households while the rest are not significant. We however, found no statistically significant difference in all the variables in the matched sample presented between the households in the treatment areas and control areas, confirming the balance that was presented in the previous section. Findings on characteristics of the household head show that the average household head is 45 years old, male (86%), and married (85%). A large proportion of households (86%) have at least one adult member who attained at least primary education. The average household size in the area was 5.6. On average households owned 0.78 hectares of land. The small landholding sizes is partly explained by households that live in the urban areas and those who are forcibly displaced for security reasons and do not have farm plots. For non￾land assets, the study has found that on average households in the study area possess 10.6 consumption assets, 4.0 productive assets, and 9.3 livestock. Though families had lived in their village for 14 years on average, 28% of surveyed households in Mambasa and 13% of Beni households had only been in their village less than three months (data not shown in table), likely reflecting the transient IDP population. Page | 29 Table 4: Characteristics of households in the study area Unmatched sample Matched sample Control Treatment t-statistics for mean differences Control Treatment t-statistics for mean differences All Age head 45.5 44.7 -1.14 44.9 45.2 0.35 45.0 Male head 0.85 0.88 1.50 0.86 0.86 0.00 0.86 Married head 0.83 0.88 2.15** 0.85 0.86 0.19 0.86 Household size 5.60 5.62 0.11 5.63 5.49 -0.72 5.56 Primary educationa 0.87 0.85 -0.95 0.86 0.85 -0.33 0.86 Christian 0.96 0.86 -3.09*** 0.94 0.94 -0.10 0.94 Residence duration (yrs) 14.5 13.7 0.75 14.0 14.1 0.03 14.0 Plot size (hectares) 0.19 0.27 -0.67 0.20 0.26 0.08 0.23 Consumption assets 0.14 0.23 0.97 0.16 0.22 0.31 0.19 Productive assets 0.87 0.80 -1.12 0.78 0.79 -0.09 0.79 Livestock owned 10.3 11.1 -2.74*** 10.4 10.6 -0.74 10.5 N 2434 1788 t statistics computed by using clustered standard errors * p < 0.10, ** p < 0.05, *** p < 0.01 Note: a At least one adult completed at least primary education Source: Authors from PDEC Baseline Survey Page | 30 We also assessed access to basic services and infrastructure, access to formal safety nets and existence of conflict and peace structures in the communities. These services and infrastructure assist in strengthening the resilience of the communities and households to various forms of shocks and crises. The results are presented in Table 5. Largely, treatment and control communities are similar both before and after the matching. Only 16% of households in the study area have electricity supplied by the government or other organizations, suggesting there is pressure to use natural resources as a source of energy. These are essentially households residing in the city of Beni. The rest of the survey areas are rural, most without electricity (as is the case of the entire territory of Mambasa and rural Beni). We also found that about 73% of households reside in communities that have mobile phone network while 87% of the households reside in communities that receive radio signals. Residents of Mambasa Territory have far lower phone signal access than Beni (56% compared to 90%. See Annex X). The lack of telephone coverage in some villages limits their access to relevant information/communications for their resilience to shocks/stress. This is the case of the Niania group where the inhabitants of certain villages must walk for several hours to make telephone calls. In terms of physical access to the communities, the findings show that up to 49% of the households live in communities that are inaccessible by road at some times of the year. Accessibility differences between territories were stark, with only 38% of Beni villages accessibly in all weather, in contrast to 70% of Mambasa villages (Annex X). In these cases, P-DEC Project implementation may depend on additional efforts to improve accessibility of these targeted communities. The findings also show that 59% of households live in communities that have access to social safety nets (food or non-food), but only 1% have access to humanitarian assistance in their community. Households in Beni had higher rates of safety nets (73%) than Mambasa (35%) (Annex X). Only 50% of the households lived in communities that had functional peace structures, and 55% reside in communities that have conflict-related early warning systems, meaning nearly half have no warning system for impending conflicts and dangers. These indicators suggest a high need for the types of support P-DEC will provide. Page | 31 Table 5: Access to infrastructure and basic services in the study communities Unmatched sample Matched sample Control Treatment t-statistics Control Treatment t-statistics All Electricity supply 0.15 0.15 -0.01 0.16 0.17 0.06 0.16 Radio signal 0.88 0.98 1.68* 0.87 0.97 1.49 0.92 Phone service 0.73 0.81 0.81 0.73 0.78 0.50 0.76 Time to town 135.3 224.3 1.66 154.4 165.8 0.26 160.1 Village population 17499.1 15639.4 -0.30 12955.9 15332.9 0.46 14144.4 Primary school in village 0.83 0.65 -1.84* 0.77 0.77 -0.05 0.77 Formal safety nets 0.60 0.58 -0.24 0.59 0.55 -0.33 0.57 Humanitarian support 0.11 0.12 0.44 0.12 0.10 -0.49 0.11 Peace structure 0.50 0.53 0.27 0.50 0.50 0.00 0.50 All weather access 0.50 0.53 0.23 0.49 0.53 0.00 0.51 Time to market (minutes) 31.2 28.6 -0.32 30.6 31.4 0.30 31.0 Time to water (minutes) 18.5 19.9 0.42 18.6 19.0 0.09 18.8 Early warning systems 0.50 0.58 0.72 0.55 0.53 0.11 0.54 N 2434 1788 Clustered standard errors in used in computing t-statistics * p < 0.10, ** p < 0.05, *** p < 0.01 Source: Authors from PDEC Baseline Survey Page | 32 In contextualizing the treatment area, the study also assessed the livelihood strategies of the households in the area (Table 6). In line with community characteristics, households in the treatment area systematically differed from households in the control area, but matching eliminated these systematic differences. Agriculture is the main source of income as reflected by the proportion of households that reported income from crop production (64%), livestock production (24%), and agricultural wage (52%). Petty trading is the second most important income source in the area. Wage employment is also an important livelihood strategy with most of the households reporting that they received a wage from agriculture (52%), mining (12%), and other non-agriculture and non-mining activities (39%). This shows availability of labor and employment opportunities. Only 12% of the households obtained income from formal employment. In Mambasa, for instance, formal employment may essentially be teachers and other state agents. Indeed, without security, electricity, and other basic infrastructure needed by entrepreneurs, the potential to create local employment remains untapped. With few exceptions, agricultural products are sold without much added value. In Mambasa, where 25% reported mining wages (Annex X), the few rare companies are in mining with their machines to the detriment of young people (artisanal operators), which creates conflicts between young unemployed people and these companies. Page | 33 Table 6: Livelihood activities of households in the study area Unmatched sample Matched sample Livelihood Activities Control Treatment t￾statistics Control Treatment t￾statistics All Crop production 0.66 0.63 -0.79 0.64 0.65 0.03 0.64 Livestock production 0.23 0.24 0.45 0.24 0.24 0.04 0.24 Agricultural wage 0.57 0.48 -2.39** 0.52 0.51 -0.41 0.52 Mining wage 0.11 0.15 1.03 0.12 0.12 -0.11 0.12 Wage other 0.39 0.41 0.58 0.39 0.40 0.37 0.39 Formal employment 0.10 0.15 2.40** 0.12 0.13 0.48 0.12 Natural resources 0.13 0.24 3.78*** 0.15 0.16 0.27 0.16 Petty trading 0.56 0.68 3.31*** 0.62 0.64 0.51 0.63 Rentals 0.091 0.13 2.15** 0.098 0.10 0.23 0.10 Remittances 0.24 0.25 0.29 0.24 0.24 0.00 0.24 N 2434 1788 Clustered standard errors in used in computing t-statistics * p < 0.10, ** p < 0.05, *** p < 0.01 Source: Authors from PDEC Baseline Survey 34 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV BASELINE VALUES OF IMPACT INDICATORS One of the objectives of the baseline study in this evaluation was to establish the baseline values of the impact indicators, which will be compared with the endline values in the PSM-DID framework. In this section, we present summaries of the baseline values of the impact indicators (Table 7). The findings show that all the t-statistics are lower than their critical values, implying that the baseline values of the impact indicators in the matched sample from the treatment and control areas are similar. The level of household resilience capacity in the study area is 37.5 (37 for the treatment group and 37.9 for the control group) compared to the maximum attainable level of 110. Low levels of household resilience are largely explained by low access to cash savings, consumption assets, and livestock (0.99, 0.91, and 0.41 out of 10, respectively. See Annex IX). The study area reported relatively higher levels of local government responsiveness, education and training, bonding social capital, bridging social capital, and access to formal safety nets, all of which were near the middle of the ten-point scale. The findings point to areas that can be prioritized by P-DEC to adequately improve on household resilience capacity. Looking to other impact indicators, on average, households cultivated approximately two out of five high potential value chains (Table 7), with 23% not cultivating any of the valued crops (data not shown). The prevalence of cultivating maize and rice differs substantially by territory (Figure 5). Females who are heading households are less involved than male-headed households in every type of high potential crop (Figure 6). Figure 5. Participation in high potential value chain crops, by territory (matched sample, N=1,788) 35 | Page Figure 6. Participation in high potential value chain crops, by gender of HH head (matched sample, N=1,788) There was a relatively high level of social cohesion within communities, but this was slightly lower when we considered between communities of different ethnic groups. In a scale of 0 to 16 social cohesion score within the village was 12.6 while social cohesion outside the village was 10.8. The local governance responsiveness is 4.7 on a scale of 0 to 7. Out of the 15 available income generating activities, we found that on average, households participated in about a third. There is also a promising finding relating to attitude towards use of violence as we found that 84% of the individuals interviewed indicated that the use of violence should be avoided. Table 7: Baseline values of impact indicators between the treatment and control areas in matched sample Impact Indicator (range) Overall sample Control Treatment t￾statistics Household Resilience Capacity (0/110) 37.5 37.9 37.0 -0.31 High potential value chains (0/5) 1.93 1.89 1.96 0.26 Livelihood diversification index (0/15) 4.53 4.55 4.50 0.08 Social cohesion (outside) (0/16) 10.8 11.4 10.2 -1.57 Social cohesion(internal) (0/16) 12.6 13.2 12.0 -1.74* Local gov’t responsiveness (0/7) 4.70 4.70 4.70 -0.06 Violence avoidancea (0/1) 0.84 0.83 0.85 0.86 Note: aSample size for violence avoidance is 1680. Otherwise N=1788. * p < 0.10, ** p < 0.05, *** p < 0.01 Though this study is not designed or powered to detect significant changes within subgroups, the evaluation team also compared key indicators across the two territories and between households with a male versus female head to inform implementers’ understanding of differential needs and challenges for these subgroups (Table 8). Those living in Beni as well as female-headed households had slightly lower values than their counterparts across measurements of participation in high potential value chains, livelihood diversification, and social cohesion. However, those in Beni reported higher confidence in local government responsiveness, and female-headed households appeared to be more commonly in communities with higher responsiveness. Those living in Beni had a higher resilience capacity than those in Mambasa (40.5 versus 33.1). Female-headed households also had a slightly higher overall resilience capacity than male-headed households (39.2 versus 37.2). Table 8. Baseline values of impact indicators by territory and gender of household head, in matched sample 36 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Territory Gender Beni Mambasa Female Male Household Resilience Capacity (0/110) 40.5 33.1 39.3 37.2 High potential value chains (0/5) 1.62 2.37 1.52 1.99 Livelihood diversification index (0/15) 4.13 5.09 3.66 4.66 Social cohesion (outside) (0/16) 10.0 12.0 10.4 10.9 Social cohesion(internal) (0/16) 12.1 13.4 12.4 12.7 LG responsiveness (0/7) 4.95 4.34 4.81 4.68 N 1788 The final table of Annex X displays disaggregated factors that comprise the resilience index. Compared to Mambasa, households in Beni reported higher levels of cash savings, formal safety nets, humanitarian aid, and local government responsiveness in particular (Figure 7). This helps to explain the higher overall resilience measure for this territory. A large driver of the higher resilience capacity measure among female-headed households is the gender equitable decision-making component, which is, as expected, substantially higher in these households where a male is typically not present to take on any decision-making (Figure 8). Female-headed households also reported higher levels of formal safety nets and humanitarian aid and government responsiveness, though several other resilience components were lower than in male-headed households. Figure 7. Resilience index components, by territory (matched sample, N=1,788) 37 | Page Figure 8. Resilience index components, by gender of household head (matched sample, N=1,788) QUALITATIVE FINDINGS For analytical reasons, qualitative findings for the different context factors are presented separately. However, in our interviews with key informants in the nine villages, it became clear that context factors are often mutually dependent. The main cause of negative changes in context are internal and external violent conflicts. These conflicts lead to the forced displacement of the population, which is sometimes also a source of land disputes between returning inhabitants. Poverty and the loss of productive and non-productive household assets were often described as a result of theft, looting, and burning in the wake of violent conflicts in the area. A breakdown or weakening of social ties was seen as a consequence of people leaving their villages and moving to other villages in search of security and peace. The loss of access to fields for farming then leads to food insecurity, which leads to malnutrition, especially among children. Finally, conflicts were described as weakening the local economy due to security-related restrictions on the flow and movement of goods, illegal taxes, and the burning of vehicles for transporting goods. As described below, the analysis team found major variation in conflicts between the two territories, while the situation on the village level was remarkably similar for villages in the same territory. Due to the interrelated nature of conflict factors, most variation in context factors follows this pattern. INTERNAL AND EXTERNAL VIOLENT CONFLICTS The analysis identified four types of violent conflicts in the villages’ recent history: Attacks by external armed groups, attacks by local armed groups, mining conflicts, and land conflicts. The Allied Democratic Forces (ADF), an Islamist rebel group from Uganda, was identified as the main external armed group carrying out attacks on local villages. In some places, the ADF was identified as the National Army for the Liberation of Uganda (NALU), now one of its factions. The analysis team classified local defense groups including the Mai Mai as local armed groups. The analysis team differentiated between villages that experienced no such attacks, attacks with a low 38 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV intensity, and attacks with a high intensity. The difference between low and high intensity attacks was made based on the consequences for villagers. Low intensity attacks involved robbing and the temporary displacement of villagers. High intensity attacks involved the killing of a significant number of villagers, burning of houses and hospitals, or rape. Table 9Error! Reference source not found. provides an overview of the variation among the nine villages that were analyzed. Table 9. Variation of Conflicts Among Study Villages Territory Village External Armed Groups Local Armed Groups Land Conflict Mining Conflict Beni Mabakanga High Low None None Mulekera￾Ngongolio High High Some None Kalunguta High None None None Masyani High None Some None Kabasha High High Some None Mambasa Makalanga High None Some None Mirindi None None Some Some Maitatu Low None Some None Pucha Low Low None None The analysis also identified land conflicts and mining-related conflicts in the villages’ recent history. Among land conflicts, two types were differentiated: land conflicts between local land owners and conflicts involving protected areas such as the Virunga National Park in Beni and the Okapi Wildlife Reserve in Mambasa. With regard to land conflicts and mining-related conflicts, the analysis differentiated between cases where no such conflict was present or at least one such conflict took place (identified as “some” in Table 9able 9). Armed conflicts were more prominent in Beni. Here, all villages experienced indiscriminate violence by ADF/NALU affecting large numbers of households involving abductions, burning of houses, massacres, and rape. In Mambasa, only one village experienced such intense armed conflict involving external armed groups. A similar pattern can be seen with regard to attacks by local armed groups on villages. While some villages in Beni experienced such attacks, in Mambasa, no key informant reported any such attacks. A local leader in Beni gave an overview of the complex security situation in his village: “We have no access to the surrounding fields. The people who currently call themselves the UPLC and who make us feel insecure day and night are local children. From last year until now we’ve had insecurity because of the ADF and ULPC. They hold people to ransom and come and ask for food. When you don’t have anything to give, it becomes a problem, they imprison you straight away and refuse you access to your fields. Even if you avoid the path controlled by the ULPC, when you go to another field in the forest you will meet the ADF.” Unlike Beni, in Mambasa territory recurrent conflicts are mostly related to mining and land disputes. Mining conflicts involve the local community and foreign investors, particularly the Chinese. According to key informants in Mambasa, mining operations violate the rights of local communities to access the land for farming and artisanal mining. The statement of a local leader shows how negatively the role of Chinese investors is often assessed: "Chinese mining investors negotiate the percentages that they will share with the owner of the concession. For example, for 100 grams, the owner receives 20g and they take 80g because they are the ones who will bear the operating costs. If, on the other hand, the owner of a concession refuses to sell it or hand it over to the Chinese, the Chinese look for armed bandits to go and kill the owner; or if there are members of the same family and one does not agree to sell his concession while the others 39 | Page have agreed, they spread rumors so that the members of the family come into conflict or kill each other. [The Chinese] came to mine Mambasa's gold without caring about living conditions or the development of Mambasa.” In Mambasa, land conflicts sometimes emerged around the demarcation of groupement boundaries. In some cases, various initiatives to resolve such conflicts peacefully have been launched by non-state actors in collaboration with the local state authorities, but to date no lasting solution has been found. The people we met encouraged community dialogue initiatives and the work of community conflict resolution structures. POPULATION MOVEMENT A major consequence of conflicts is displacement. In Beni, villages often experienced population movements in both directions: Villagers had to regularly flee violence and villages experienced an influx of internally displaced persons (IDPs) from neighboring villages. Mambasa is experiencing displacement mostly in the form of an influx of IDPs, while the host population was often not forced to move themselves. For example, a respondent from Mambasa said that “there are many displaced people and the capacity of Mambasa to host them is becoming stretched.” FOOD SECURITY According to our informants, villages in Beni and Mambasa face increased food insecurity due to armed conflict and insufficient or poor harvests. Several respondents in Beni and Mambasa indicated that currently, many households in their villages consume less than three meals a day. Some respondents report that this results in severe acute malnutrition especially among children and pregnant and breast￾feeding women. The increased food insecurity has reportedly also increased the prevalence of diseases such as Kwashiorkor, a form of severe protein malnutrition. Agriculture being the main means of subsistence for the populations of Beni and Mambasa, armed conflict and the arrival of displaced people have led to an increase in food insecurity. Armed conflict contributes to a scarcity of agricultural products on the market because farmers lack access to fields and traders avoid roads that are insecure and impassable. This leads to increased prices for products on the market. Agriculture being the main means of subsistence for the populations of Beni and Mambasa, armed conflict and the arrival of displaced people have led to an increase in food insecurity. The arrival of internally displaced persons has exacerbated the shortage of foodstuffs. For example, a respondent in Mambasa stated that “access to food has been disrupted since the arrival of IDPs, and host families are unable to meet their own nutritional needs, including those of their children.” ECONOMIC SITUATION The main sources of household income reported in KIIs include farming, raising small livestock such as goats, cows and chickens, hunting and selling wild animals from Virunga National Park and the Okapi Wildlife Reserve, small-scale trading in various products, manual work such as cutting and sewing, making embers and sawing planks, working in public or private institutions, and exploiting and trading in mineral resources. The latter was reported exclusively in Mambasa. Farming is more often the only major source of income in Beni than in the Mambasa study areas. Farming has been adversely affected by the worsening security situation in both territories. Most of the areas once considered to be agricultural breadbaskets for both Mambasa and Beni have lost population due to displacement to areas where agriculture is not developed. As a result, people who were once active in agriculture have little or no access to their fields. As our key informants reported, this situation has had a significant impact on food prices on local markets. In Mambasa, mining activities are an additional constraint for local agriculture. Due to mining 40 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV concessions, our key informants report that there is now insufficient agricultural land available in Mambasa for farming activities. Some residents prefer mining to farming, which they see as more lucrative. According to some of those interviewed, however, artisanal mining is under threat from Chinese mining investors. These investors are said to be exploiting several mining concessions in or near the Okapi Wildlife Reserve, but they create few employment opportunities for local labor due to the use of machines. Some respondents describe these developments as a major contributor to unemployment among many local people and recruitment into armed groups. Insecurity also has a negative impact on small-scale trade and logging. In the case of small-scale trade, for example, the insecurity resulting from armed clashes hampers trade between the study areas and neighboring territories and countries such as Beni, Bunia, Butembo, Kisangani, Tshopo, and Uganda. Restrictions on the movement of the population and the looting and burning of vehicles for transporting goods add to the constraints. Several key informants claimed that this situation is at the root of food scarcity and rising prices of products on local markets. EXTERNAL SUPPORT According to our interviewees, vulnerable populations received various types of aid depending on their vulnerability. This aid was provided by (i) the Congolese government, (ii) NGOs and UN agencies, and (iii) philanthropic support. The types of humanitarian aid mentioned by respondents included food aid, essential household items, cash aid, and the construction of shelters for displaced people. In addition to this aid, there were developmental activities such as awareness-raising and training in the areas to promote social cohesion, good governance, environmental protection, and socio-economic reintegration. In Beni, the population mostly received humanitarian support from external organizations. There was also some government support mentioned. In Mambasa, external organizations conducted development work including training in agriculture and nutrition, and no government support was mentioned. CONCLUSIONS The objectives of this study are to identify suitable impact indicators, establish a valid counterfactual, and collect data to serve as a baseline for an impact evaluation of the P-DEC Project. We identified household resilience capacity index as the primary impact indicator, using the TANGO Light Approach to construct it. Secondary impact indicators include social cohesion, local governance responsiveness, livelihood diversification index, participation in high potential value chains, and attitudes towards the use of violence. We established a counterfactual group by matching households from the treatment area with households from similar control areas by using propensity score matching. This exercise successfully matched 894 treatment households to 894 control households. Based on the PSM diagnostics, we conclude that the matching in of the baseline data has generated a valid counterfactual for the evaluation of the P-DEC Project. To ensure that the counterfactual remains valid at endline, we need to make sure that we minimize attrition in both the treatment and control sub-samples. To reduce the risk of attrition and its effects, SoCha may consider implementing shorter annual surveys to monitor few indicators. The baseline study measured low household resilience capacity in the study area, at 37.5 index points out of the possible maximum of 110. Upon examination of the components of this resilience index, extremely weak economic factors are driving the low capacity. Cash savings, consumption assets, and livestock all rated less than one on a scale of 0 to 10. Social cohesion was relatively high within villages 41 | Page (12.6 out of 16), with slightly less cohesion between villages (10.8 of 16). Respondents rated local governance responsiveness at 4.36 (out of 7). Livelihood diversification is low, at 4.48 out of 15 possible index points. A relatively high proportion of individuals surveyed (84%) conveyed an attitude that violence should be avoided. The qualitative analysis showed different conditions for P-DEC interventions between Beni and Mambasa. While the study villages in Beni were the targets of intense attacks by external armed groups, in Mambasa communal conflicts such as land disputes and mining conflicts were much more prominent. Violent conflict is a major source of negative change in other context factors as well, especially regarding population movement. While villages in Beni reported experiences with both the displacement of the host population and the influx of IDPs, villages in Mambasa often only experienced a major influx of IDPs. From the point of view of the local economy, this situation is characterized by a drop in agricultural production, as the population no longer has access to fields, the lack of foodstuffs and an increase in food prices, a reduction in trade flows with other regions due to insecurity, an increase in unemployment, and the closing of markets. Therefore, the conditions in Beni are overall much more challenging for P-DEC than in Mambasa. P-DEC’s approach focuses on communal conflict, capacity building, and value chain development. This approach requires a level of stability in target villages. For example, to successfully develop a value chain, ongoing agricultural activity, and open markets are important prerequisites. Moreover, in an active conflict zone, communal conflicts may be of less importance for peacebuilding than addressing the threat of external armed groups. RECOMMENDATIONS As a baseline report, the evaluation team’s ability to make recommendations is naturally limited. Complete recommendations relevant to the evaluation questions will be developed after follow-up data collection. However, at this early stage, the evaluation team has identified some issues that USAID and P-DEC implementers should take into consideration as the project moves forward. 1. Qualitative data indicate substantial contextual challenges that are likely to affect implementation plans, particularly in Beni. USAID and P-DEC should reflect on what is necessary for integrated programs like P-DEC to adapt to changing contexts such as acute crisis in Beni driven by attacks by external armed groups. Partners should establish a robust adaptive management plan and consider what sorts of flexible approaches can be built into the design of resilience activities to bolster success. 2. Implementers should document key contextual factors (e.g. security shifts, shocks) and implementation adaptations (e.g. changes in locations that receive benefits, timing, approaches used) along the way so that the evaluation team can consider implications for the evaluation. For example, in addition to the main “intention to treat” analysis among all originally targeted communities, the endline might also employ a sub-analysis that removes communities not reached as intended, to test the treatment effect among those that actually received treatment. Such analysis requires clear documentation of what activities occurred where. 42 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV WORKS CITED Austin, P. (2009). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. STatistics in Medicine, 28, 3083–3107. Caliendo, M., & Kopeinig, S. (2008). Some Practical Guidance for the Implementation of Propensity Score Matching. Journal of Economic Surveys, 31 - 72. Constas, M., d’Errico, M., & Garbero, A. (2019). Constas, M., M. d’Errico, and A. Garbero. 2019. Quantitative analyses for resilience measurement. Guidance for constructing variables and exploring relationships among variables: Resilience measurement technical working group. Rome: Food and Agriculture Organization of the United Nations. Dehejia, R., & Wahba, S. (2002). Propensity Score Matching Methods for Non-Experimental Casual Studies. Review of Economics and Statistics, 84(1), 151 -161. FAO. (2016). Resilience Index Measurement and Analysis – II. Rome. Fredriksson, A., & Oliveira, G. (2019). Impact evaluation using Difference-in-Differences. RAUSP Management Journal, 54(4), 519 -532. doi:https://doi.org/10.1108/RAUSP-05-2019-0112 Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. (2016). Impact Evaluation in Practice (2nd ed.). Washington, D.C.: The World Bank. Harris, H., & Horst, S. (2016). A Brief Guide to Decisions at Each Step of the Propensity Score. Practical Assessment, Research, and Evaluation, 21(4). Mercy Corps. (2021). Partnership for the Development of Eastern Congo (P-DEC). Portland, USA: Mercy Corps. Smith, L., Brown, D., Hedley, D., & Frankenberger, T. (2022). Resilience in the Sahel-Enhanced (RISE) Initiative Impact Evaluation. Washington, D.C.: Resilience, Evaluation, Analysis and Learning (REAL) Associate Award. Stuart, E., Huskamp, H., Duckworth, K., Simmons, J., Song, Z., Chernew, M., & Barry, C. (2014). Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Services and Outcomes Research Methodology, 166 -182. TANGO International. (2018). Methodological Guide:A Guide for Calculating Resilience Capacity. Washington, D.C.: TANGO International. USAID. (2022). 2022 Resilience Policy Revision. USAID. 43 | Page ANNEX I: TIMELINE The impact evaluation takes on the following timeline: Stage Evaluation stage Date 1 Baseline data collection April-May 2023 2 Preliminary results presentation June 2023 3 Draft baseline report shared with USAID August 2023 4 Final baseline report completed August 2023 5 Preparation for endline data collection March 2026 6 Endline data collection April-May 2026 7 Preliminary results presentation June 2026 8 Draft baseline report shared with USAID July 2026 9 Final baseline report completed August 2026 10 Final results dissemination activities Aug.-Sept. 2026 44 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX II: EVALUATION STATEMENT OF WORK ACTIVITY OVERVIEW The eastern part of the Democratic Republic of Congo (DRC) is a region with one of the most complex humanitarian/security situations in the world. It is home to more than 100 armed groups, that are responsible for mass executions, kidnappings, looting of property, illegal taxation, and other abuses against civilian populations.4 Many of the armed groups operating in the region do so with impunity. They attract members, often not for ideological reasons, but for basic security and livelihoods in an area with minimal economic development and access to resources. These armed groups further fuel encroachment and illicit trade of natural resources (animal poaching, charcoal, minerals, and fishing) within the national parks in the region.5 The crisis in the eastern DRC is rooted in conflicts around power, governance, illicit trade in resources such as minerals, fish and charcoal, and a general absence of economic opportunities and essential services for the population.6 Over the past decades, the U.S. Government, through USAID, has been assisting the Government of the DRC (GDRC) to address the causes of conflicts in the eastern DRC.7 USAID/DRC has been supporting community-led efforts to manage, mitigate, and prevent conflict. USAID’s activities empower local communities and civil society organizations to engage with their elected officials and other leaders to reduce violence. USAID has also supported communities conduct conflict analyses and use the results to influence decision-makers.8 The Partnership for the Development of Eastern Congo (P-DEC) program is a five-year USAID￾funded project that implements an integrated set of cross-sectoral interventions to address the development needs of communities in the eastern DRC. The project seeks to build community trust, strengthen the resilience of individuals and communities, fortify existing governance structures, and create opportunities for long-term development and the self-reliance of local communities.9 P-DEC adopts bottom-up peacebuilding approaches to reduce conflict and bridge social divisions. Through a cohesive and holistic approach, the P-DEC Project aims to amplify the impact of USAID investments and complement other planned interventions by the GDRC and external actors in the public and private sectors. USAID recognizes that there are no proven solutions for the security and development challenges in the eastern DRC. Therefore, innovation, exploration, and experimentation are important aspects of the P-DEC’s program.5,6 P-DEC is aligned with the GDRC’s national strategy for the implementation of the Demobilization, Disarmament, Community Recovery and Stabilization Program (P-DDRCS) for ex-combatants which 4 Mercy Corps/DRC 2021 Annual Work Plan-Partnership for the Development of Eastern Congo (P-DEC) Program 5 United Nations (2022). Combating Illicit Trade in Natural Resources That Fuels Conflict in the DRC. Available at: https://press.un.org/en/2022/sc15078.doc.htm 6 IPIS 2021. Program for the Development of Eastern Congo (P-DEC). Available at: https://ipisresearch.be/project/program-for-the￾development-of-eastern-congo-p-dec/ 7 Duke University (2020). Partnership for the Development of Eastern Congo https://researchfunding.duke.edu/partnership-development￾eastern-congo 8 USAID 2016. Peace and Security. Available at: https://2017-2020.usaid.gov/democratic-republic-congo/fact-sheets/peace-and-security 9 USAID/DRC. Request for Information No. 72066020RFI00005. Partnership for the Development of Eastern Congo (P-DEC) Program. Available at: https://cd.usembassy.gov/wp-content/uploads/sites/160/Request-for-Information-No.-72066020RFI00005-P-DEC.pdf 45 | Page is the government’s strategy to promote peace and security in the eastern part of the country.10 As a result, P-DEC’s strategy is integrated, agile, multi-faceted, socially inclusive and conflict sensitive, to address the root causes of conflict and crisis and ensure resilient, economically and environmentally sustainable outcomes through long-term systemic approaches. P-DEC is implemented by a consortium of 12 international and national organizations. These include: Mercy Corps (prime implementing partner and leader of the consortium), International Alert, Wildlife Conservation Society (WCS), Alliance for Responsible Mining (ARM), International Peace Information Service (IPIS), HIVE, Association for the Promotion of Hygiene and the Integral Development of the Vulnerable (APROHDIV), Justice Plus ,Women’s Solidarity for Peace and Development (SOFEPADI), North Kivu Community Radio and TV Collective (CORACON), Fair Congo Foundation and Pole Institute. The program is implemented across several communities in the territories of Beni (North Kivu province) and Mambassa (Ituri province).11 Throughout the life of the P-DEC Project, various groups in these project communities will be engaged and consulted to propose and develop robust mechanisms to promote meaningful social inclusion, ensuring that the voice of marginalized individuals and groups are involved in all peace building and conflict resolution efforts.6 P-DEC will prioritize youth, engaging them in civic economic and social activities to steer them away from armed groups and associated illicit activities. Three intermediate results will contribute to P-DEC’s overall goal to strengthen the foundations for durable peace in North Kivu and Ituri: • IR1: Communities are resilient to conflicts and crises; • IR2: Institutions are more accountable, trusted, and citizen-responsive; • IR3: Inclusive and diversified economic growth improves stability P-DEC’s Theory of Change (Figure 5) posits that “If communities mobilize to prevent and resolve conflicts peacefully; citizens and government institutions improve accountability and effectiveness of service delivery; and economic growth promotes inclusion and reduces drivers of conflict and support for armed groups; and if these efforts leverage the unique capabilities of PA anchor institutions; then the foundations for durable peace will be strengthened in eastern Congo”. 10 United Nations (2022). DRC: MONUSCO Supports the New Demobilization Program for Ex-combatants. Available at: https://peacekeeping.un.org/en/drc-monusco-supports-new-demobilization-program-ex-combatants 11 ARM Partnership for the Development of Eastern Congo – Tujenge. Available at : https://www.responsiblemines.org/en/project/partnership-for-the-development-of-eastern-congo￾tujenge/#:~:text=The%20Eastern%20Congo%20Development%20Partnership,on%20areas%20bordering%20the%20Okapi 46 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Figure 5 - P-DEC’s Theory of Change PURPOSE AND INTENDED USERS OF THE IMPACT EVALUATION BASELINE FINDINGS The baseline will provide benchmarks values for key indicators before P-DEC’s interventions. MSSP anticipates issuing a future solicitation to provide an endline measurement of the same indicators after P-DEC has completed interventions. MSSP will quantify the impact of the P-DEC interventions using statistical analysis of the difference between baseline and endline measurements in matching groups that received, or did not receive interventions from P-DEC. The baseline data will provide descriptive statistics on the population of the areas where P-DEC plans to implement programming. These data will be useful for the following key stakeholders: USAID/DRC Peace and Security Office, Mercy Corps and consortium partners, the government of the DRC, and other international donors. The findings will provide a point of reference against which the impact of P-DEC interventions will be evaluated at the endline. TASKS The impact evaluation will be managed by SoCha, an international development company based in the United States that is contracted to provide technical and advisory services to the USAID/DRC Mission through the 5-year Mission Strategic Support Program (MSSP). MSSP works to strengthen the Mission’s strategic monitoring, evaluation, and learning (MEL) systems; manage high quality data; provide geo-intelligence; produce rigorous analyses that inform decision making; institutionalize collaborating, learning, and adapting (CLA) practices; expand localization through targeted and deliberate capacity building; generate action-oriented data visualizations and communications; and provide Mission personnel with third-party monitoring in areas they cannot access. Working under the supervision of MSSP, the evaluation team will have demonstrated experience in designing and implementing mixed-methods impact evaluations using a quasi-experimental design based 47 | Page on the difference-in-difference (DID) approach. The evaluation team will review existing program documents and conduct an analysis of P-DEC’s theory of change (that considers the causal links between inputs, activities, outcomes, and impacts) of the P-DEC Project. The following key tasks will be accomplished during the assignment: DEVELOP AN INCEPTION REPORT FOR THE BASELINE IMPACT EVALUATION The evaluation team shall develop an inception report which must include a detailed methodology based on a quasi-experimental design and difference-in-difference (DID) approach. The inception report must include the evaluation design; selection of treatment and comparison communities (i.e., the counterfactual); key outcomes to be measured; sampling design (sampling frame, sample size and power calculations, sampling strategy for households, adjustment for survey non-response, etc); procedure for household listing; data collection tools (questionnaires and interview guides); data collection procedure; workplan, staffing plan, and schedule of tasks; roles and responsibilities of key personnel; quality assurance arrangements; potential limitations, risks and mitigation measures etc. MSSP will provide relevant documents and data to guide the evaluation team to develop the inception report which will be finalized in collaboration with MSSP during the design phase of the contract. DEVELOP DATA COLLECTION TOOLS (HOUSEHOLD QUESTIONNAIRE AND INTERVIEW GUIDE) During the design phase, the evaluation team shall develop the household survey questionnaire that will be used to collect quantitative data. The questionnaire will be based on existing instruments developed by USAID for measuring resilience. The questionnaire will be divided into sections with well formulated questions that will gather data to measure key indicators. The questions must be in simple language including phrasing of questions and adaptation of response codes, so that they are appropriate to the context and the P-DEC activity. The questions should be free from ambiguity and the evaluation team should avoid formulating double-barreled and leading questions. Clear instructions with skip patterns should be incorporated in the questionnaire. In addition, the evaluation team will develop a semi-structured interview guide that will be used to conduct key informant interviews across selected communities. The data collection tools (questionnaires and interview guides) should first be developed in French and in English. The evaluation team must share the French and English versions of these tools to MSSP for review and validation. Upon validation, the evaluation team shall translate the French version of these data collection tools into the local languages spoken in the communities where data will be collected. To ensure the accuracy of the translation, the data collection tools in the local language should be back translated into French (preferably by a translator). The back-translated version (from the local language) should be compared with the content of the original versions to determine whether the translation clearly conveyed the content of the original data collection tools. The evaluation team will develop a code book that will include codes for location, respondent type, questions, and response categories. Prospective evaluation teams should propose any additional translations they feel are necessary and describe in detail their process for achieving accurate translations of the questionnaire into all languages. SCRIPT THE QUESTIONNAIRE The Evaluation team shall script the questionnaire for administration using a suitable software application. Vendors should propose the best software application for administering the survey and 48 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV justify why the application was selected based on the advantages of using the selected application compared to alternatives. The evaluation team must not change the approved script unless authorized by MSSP in advance and in writing. DEVELOP A STAFFING PLAN The evaluation team will hire enumerators, interviewers and field supervisors who should be individuals with previous experience collecting household survey data. The hiring of qualified local enumerators who can speak and understand the languages spoken in the targeted communities should be prioritized. These enumerators must be able to interact with individuals in the various communities, establish and build rapport with the community members. Overall, all individuals mobilized by the evaluation team must have the minimum required skills to support the collection of high-quality data. Vendor proposals should provide a detailed explanation of their staffing plan and how they will identify and evaluate the suitability of potential field staff, including local language fluency, education credentials, and required skills. DEVELOP FIELD (ENUMERATOR’S) MANUAL The evaluation team shall develop a field (enumerator’s) manual. This manual will provide instructions on how to carry out enumeration of individuals and households in the communities. This manual will be used during the training and will provide guidance on the objectives of the baseline evaluation, confidentiality, interviewing techniques, instructions on filling out the questionnaires in the tablets, coding and skip patterns, etc. It must also include guidance to conduct the key informant interviews. The final version of the manual must be approved by MSSP. The evaluation team should also develop a guide for the conduct of semi-structured interviews to ensure that interviewers follow this guide to conduct the key informant interviews and collect high quality qualitative data. TRAIN FIELD STAFF The evaluation team shall develop training modules, a training agenda and train all the enumerators, interviewers and field monitors recruited. MSSP will provide support to the evaluation team during the training, which should be organized for at least a week. The training must ensure that enumerators, interviewers, and field supervisors understand the activity being evaluated, why the data is being collected, the questions to be asked, how to recruit respondents, how to administer the questionnaire using the questionnaire scripts and how to conduct the semi-structured interviews. The evaluation team should train a few extra individuals (at least 10% more than the number of enumerators/interviewers) needed to make up for any absentee or nonperformance during the period of data collection. However, the final number of individuals to be trained is at the discretion of the evaluation team based on logistical considerations. PILOT-TEST DATA COLLECTION TOOLS After the training, the evaluation team shall deploy the enumerators, interviewers and field supervisors to pilot the questionnaire programmed in the tablets. Pilot-testing should take place in Goma across 30 households. Trained interviewers will also pilot the interview guide by conducting at least 12 interviews with key informants. During the process, the field team should also pilot the strategies that will be used to recruit potential respondents and key informants during the actual data collection. The Evaluation team must deliver a pretest report with an item-by-item accounting of the results of the pretest for all versions of the questionnaire. 49 | Page DISSEMINATE PILOT-TESTING RESULTS AND FINALIZE THE TOOLS After pilot-testing, a feedback session should be organized, and the data collection tools (questionnaire and interview guide) should be checked for any errors. The evaluation team must consider any issues that emerged during the pilot-testing to revise and finalize the data collection tools. The Evaluation team must modify the scripted questionnaires and its translated versions to incorporate any changes following the pilot test. Vendors should describe their approach to pilot￾testing and how this approach will enhance the quality of the survey data. Actual data collection shall only commence when all aspects of the impact evaluation design, including the data collection tools, have been tested with the satisfaction of MSSP. DEVELOP AN IMPLEMENTATION PLAN The Evaluation team must create a detailed implementation plan for achieving the survey sample design. The implementation plan must include all equipment, logistical and staffing arrangements, including a detailed assessment of the feasibility of administering the survey to all selected sampling units. The implementation plan must provide details on when and where interviews will take place and which data collectors will administer which interviews. The implementation plan must provide detailed specifications of the Evaluation team’s digital technology, including software and devices. The Evaluation team must submit a security plan that assesses risks and risk mitigation for each sampling unit. The Evaluation team must submit all requested changes to the implementation plan, for example changes to timelines or selected sampling units, to MSSP for approval in advance and in writing before implementing any changes. The Evaluation team must submit weekly reports on the implementation of the survey that show actual achievements compared to the implementation plan. Vendor proposals should provide a detailed description of their implementation plan, identify risk factors that may affect implementation, and describe how such risk factors will be mitigated. IMPLEMENT DATA COLLECTION The evaluation team will deploy highly trained enumerators, interviewers and field supervisors to select treatment and comparison communities to collect both quantitative (household survey) and qualitative data (key informant interviews). Data will be collected from communes in Beni and Mambassa territories (treatment communities) and communes in Butembo and Mahagi territories (communities). Nevertheless, during the design stage the evaluation team shall work with MSSP and P-DEC partners to identify targeted communes where data will be collected. Data collection will take place in each commune based on a strategy that will be agreed between the evaluation team and MSSP. Data should be collected from at least 2,700 households (1,350 households in treatment communes and 1,350 households in comparison communes) across 150 communes (75 communes for each treatment arm). Households may be randomly chosen with equal allocation per commune. Overall, the final sample and approach to enumerate households will be communicated to the evaluation team during the inception phase of the assignment. Furthermore, an estimated 78 key informant interviews will be conducted in a subset of treatment and comparison communities. The evaluation team will be responsible for contacting local officials and community leaders, obtaining permissions, communicating the purpose of the baseline survey, and obtaining community consent. Enumerators should use the tablets provided to collect household data while interviewers will use the validated interview guide conduct interviews with key informants (P-DEC team and local 50 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV leaders) to measure context variables and local variation in interventions. The data collection approach must provide MSSP with the possibility to monitor data collection in real-time. QUALITY ASSURANCE AND CONTROL The Evaluation team must implement an effective quality assurance (QA) and quality control (QC) plan to ensure the quality of all deliverables. The Evaluation team must deliver a QA/QC plan that includes specific details on the QA/QC measures that will be taken throughout the survey and during data collection, such as supervisor accompaniment, field reporting structures, and feedback and mechanisms that will allow for timely corrections of any problems to minimize any impact on the survey data. The Evaluation team must provide all information required for an independent quality control team separately managed by SoCha to monitor data collection. Vendor proposals should describe their QA/QC plan. PROCESS AND FINALIZE THE DATASET The evaluation team must deliver a clean, finalized data set in SPSS and CSV formats. To assist the Evaluation team to produce the dataset, MSSP will provide the evaluation team with a codebook in Excel format that includes all questionnaire items. The evaluation team is required to perform data checks that include, but are not limited to, the following: • Reviewing the data file to ensure it is structurally sound and consistent with the codebook, such that all variables included in the codebook are present in the dataset, are the right type, and contain matching values, including administrative variables and metadata • Identifying missing data that must be captured through follow-up interviewing • Checks for anomalies in the data, which, among others, may include the following: o Eligibility of the respondent o Values are within each variable’s defined range o Outlier interview durations and daily number of interviews per interviewer o Overlapping interview times conducted by the same interviewer o Verify metadata on the location, time, interviewer, etc. matches the sampling plan o GPS coordinates of surveyed areas are within the country where the survey was conducted o Duplicate GPS coordinates for households o Interviews that are identical or respondents who have been interviewed twice (duplicate cases) o Cases that have the same answers across a series of questions regardless of the interviewer (pattern response) o Interviewers that had the same responses for particular questions across all of his/her interviews (substantive response bias) The production of the dataset will include several rounds of data checking and cleaning in close consultation with MSSP to arrive at the final dataset that passes all quality checks. After checking the data, MSSP will send the evaluation team data check reports that identify errors and inconsistencies item-by-item in the dataset. The evaluation team must promptly address all concerns raised by MSSP in the data checking process and complete the timely revision of the dataset. Data checking will continue until the evaluation team has satisfactorily resolved all errors and inconsistencies. The evaluation team should check for the accuracy of all transcription of qualitative data to ensure that they are of high quality by reading through the transcripts and comparing with the audio-recordings. Technical proposals should describe the firm’s (or teams) experience and expertise with data processing and cleaning and how data quality will be controlled. 51 | Page PERFORM DATA ANALYSIS After the approval of the clean dataset, the evaluation team may need to weight the data for it to be representative prior to analysis. The evaluation team should use any quantitative data analysis software (e.g., R, SPSS or STATA) to analyze household data. Quantitative data for the outcome measures should be clearly presented in tabular format. The evaluation team must provide intelligent verbatim transcripts of all key-informant interviews in English or French.12 The evaluation team will use any qualitative software (e.g., MAXQDA or NVivo) to analyze qualitative data. DEVELOP A DRAFT BASELINE IMPACT EVALUATION REPORT After data analysis, the Team Leader (Impact Evaluation Specialist) proposed by the evaluation team should develop the draft baseline impact evaluation report. The evaluation team shall submit the draft report to MSSP for review and comments. The report should describe the methodology of the impact evaluation and summarize the key findings of the baseline impact evaluation. MSSP shall provide the evaluation team with a template for the report. The draft baseline report will be reviewed by MSSP to ensure that it covers the scope and meets the required quality criteria. PRESENT PRELIMINARY FINDINGS The evaluation team will present the findings of the baseline survey and key informant interviews to MSSP. This will be an opportunity for the evaluation team to discuss the findings and address any factual errors or misunderstandings, inputs and feedback prior to finalizing the baseline impact evaluation report. SUBMIT FINAL BASELINE IMPACT EVALUATION REPORT The evaluation team will finalize the baseline impact evaluation report based on comments and feedback provided by MSSP. The evaluation team must address all the comments provided by MSSP and take into consideration paragraphing, grammar, writing style and formatting when finalizing the report. At this stage, the evaluation team shall submit two versions of the report-the tracked changes (showing the changes that have been made in the draft report based on comments) and the clean version. The report must be developed in a structured format that will be shared with the evaluation team. EXPECTED DELIVERABLES The expected deliverables for this assignment and submission dates are summarized below: Deliverables Delivery date Inception Report (including the evaluation design, treatment and comparison communities, sampling approach, sample size, field procedures, data collection, data analysis, quality assurance, workplan) etc. February 28, 2023 12 https://summalinguae.com/data/understanding-intelligent-verbatim-transcription/ 52 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Final data collection tools (household questionnaires and interview guide) in French and/or local languages February 28, 2023 Staffing plan (List of qualified enumerators, interviewers and field supervisors and repartition plan) February 28, 2023 Implementation plan February 28, 2023 Field (enumerator’s) manual in French March 5, 2023 Training modules March 5, 2023 Training report March 5, 2023 Pilot-testing report in French March 17, 2023 PowerPoint presentation of the results from the pilot-test March 17, 2023 Field data collection report in French or English April 17, 2023 Clean survey dataset and Data files: transcripts and audio-recordings of key informant interviews April 17, 2023 Draft baseline impact evaluation report in English April 21, 2023 Power point presentation (in English) summarizing the findings of the baseline impact evaluation April 25, 2023 Final baseline impact evaluation report in English May 10, 2023 CONSULTANCY REQUIREMENTS The evaluation team is expected to meet the following minimum requirements: ● Legally registered in the DRC to enable the firm to perform the above-mentioned tasks; ● Experience designing and conducting community-based household surveys in North Kivu and Ituri provinces. ● Have staff with demonstrated knowledge, skills, and experience in conducting impact evaluations of development programs; ● Demonstrated ability to organize field logistics and implement quality assurance protocols for field data collection; ● Able to mobilize a team with individuals having technical skills in qualitative and quantitative data collection; ● Able to mobilize a team that can communicate in French and the four national languages (Kikongo, Lingala, Swahili, and Tshiluba) TEAM COMPOSITION AND REQUIRED COMPETENCIES The evaluation team must propose experts with proven experience and expertise in designing mixed￾methods impact evaluations using quasi-experimental designs. Given the complexity of this baseline impact evaluation, MSSP expects that the key personnel for this assignment must include the following team members and all key personnel must be based in the DRC to undertake this assignment. CVs 53 | Page of all key personnel must be submitted as an annex in the technical proposal. Below is a summary of the required qualifications and experience of the key personnel: Impact Evaluation Specialist (Team Leader): required skills and experience • Advanced degree (Masters/PhD) in Public Policy, Economics, Development Studies, Comparative Research or any other relevant university degree • Minimum 7 years’ experience designing impact evaluations in the international development sector using quasi-experimental methods (e.g., double-difference techniques, regression discontinuity design, propensity score matching etc.) • At least 5 years’ experience leading household surveys and expertise in statistical analyses (familiarity with data analysis software, R, SPSS and STATA highly desirable); • Experience implementing impact evaluation in the peace and international development sector • Fluent in English and French Impact Evaluation Coordinator: required skills and experience • Master’s Degree or equivalent in Social Sciences, Statistics, Development Studies or another related technical field • At least 5 years of working experience implementing and managing community-based household surveys and qualitative data collection • Excellent organizational skills and ability to prioritize and manage multiple tasks. • Native or near-native proficiency in English (oral and written) is required. • Strong writing and interpersonal communication skills. Statistician: required skills and experience • Advanced University degree in Statistics, Economics, or a related field. • At least 7 years’ experience as a senior statistician and/or econometrician with extensive experience in survey sample design, power calculations and statistical analysis. • The candidate should be fluent in French and/or English SECURITY CONSIDERATIONS The evaluation team should be able to demonstrate a solid understanding of the realities on the ground and in managing the complex security environment in the eastern DRC. The evaluation team should have a clear strategy to deploy enumerators, interviewers and field supervisors in both remote and insecure communities in a manner that minimizes the risk to these individuals. The evaluation team must carefully analyze the security environment of each community and sampling unit to be visited and respect all travel advisories and restrictions imposed by the government or other local security agencies prior to deploying teams to the field. The evaluation team should include a security plan for enumerators, interviewers and field supervisors as an annex to their technical proposal. 54 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX III: DATA COLLECTION AND ANALYSIS TOOLS HOUSEHOLD SURVEY QUESTIONNAIRE Q code. PRESENTATION, EXPLANATION OF THE SURVEY AND PRIOR CONSENT Survey description for all respondents: This survey has been commissioned by SoCha/MSSP at the request of USAID as part of the impact evaluation of the interventions undertaken by some of its partners in the DRC. We are conducting this survey at the start of the intervention, and will return later to conduct a second survey. By comparing the information collected by the two surveys, we will determine what changes were caused by the intervention. We are implementing the survey in areas where USAID has interventions and where USAID does not have interventions to be able to compare the differences between the two areas. We would like to interview some members of your household now before the interventions are rolled out and later after the interventions to know what changed over time. All together the interviews will take about 1 hour. I guarantee you that your personal data will not be identified in the results. The information provided in this survey will be handled as confidential and anonymous. You are free to participate in this survey and withdraw at any moment. You are also free not to answer any question and even to stop the survey at any time if you feel uncomfortable or for any other reason. Your participation or non-participation has no negative or positive implications for you. However, it will contribute to USAID programming in your area. Q1 Household contact: Hello, my name is _____. I am from an independent research organization called IES. We are conducting a survey. I would like to interview the head of the household and one other person who lives here. Is the head of household available? 0. No 1. Yes Q2 [IF NO TO Q1] Will the household head be available before 4:00pm for me to come back? 0. No 1. Yes Q3 [IF NO TO Q2] Is there anyone who is aged 18 years or more who can respond to the questions on behalf of the household head? If no to this question, end the interview and go to the next household. 0. No 1. Yes Q4 Do you consent to participate in the survey? If no to this question, end the interview and go to the next household. 0. No 1. Yes Q5 [IF YES TO Q4] Name of Respondent Q6 Age of Respondent (Years) Q7 Sex of Respondent 0. Female 1. Male 55 | Page Q8 Number of people in the household Q9 HOUSEHOLD ROSTER I also need to select one person who lives in this household for an interview. Please list the first name of each household member, their age, and their sex. Please start with the head of the household. If the primary female decision maker is not the head of the household, please tell me her name next. After that, please list the other household members. (A household member is someone characterized by two of the three following conditions: (1) shares the same food pot, (2) shares the same roof, or (3) has a common decision maker). Program the tablet to randomly select one household member age 18 or older. The head of the household or the primary female decision-maker may be selected again. Name: 1) 2) 3) 4) 5) 6) 7) 8) ….n Age: 1) 2) 3) 4) 5) 6) 7) 8) ….n Sex: 1) 2) 3) 4) 5) 6) 7) 8) ….n Head of household? 1) Y/N 2) Y/N Primary female decision maker? 1) Y/N 2) Y/N MODULE A: ADMINISTRATIVE INFORMATION A1 Names of the interviewer (LASTNAME, POSTNAME AND FIRST NAME) PRECODE A2 Household GPS coordinates (Do not proceed until GPS coordinates are captured at the dwelling) A3 Territory 1. BENI 2. MAMBASA A4 Survey area 0. Non-P-DEC 1. P-DEC A5 Group/ Commune 1. MULEKERA 2. RWENZORI 3. BULIKI 4. OICHA 5. MANGINA 6. MUTWANGA 7. KYONDO 8. NYANGWE 1. ANDIBUTA 2. BAFWHIYO 3. BAPONGOMO 4. ANDIKAU 5. BABEKE OF ISAYE 6. MPUTU 7. BABOMBI 8. TETURI A6 Survey village 1. Mabakuha 2. Bucha 3. Biakato 4. Katala 5. Malutu 6. Kanya 7. Mabanzaku 8. Bandisende 9. Funga Mukaba 10. Bavasea 11. Komboni 12. Memekidele 13. Mamulebu 14. Kanzambu 15. Tokoleko 41. Makalanga 42. Nyangwe 43. Makoko 1 44. Kero Zanzibar 45. Mirindi 46. Parana 47. Tobola 2 48. Bapongomo 49. Bandikambwa 50. Bafwakoa 51. Bafwazobange 52. Bafwanangala 53. Basanjasili 54. Bafwakedu 55. Niania 56 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV 16. Kasithu 17. Mangodomu 18. Masimbembe 19. Linzo 20. Mangina 21. Home 4 22. Kavanda 23. Kyomole 24. Mulakirwa 25. Vahyana 26. Sivirwa 27. Kaviranga 28. Nzenga 29. Thalihya 30. Bukokoma 31. Mapou 32. Mutsora 33. Mbimbi 34. Pakanza 35. Tenambo 36. Masosi 37. Nzanza 38. Mabasele 39. Oicha 1er 40. Bakaiku 56. Maitatu 57. Pucha 58. Ekwe 59. Muchacha 60. Kasabinyole 61. Ngadi 62. Nzuma 63. Paida 64. Mabakanga 65. Boikene 66. Kabasha/Kazebere 67. Pabuka 68. Supa-Kalau 69. Lisasa 70. Mumbe/Kalunguta 71. Tamende 72. Saiyo 73. Butsili 74. Kasangaruha 75. Ngongolio 76. Masyani 77. Matonge 78. Matembo 79. Kalinda 80. Bunzi HEAD OF HOUSEHOLD INTERVIEW Interviewer: The head of the household must answer the following items. A7 READ SURVEY DESCRIPTION Do you consent to participate in the interview now and in the future? 0. No 1. Yes A8 HH Interview Date (Automated) A9 HH Interview time Hours and Minutes (automated) A10 Name of head of household (LASTNAME, POSTNAME AND FIRST NAME) Lastname Postname First name A11 How long have you lived in this village/neighbourhood? (Record years and months) Years___ Months ___ MODULE B: HOUSEHOLD SOCIODEMOGRAPHIC CARACTERISTICS B1 Is the household a pygmy ethnic group member (observation) 0. No 1. Yes B2 To which religion do most members of this household belong? 1. Islam 2. Christian 3. Traditional 4. None 5. No response B3 Sex of household head 0. Male 1. Female B4 Age of household head in years (number) Years 57 | Page B5 Marital status of household head 1.Single 2. Mar married – monogamy 3. Married- polygamy. 4. Divorced/Separated 5. Widowed 6. No response MODULE C: EDUCATION/TRAINING Now I will ask you some questions about education and training. C1 Do any household adults have a primary school or higher education? 0. No 1. Yes C2 Is there an adult in your household who can read or write any language? 0. No 1. Yes C3 Have you or anyone in your household ever received any vocational (job) or skill training? 0. No 1. Yes -8 Don’t know -9 Refused C4 Have you or anyone in your household ever received any business development training (including financial literacy)? 0. No 1. Yes -8 Don’t know -9 Refused C5 Have you or anyone in your household ever received any conflict management or conflict presention training? 0. No 1. Yes -8 Don’t know -9 Refused C6 Have you or anyone in your household ever received any natural resource management training, e.g. management of forests, minerals, agroecological production, etc? 0. No 1. Yes -8 Don’t know -9 Refused C7 Have you or anyone in your household ever received adult education? 0. No 1. Yes -8 Don’t know -9 Refused C8 Have you or anyone in your household ever received training in how to use your mobile phone to get market information like prices? 0. No 1. Yes -8 Don’t know -9 Refused MODULE D: HOUSEHOLD ASSETS Now I will ask you about the possessions and assets owned by your household. D1 For each type of asset that I mention, please tell me the number of items owned by the household. Indicate zero if the household does not own the item or -9 if the respondent refuses D1A Consumption Assets Bed Number D1B Sofa set Number D1C Charcoal stove Number D1D Dining set Number D1E Chairs (not part of dining set) Number D1F Tables (not part of dining set) Number D1G Mobile phone Number 58 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV D1H Radio/Tape Player Number D1I Television Number D1J Jewelry- gold, silver, wristwatches Number D1K Bicycle Number D1L Cart (animal drawn) Number D1M Passenger car or truck Number D1N Motor bike (2 or 3 wheels) Number D1O Generator Number D1P Solar lamp Number D1Q Other (Please specify) Number D2A Productive Assets Plough (oxen-pulled) Number D2B Mechanical plough Number D2C Sickle Number D2D Axe Number D2E Pruning/cutting shears Number D2F Hoe Number D2G Spade or shovel Number D2H Beehive Number D2I Knapsack chemical sprayer Number D2J Mechanical water pump Number D2K Motorized water pump Number D2L Grain mill Number D2M Small tractor Number D2N Hand-held motorized tiller Number D2O Agricultural land (hectares) Number D2P Chainsaw (tronçonneuse) Number D2Q Sewing Machine Number D2R Welding Machine Number D2S Others (Please specify) Number D2T Others (Please specify) Number D3A Livestock Assets Oxen Number D3B Goats Number D3C Sheep Number D3D Pigs Number D3E Poultry Number D3F Rabbits Number D3G Fishpond Number D3H Others (Please specify) Number MODULE E: SOCIAL CAPITAL I will now ask you a few questions to assess the relations between your household and those residing inside and outside your village. E1 If your household had a problem and needed help urgently (e.g., food, money, labor, transport, etc.), who IN THIS VILLAGE could you turn to for help? Read list; select all that apply 1. Relatives 2. Non-relatives in my ethnic group/clan 3. Non-relatives in other ethnic group/clan 4. No one 5. Other (specify) -8 Don’t know -9 Refused 59 | Page E2 If your household had a problem and needed help urgently (e.g., food, money, labor, transport, etc.), who OUTSIDE THIS VILLAGE could you turn to for help? Read list; select all that apply 1. Relatives 2. Non-relatives in my ethnic group/clan 3. Non-relatives in other ethnic group/clan 4. No one 5. Other (specify) -8 Don’t know -9 Refused E3 Who INSIDE THIS VILLAGE could you help if they needed help urgently (e.g., food, money, labor, transport, etc.)? Read list; select all that apply 1. Relatives 2. Non-relatives in my ethnic group/clan 3. Non-relatives in other ethnic group/clan 4. No one 5. Other (specify): -8 Don’t know -9 Refused E4 Who OUTSIDE THIS VILLAGE could you help if they needed help urgently (e.g., food, money, labor, transport, etc.)? Read list; select all that apply 1. Relatives 2. Non-relatives in my ethnic group/clan 3. Non-relatives in other ethnic group/clan 4. No one 5. Other (specify): -8 Don’t know -9 Refused MODULE F: AVAILABILITY/ACCESS TO HUMANITARIAN AID F1 In the past 12 months, did your household receive any formal support from the government or another organization? [IF NO, SKIP TO Question G1] 0. Non 1. Yes -8 Don’t know -9 Refused F2 What type of support did your household receive? F2A Emergency food assistance (Support received during after experiencing loss) 0. Non 1. Yes F2B Emergency cash assistance (Support received during after experiencing loss) 0. Non 1. Yes F2C Conditional food transfer (FFW) (Support received because the household fulfils some conditions like being very poor or working) 0. Non 1. Yes F2D Conditional cash transfer (CFW) (Support received because the household fulfils some conditions like being very poor or working) 0. Non 1. Yes F2E Unconditional food transfer (non-emergency) (Food received without fulfilling any condition like being very poor or working) 0. Non 1. Yes F2F Unconditional cash transfer (non-emergency) (Cash received without fulfilling any condition like being very poor or working) 0. Non 1. Yes F2G Household materials and non￾food items 0. Non 1. Yes F2H Educational assistance/school feeding 0. Non 1. Yes 60 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV F2I Agricultural inputs (seeds, fertilizer, etc.) 0. Non 1. Yes F2J Livestock inputs (feed, fodder, medicine, etc.) 0. Non 1. Yes F2K Water Sanitation and Hygiene (WASH) 0. Non 1. Yes F2L Nutrition/supplemental feeding 0. Non 1. Yes F2M Other (specify): 0. Non 1. Yes MODULE G: DIVERSIFICATION OF LIVELIHOODS I am now going to ask you about the different sources of income in your household. G1 In the past 12 months, please tell me whether any member of your household received money in cash or in-kind from the following sources. G1A Own farming/crop production and sales 0. Non 1. Yes G1B Own livestock production and sales 0. Non 1. Yes G1C Agricultural wage labor (within the village) 0. Non 1. Yes G1D Agricultural wage labor (outside the village) 0. Non 1. Yes G1E Non-agricultural wage labor (within the village) 0. Non 1. Yes GIF Wage labour in forestry and natural resources 0. Non 1. Yes G1G Wage labour in mining 0. Non 1. Yes G1H Non-agricultural wage labor (outside the village) 0. Non 1. Yes G1I Salaried work (e.g. teacher, health workers, etc) 0. Non 1. Yes G1J Sale of wild/forest products (e.g., charcoal, firewood) 0. Non 1. Yes G1K Honey production 0. Non 1. Yes G1L Petty trade (reselling other products, e.g., grains, veggies, oil, sugar, etc.) 0. Non 1. Yes G1M Petty trade (own products, e.g., local beer, palm oil, food) 0. Non 1. Yes G1N Other self-employment/own business (agricultural, e.g., buying/selling chat) 0. Non 1. Yes G1O Other self-employment/own business (non-agricultural, e.g., stone cutting, hair braiding, barber, tailors, tinsmith, potter, etc.) 0. Non 1. Yes G1P Rental of land, house, rooms 0. Non 1. Yes G1Q Remittances from a friend or relative living outside the village 0. Non 1. Yes 61 | Page (in another province, city, country) G1R Gifts/inheritance 0. Non 1. Yes G1S Food assistance (WFP, World Vision, etc) 0. Non 1. Yes G1T Cash Transfers 0. Non 1. Yes G1U Non-Food Transfers (Blankets, clothes, household items, WASH, etc) 0. Non 1. Yes G1V Other (specify) 0. Non 1. Yes MODULE H: PARTICIPATION IN HIGH-POTENTIAL VALUE CHAINS H1 Were you or any member of the household involved in the production, processing, or trading of the following products? H1A Maize 0. Non 1. Yes H1B Rice 0. Non 1. Yes H1C Coffee 0. Non 1. Yes H1D Cocoa 0. Non 1. Yes H1E Beans 0. Non 1. Yes MODULE I: ACCESS TO FINANCIAL SERVICES/ SAVING, LOAN AND REMITTANCE Now I will ask some questions about saving money. I1 Do you or any other household member regularly save money (12 last months)? 0. Non 1. Yes I2 Where are the savings primarily held? 1. At home 2. MFI (Financial cooperatives, etc) 3. Village savings/credit group (VSLAs, etc) 4. Solidarity Groups (MUSO) 5. Commercial Bank 6. Mobile banking 7. Other (specify) -8 Don’t know -9 Refused A12 Do you or anyone in your household own a mobile phone? [IF NO, SKIP to A14] 0. No 1. Yes A13 What is your mobile phone number so that we can contact you again later? (Constrain the number of digits in the tablet) FEMALE DECISION MAKER QUESTIONNAIRE Now I would like to speak with the primary female decision maker A14 [Interviewer code, do not read] Primary female decision maker is the head of household. 0. No 1. Yes 62 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV [IF YES, SKIP TO A16] A15 READ SURVEY DESCRIPTION Do you consent to participate in the interview? 0. No 1. Yes A16 Interview Date (Automated) A17 Interview time Hours and Minutes (automated) MODULE J: GENDER EQUITABLE DECISION MAKING J1 Overall, who primarily makes most household decisions? 1. I am solely responsible for decision 2. I ask other HH members, but I most always have final say 3. Spouse/partner and I jointly 4. Spouse/partner or other male asks for my opinion but I don’t have final say 5. Spouse/partner or other male informs me of decision, but I don’t have final say 6. I have no say in the decision 7. Not applicable 8. Other (specify) J2 Who primarily makes decisions on how is household income is used? 1. I am solely responsible for decision 2. I ask other HH members but I most always have final say 3. Spouse/partner and I jointly 4. Spouse/partner or other male asks for my opinion but I don’t have final say 5. Spouse/partner or other male informs me of decision but I don’t have final say 6. I have no say in the decision 7. Not applicable 8. Other (specify) J3 Who primarily makes household decisions over health care and nutrition? 1. I am solely responsible for decision 2. I ask other HH members but I most always have final say 3. Spouse/partner and I jointly 4. Spouse/partner or other male asks for my opinion but I don’t have final say 5. Spouse/partner or other male informs me of decision but I don’t have final say 6. I have no say in the decision 7. Not applicable 8. Other (specify) J4 Who primarily makes decisions about major household purchases? 1. I am solely responsible for decision 2. I ask other HH members but I most always have final say 3. Spouse/partner and I jointly 4. Spouse/partner or other male asks for my opinion but I do not have final say 5. Spouse/partner or other male informs me of decision but I do not have final say 6. I have no say in the decision 7. Not applicable 8. Other (specify) 63 | Page J5 Who primarily makes decisions about your children’s education? 1. I am solely responsible for decision 2. I ask other HH members but I most always have final say 3. Spouse/partner and I jointly 4. Spouse/partner or other male asks for my opinion but I don’t have final say 5. Spouse/partner or other male informs me of decision but I don’t have final say 6. I have no say in the decision 7. Not applicable 8. Other (specify) A18 [IF NO, SKIP TO A26] Do you own a mobile phone? 0. No 1. Yes A19 What is your mobile phone number so that we can contact you again later? (Constrain the number of digits in the tablet) INDIVIDUAL QUESTIONNAIRE Now I would like to speak with the individual household member that was selected. A20 [Interviewer code, do not read] Selected individual is the head of household [IF YES, SKIP TO A24] 0. No 1. Yes A21 READ SURVEY DESCRIPTION Do you consent to participate in the interview? 0. No 1. Yes A22 Individual Interview Date (Automated) A23 Individual Interview time Hours and Minutes (automated) A24 What is the highest level of education of this individual 0. Never studied 1. Alphabet 2. Primary 3. Secondary 4. College/University -9 Refused A25 What is the main income source of this individual? 1. Agriculture 2. Fishing 3. Trade/ entrepreneurship 4. Mining activities 5. Public servant 6. Other (to specify) 7. No activity MODULE K: SOCIAL COHESION INTRA-COMMUNITY Now I would like to ask you some questions about social life in this village. K1 Over the last 12 months, how many times have people in the village gotten together to provide labor to someone else in the village who needed help? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused K2 Over the last 12 months, how many times have people in the village gotten together to provide food to someone else in the village who needed help? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused 64 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV K3 Over the last 12 months, how many times have people in the village gotten together to provide other types of help to someone else in the village who needed it? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused K4 Over the last 12 months, how often did members of this village get together with each other for social events (e.g., weddings, sports events, celebrations, etc.)? 1. Never 2. Once 3. 2-5 times 4. 6 or more times -8 Don’t know -9 Refused MODULE L: SOCIAL COHESION INTER-COMMUNITY Now I would like to ask you some questions about how people in this village interact with people in other villages. L1 Over the last 12 months, how many times have people in the village gotten together with people in a different village that belong to another ethnic group to provide labor to someone who needed help? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused L2 Over the last 12 months, how many times have people in this village gotten together with people in a different village that belong to another ethnic group to provide food to someone who needed help? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused L3 Over the last 12 months, how many times have people in this village gotten together with people in a different village that belong to another ethnic group to provide other types of help to someone who needed it? 1. None, no one needed/asked for help 2. None, no one was able to help 3. Once or twice 4. 3-5 times 5. 6 or more times -8 Don’t know -9 Refused L4 Over the last 12 months, how often did members of this village get together with members of other villages that belong to another ethnic group for social events (e.g., weddings, sports events, celebrations, etc.)? 1. Never 2. Once 3. 2-5 times 4. 6 or more times -8 Don’t know -9 Refused MODULE M: LOCAL GOVERNANCE Now I will ask about services from the government. M1 Over the last 2 years, did you or anyone in your village approach the local government about improving any of the following assets or services? M1A Roads 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1B Schools 1=yes 2=no 65 | Page 3=not applicable -8 Don’t know -9 Refused M1C Health center/post/clinic 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1D Piped water/boreholes/wells 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1E Natural resource conservation 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1F Irrigation systems 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1G Security 1=yes 2=no 3=not applicable -8 Don’t know -9 Refused M1H Other (specify) Specify M2 To what extent were the requested improvements to the assets or services addressed by the local government? M2A [ASK IF M1A = Yes] Roads 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2B [ASK IF M1B = Yes] Schools 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2C [ASK IF M1C = Yes] Health center/post/clinic 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 66 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2D [ASK IF M1D = Yes] Piped water/boreholes/wells 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2E [ASK IF M1E = Yes] Natural resource conservation 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2F [ASK IF M1F = Yes] Irrigation systems 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2G [ASK IF M1G = Yes] Security 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused M2H [ASK IF M1H = Yes] Other (specify) 1. Completely addressed/being addressed 2. Partially addressed 3. Positive response, will be addressed 4. Promised but not yet addressed 5. Not addressed, response pending 6. Not addressed, attempts by leaders failed 7. Leaders did nothing -8 Don’t know -9 Refused MODULE N: ATTITUDE TOWARDS THE USE OF VIOLENCE I have one last question on your opinion related to the use of violence. 67 | Page N1 Which of the following statements is closest to your point of view? (do not read the last option) 1. The use of violence must be avoided 2. It is necessary to use violence to defend a just cause -8 Refused -9 Refused A26 Do you own a mobile phone? [IF NO, END THE INTERVIEW] 0. No 1. Yes A27 What is your mobile phone number so that we can contact you again later? (Constrain the number of digits in the tablet) END OF INTERVIEW THANK YOU FOR YOUR PARTICIPATION 68 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV COMMUNITY QUESTIONNAIRE (CQ) Q code. PRESENTATION, EXPLANATION OF THE SURVEY AND PRIOR CONSENT This community survey was commissioned by SoCha/MSSP at the request of USAID as part of the impact evaluation of the interventions undertaken by some of its partners in the DRC. It will make it possible to have reference values for certain key indicators of these interventions, which will be compared with the final values of the same indicators at the end of the interventions. We are implementing the survey in areas where USAID has interventions or not. As community representative, we will also interview you now before the interventions are rolled out and later after the intervention also to compare the changes before and after the interview. The interview will take about 1 hour. We are independent and guarantee no personal data will be identified in the results. The information provided is confidential and anonymous. You are free to participate or not in this survey and to withdraw at every moment. You are also free not to answer any question and even to stop the survey at any time if you feel uncomfortable or for any other reason. Your participation or non-participation has no negative or positive implications. However, it will contribute to USAID programming in your area and you have been selected to participate as a community representative. M1 Household contact: Hello, my name is _____. I am from an independent research organization called IES. We are conducting a survey. I would like to interview the head of the household and one other person who lives here. Is the head of household available? 2. No 3. Yes M2 Sex of Respondent 2. Female 3. Male M3 Is the respondent the head of the village/neighbourhood? 0. No 1. Yes M4 Specify the title of respondent M5 Participant name M6 Territory 1. BENI 2. MAMBASA M7 Survey area 0. Non-P-DEC 1. P-DEC M8 Group/ Commune 1. MULEKERA 2. RWENZORI 3. BULIKI 4. OICHA 5. MANGINA 6. MUTWANGA 7. KYONDO 8. NYANGWE 9. ANDIBUTA 10. BAFWHIYO 11. BAPONGOMO 12. ANDIKAU 13. BABEKE OF ISAYE 14. MPUTU, 15. BABOMBI 16. TETURI M9 Survey village 81. MBIMBI 82. PAKANZA 83. TENAMBO 84. MASOSI 85. NZANZA 86. MABASELE 87. OICHA 1ER 123. TAMENDE 124. SAIYO 125. BUTSILI 126. KASANGARUHA 127. NGONGOLIO 128. MASYANI 129. MATONGE 69 | Page 88. KASITHU 89. MANGODOMU 90. MASIMBEMBE 91. LINZO 92. MANGINA 93. HOME 4 94. KAVANDA 95. KYOMOLE 96. MULAKIRWA 97. VAHYANA 98. SIVIRWA 99. NZENGA 100. THALIHYA 101. BUKOKOMA 102. MAPOU 103. MUTSORA 104. MABAKUHA 105. BUCHA 106. BIAKATO 107. KATALA 108. MALUTU 109. KANZAMBU 110. TOKOLEKO 111. MANGAMALE 112. MASENZE 113. PEDE 114. MABANZAKU 115. BANDISENDE 116. FUNGA MUKABA 117. BAVASEA 118. KOMBONI 119. MEMEKIDELE 120. MAMULEBU 121. KOMBOSO 122. AKOKORA 130. MATEMBO 131. KALINDA 132. BUNZI 133. KABASHA/KAZEBERE 134. PABUKA 135. SUPA-KALAU 136. LISASA 137. MUMBE/KALUNGUTA 138. KASABINYOLE 139. NGADI 140. NZUMA 141. PAIDA 142. MABAKANGA 143. BOIKENE 144. MAKALANGA 145. NYANGWE 146. MAKOKO 1 147. KERO ZANZIBAR 148. MIRINDI 149. PARANA 150. TOBOLA 2 151. BAPONGOMO 152. BAFWAKOA 153. BAFWAMAZUA 154. BATI ASAY 155. BAFWAZOBANGE 156. BAFWANANGALA 157. BASANJASILI 158. BAFWAKEDU 159. NIANIA 160. MAITATU 161. PUCHA 162. EKWE 163. MUCHACHA 164. YOMBE CHARACTERISTICS OF THE VILLAGE CQ1 How many households are in this village? Estimated CQ2 What is the total population of this village? Estimated CQ3 What are the three largest ethnic groups/clans in this village? 1 2 3. 8. Don’t know 9. Refused CQ4 How many minutes does it take to get to the nearest Town by motorbike? _____ minutes -8. Don’t know -9. Refused AVAILABILITY OF FORMAL SAFETY NETS CQ5 Are there programs or places in this village where people can receive food assistance? 1. Yes 2. No -8 Don’t know -9 Refused CQ6 Are there programs or places in this village where people can receive housing 1. Yes 2. No -8 Don’t know 70 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV materials and other non-food items? -9 Refused CQ7 Are there programs or places in this village where people can receive assistance when they are affected by conflicts? 1. Yes 2. No -8 Don’t know -9 Refused ACCESS TO BASIC SERVICES AND INFRASTRUCTURE CQ8 Is this village supplied with electricity? (Not including solar or other off-grid sources) 1. Yes 2. No -8 Don’t know -9 Refused CQ9 Does this village have mobile phone service? 1. Yes 2. No -8 Don’t know -9 Refused CQ10 Can we capture radio stations in this village? CQ11 What type of route is used to reach this village? 1. Dirt road 2. Gravel road 3. Mixed Dirt and gravel 4. Footpath /trail 5. Other -8 Don’t know -9 Refused CQ12 Are there times of the year when people cannot travel because of poor conditions on the main route? 1. Yes 2. No -8 Don’t know -9 Refused CQ13 Is there a primary school in this village? 1. Yes 2. No -8 Don’t know -9 Refused CQ14 Is there a RECO in this village? 1. Yes 2. No -8 Don’t know -9 Refused CQ15 Is there health service (post, clinic, or center) in this village? 1. Yes 2. No -8 Don’t know -9 Refused CQ16 How much time does it take to reach the nearest market by motorbike? CQ17 How much time does it take to reach the nearest source of potable water on foot? 71 | Page CQ18 Is there any community peace structure in this village, such as…? 1. Yes 2. No -8 Don’t know -9 Refused CQ19 (If Yes) Is the community peach structure functional currently such that it provides conflict mediation to people in need? 1. Yes 2. No -8 Don’t know -9 Refused CQ20 Is there any mechanism in this village for early warning of attacks by armed groups? 1. Yes 2. No -8 Don’t know -9 Refused M10 Telephone Number of Respondent M11 GPS Coordinates THANK YOU FOR YOUR PARTICIPATION 72 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV P-DEC IMPACT EVALUATION VILLAGE-LEVEL KEY-INFORMANT INTERVIEW GUIDE Overview: In each village, four community representatives will be interviewed in four separate interviews. All four guides must be administered in each village. Two men and two women should be interviewed, one woman at minimum. Seven types of community representative are eligible for interviews. These types are as follows: a) Chef de village b) RECO c) Head teacher d) Religious leader e) Women’s group leader f) Youth group leader g) Local peace committee leader The questions will be distributed among the four interviews so that Questions 1-3 in the KII guide are discussed in each interview. Questions 3-8 are distributed between the four versions of the guide. Text in regular font: Read out. Text in italics: Explanations and instructions; do not read to the respondent. ● Bring a consent form in the local language. ● Introduce yourself and explain the objective of the interview. ● Make sure others cannot overhear the conversation. ● Ask for permission to record the interview. Explain how the recording will be used. Only proceed if respondent agrees to be recorded. ● Read out the consent form. Only proceed if respondent agrees verbally or signs it. ● Ask respondent for any questions they might have. Respondent 1 1. Since when have you lived in this village? If moved after August 2020: Where did you live before? 2. What is your role in the village (confirm one of the roles a-g)? Since when are you in this role? a) Chef de village (mandatory using KII Guide 1) b) RECO c) Head teacher d) Religious leader e) Women’s group leader f) Youth group leader g) Local peace committee leader Major Village-Level Events Since August 2020 I will ask you about events that have taken place since August 2020. That was about two and a half years ago. A few months before August 2020, the end of the Ebola outbreak was officially declared. For each of my questions, please try to recall the time that has passed since the end of the Ebola outbreak in August 2020. 73 | Page For each of the following sets of questions, read out additional explanations if provided, but wait for an answer before asking any of the follow-up questions. Ensure through probing that for each event affecting a larger area, answers provide information on how the event has affected the respondent’s village. Please note that answers to questions may overlap. For example, a violent conflict may also cause migration, change the availability of food, etc. In such cases, refer to previous answers that may apply to the question at hand and confirm with respondent, then ask about additional events of the kind described in the question text. 3. Since August 2020, has your village been affected by any violent conflicts? These could be conflicts that involve outsiders, including attacks by armed groups, or conflicts between members of your community. The conflicts may be related to identity, natural resources, or recruitment by armed groups. If respondent answers no, move to next question. Let’s go through these conflicts one by one. Please tell me: a. When did the conflict take place? b. If it did not involve outsiders, what was the reason for the conflict? c. If it did involve outsiders, who were those outsiders? d. What consequences did the conflict have for your village? e. Approximately how many households in your village were directly affected? f. From your village, who got involved in mitigating or solving the conflict? g. How did the government respond? 4. Since August 2020, have there been any moments when people were joining or leaving your village in large numbers because of a violent conflict? If respondent answers no, move to next question. Let’s go through these moments one by one. Please tell me: a. When did it take place? b. What was the reason for people to join or leave your village? c. How many households left or joined your village? 5. Since August 2020, has your community received any support from outside organizations or the government? This could include support for resolving or preventing conflicts and reducing the negative effects of conflicts, training of community members or community leaders, or activities intended to increase social cohesion. If respondent answers no, move to next question. Let’s go through these types of support one by one. Please tell me: a. Could you describe this type of support in more detail? b. When was it provided to your village? c. Who provided it? If you are not sure, please tell us what you know. d. How many households directly benefited from it? 74 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Respondent 2 1. Since when have you lived in this village? If moved after August 2020: Where did you live before? 2. What is your role in the village (confirm one of the roles a-g)? Since when are you in this role? a) Chef de village (mandatory using KII Guide 1) b) RECO c) Head teacher d) Religious leader e) Women’s group leader f) Youth group leader g) Local peace committee leader Major Village-Level Events Since August 2020 I will ask you about events that have taken place since August 2020. That was about two and a half years ago. A few months before August 2020, the end of the Ebola outbreak was officially declared. For each of my questions, please try to recall the time that has passed since the end of the Ebola outbreak in August 2020. For each of the following sets of questions, read out additional explanations if provided, but wait for an answer before asking any of the follow-up questions. Ensure through probing that for each event affecting a larger area, answers provide information on how the event has affected the respondent’s village. Please note that answers to questions may overlap. For example, a violent conflict may also cause migration, change the availability of food, etc. In such cases, refer to previous answers that may apply to the question at hand and confirm with respondent, then ask about additional events of the kind described in the question text. 3. Since August 2020, has your village been affected by any violent conflicts? These could be conflicts that involve outsiders, including attacks by armed groups, or conflicts between members of your community. The conflicts may be related to identity, natural resources, or recruitment by armed groups. If respondent answers no, move to next question. Let’s go through these conflicts one by one. Please tell me: a. When did the conflict take place? b. If it did not involve outsiders, what was the reason for the conflict? c. If it did involve outsiders, who were those outsiders? d. What consequences did the conflict have for your village? e. Approximately how many households in your village were directly affected? f. From your village, who got involved in mitigating or solving the conflict? g. How did the government respond? 4. Since August 2020, have there been any major, unusual changes in how much food was available to households in your community? Such changes could have taken place because of conflicts including attacks by armed groups, bad harvests, or major price increases. If respondent answers no, move to next question. Let’s go through these changes one by one. Please tell me: a. When did it take place? b. What was the reason for the change? c. How many households were directly affected? 75 | Page 5. How do households in your village earn money? Are there any major changes in this regard since August 2020? For example, mines or shops may have been opened or closed, villagers may have changed the crops they are planting, or agro-dealers may have started or ceased to come to the village. If respondent answers no, move to next question. Let’s go through these changes one by one. Please tell me: a. Could you describe the change in more detail? b. When did it take place? c. How many households were directly affected? Respondent 3 1. Since when have you lived in this village? If moved after August 2020: Where did you live before? 2. What is your role in the village (confirm one of the roles a-g)? Since when are you in this role? a) Chef de village (mandatory using KII Guide 1) b) RECO c) Head teacher d) Religious leader e) Women’s group leader f) Youth group leader g) Local peace committee leader 3. Since August 2020, have there been any moments when people were joining or leaving your village in large numbers because of a violent conflict? If respondent answers no, move to next question. Let’s go through these moments one by one. Please tell me: a. When did it take place? b. What was the reason for people to join or leave your village? c. How many households left or joined your village? 4. Since August 2020, has your community received any support from outside organizations or the government? This could include support for resolving or preventing conflicts and reducing the negative effects of conflicts, training of community members or community leaders, or activities intended to increase social cohesion. If respondent answers no, move to next question. Let’s go through these types of support one by one. Please tell me: a. Could you describe this type of support in more detail? b. When was it provided to your village? c. Who provided it? If you are not sure, please tell us what you know. d. How many households directly benefited from it? Village Needs I would now like to talk to you about what it is like to live in your village. 5. Please take a moment to think of issues that negatively affect life in your village. What are the three most important issues? Let’s talk about them one by one. a. How does this issue negatively affect life in your village? b. With regard to this issue, who could improve the situation and how? 76 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV c. How could people in your village improve the situation? d. How could the government improve the situation? e. How could local or foreign organizations improve the situation? Respondent 4 1. Since when have you lived in this village? If moved after August 2020: Where did you live before? 2. What is your role in the village (confirm one of the roles a-g)? Since when are you in this role? a) Chef de village (mandatory using KII Guide 1) b) RECO c) Head teacher d) Religious leader e) Women’s group leader f) Youth group leader g) Local peace committee leader 3. Since August 2020, have there been any major, unusual changes in how much food was available to households in your community? Such changes could have taken place because of conflicts including attacks by armed groups, bad harvests, or major price increases. If respondent answers no, move to next question. Let’s go through these changes one by one. Please tell me: a. When did it take place? b. What was the reason for the change? c. How many households were directly affected? 4. How do households in your village earn money? Are there any major changes in this regard since August 2020? For example, mines or shops may have been opened or closed, villagers may have changed the crops they are planting, or agro-dealers may have started or ceased to come to the village. If respondent answers no, move to next question. Let’s go through these changes one by one. Please tell me: a. Could you describe the change in more detail? b. When did it take place? c. How many households were directly affected? Village Needs I would now like to talk to you about what it is like to live in your village. 5. Please take a moment to think of issues that negatively affect life in your village. What are the three most important issues? Let’s talk about them one by one. a. How does this issue negatively affect life in your village? b. With regard to this issue, who could improve the situation and how? c. How could people in your village improve the situation? d. How could the government improve the situation? e. How could local or foreign organizations improve the situation? 77 | Page ANNEX IV: SOURCES OF INFORMATION This baseline relies on several sources of information as shown in the table below. Identities of all respondents are kept confidential to maintain their privacy. Type of data Number collected Details Household surveys 2,434 households (1,229 in P￾DEC targeted areas; 1,205 in non-P-DEC areas) Respondents include heads of household, female decision￾makers in household, and randomly selected individuals Community surveys 80 village heads (40 each in P￾DEC/non-P-DEC areas) Structured qualitative KIIs 80 individuals from 20 villages across Beni and Mambasa Respondents include village chiefs, health workers, head teachers, religious leaders, women’s group leaders, youth group leaders, and local peace committee leaders 78 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX V: EVALUATION TEAM MEMBERS This evaluation was carried out by several team members, with support and oversight from the SoCha MSSP field office and home office.Team Leader: Levison Chiwaula, PhD. • Qualifications and experience: PhD in Economics with 20 years of experience in applied development economics research, evaluation, and assessment. Dr. Chiwaula is currently Professor of Economics at the University of Malawi. • Role: Designed evaluation; designed data collection tools; completed analysis; drafted report Impact Evaluation Coordinator: Emmanuel Muzigirwa, PhD • Qualifications and experience: PhD in development economics with more than 14 years of experience in monitoring, evaluation, and research in DR Congo. Dr. Muzigirwa is currently director of Innovations & Social Entrepreneurship (IES) in DR Congo. • Role: Supported development of data collection tools; trained enumerators; managed data collection; cleaned quantitative data Statistician: Schadrack Kambale • Qualifications and experience: Master’s degree in statistics and over four years of experience planning and implementing quantitative studies and data collection. • Role: Supported tool design, data management, and analysis. Field data collection partner: Innovations & Social Entrepreneurship (IES) • Qualifications and experience: A “think-do tank” providing monitoring and evaluation consultancy services and action research in complex settings in DR Congo since 2015. • Role: Field data collection partner, led by Emmanuel Muzigirwa 79 | Page ANNEX VI: POWER CALCULATIONS FOR THE ORIGINAL SAMPLE 80 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX VII: DISTRIBUTION OF SAMPLE BETWEEN TREATMENT AND CONTROL REGIONS Treatment Area Control Area Territory Villages Sample size Territory Villages Sample size BENI Kabasha/kazebere 33 BENI Sivirwa 27 Mumbe/kalunguta 29 Kavanda 36 Saiyo-mulekera 34 Kaviranga 32 Bunzi 31 Kyomole 32 Butsili 29 Mulakirwa 30 Kalinda 29 Thalihya 29 Kasangaruha 30 Home 4 29 Masyani 29 Kasithu 29 Matembo 35 Linzo 31 Matonge 29 Mangina 29 Ngongolio 33 Mangodomu 29 Tamende 31 Masimbembe 29 Boikene 31 Vahyana 36 Kasabinyole 33 Bukokoma 30 Mabakanga 29 Mapou 29 Ngadi 33 Mutsora 29 Nzuma 29 Nzenga 32 Paida 31 Baikaku 29 MAMBAS A Ekulungu 30 Mabasele 30 Maitatu 28 Masosi 29 Pucha 29 Mbimbi 29 Bagdutambila 32 Nzanza 29 Bafwakoa 30 Oicha 1er 29 Bafwanangala 30 Pakanza 29 Bafwazobange 29 Tenambo 35 Basanjasili 36 MAMBAS A Akokora 26 Nia-nia 36 Komboso 29 Banana 1 ( ecole) 33 Nduye 2 31 Banana 2 ( gite) 29 Bavasea 32 Bandikambwa-seti￾mabu 29 Salate 2 32 Kilima mwenza 29 Bandisende 33 Kilonge 29 Bianze babama 30 Parana 30 Butama 31 Sayo-nyangwe 29 Pede 28 Bavalakaniki 33 Sayo-teturi 29 Madidi 29 Alima 29 Makalanga 32 Biakato 29 Makoko 1 29 Katala 31 81 | Page Treatment Area Control Area Territory Villages Sample size Territory Villages Sample size Mirindi 29 Lalia 30 Nyangwe 31 Malutu 28 TOTAL 1229 1205 82 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX VIII: RESULTS OF THE BALANCE TEST FOR VARIABLES USED IN MATCHING HOUSEHOLDS FROM TREATED AND CONTROL AREAS Mean t-test V(T)/ Variable Treated Control %bias t p>t V(C) AgeHead 45.16 44.89 1.90 0.40 0.69 1.01 AgeHeadsqd 2232.60 2206 1.90 0.41 0.68 1.02 SexHead 0.86 0.86 0.00 0.00 1.00 . Married_head 0.86 0.85 1.30 0.27 0.79 . hhsize 5.49 5.63 -6.10 -1.29 0.20 1.01 hhsizesqd 34.86 36.30 -5.40 -1.14 0.25 0.98 Education_adult 0.85 0.86 -2.20 -0.47 0.64 . AssetsConsumer 10.64 10.40 3.20 0.73 0.46 1.13 AssetsProductive 4.01 4.03 -0.70 -0.16 0.87 1.01 AssetsLivestock 3.40 3.79 -5.40 -1.14 0.25 1.69* crop_production 0.65 0.64 0.20 0.05 0.96 . livestock_produc tion 0.24 0.24 0.30 0.06 0.96 . agricultural_wage 0.51 0.52 -3.40 -0.71 0.48 . mining_wage 0.12 0.12 -1.30 -0.29 0.77 . formal_employm ent 0.13 0.12 3.00 0.65 0.52 . natural_resources 0.16 0.15 2.30 0.52 0.60 . petty_trade 0.64 0.62 4.40 0.93 0.35 . rentals 0.10 0.10 1.40 0.31 0.75 . Residence_length 14.05 14.00 0.30 0.06 0.95 1.07 Christian 0.94 0.94 -0.80 -0.20 0.84 . Time_town 165.83 154.41 5.00 1.33 0.18 0.99 Primary_school 0.77 0.77 -1.30 -0.28 0.78 . Formal_safety_n ets 0.55 0.59 -8.00 -1.67 0.10 . Peace_structure 0.50 0.50 0.00 0.00 1.00 . All_weather_acce ss 0.53 0.49 7.40 1.56 0.12 . Electricity_suppl y 0.17 0.16 1.60 0.32 0.75 . Time_market 31.40 30.64 2.10 0.46 0.65 0.49* Time_water 19.01 18.61 2.70 0.58 0.56 0.63* Humanitarian_su pport 0.10 0.12 -4.50 -0.98 0.33 . Early_warning 0.53 0.55 -3.80 -0.81 0.42 . Bonding_Social_ Capital 2.18 2.20 -1.10 -0.23 0.81 1.01 Bridging_Social_ Capital 2.23 2.24 -0.20 -0.04 0.97 0.94 83 | Page Com_LG_respon siveness 4.70 4.70 -1.20 -0.25 0.80 1.14 84 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV ANNEX IX: BASELINE VALUES OF COMPONENTS OF THE HOUSEHOLD RESILIENCE INDEX Component (range 0 to 10) Comparison Treatment Total Cash savings 1.03 0.95 0.99 Formal safety nets 5.91 5.51 5.71 Humanitarian aid 1.19 1.04 1.11 Consumption Assets 0.90 0.92 0.91 Productive Assets 1.30 1.29 1.30 Livestock 0.44 0.39 0.41 Bonding Social Capital 5.49 5.46 5.47 Bridging Social Capital 4.47 4.47 4.47 Education and Training 5.76 5.72 5.74 Gender equitable decision making 5.61 5.51 5.56 Local gov’t responsiveness 5.81 5.78 5.80 N 1788 Clustered standard errors in used in computing t-statistics * p < 0.10, ** p < 0.05, *** p < 0.01 Source: Authors from PDEC Baseline Survey 85 | Page ANNEX X. CHARACTERISTICS AND OUTCOMES, BY TERRITORY AND GENDER OF HOUSEHOLD HEAD Using the matched sample, tables below display characteristics and outcomes separately for each of the two territories and for female-headed versus male-headed households. Table X.i. Household characteristics by territory and gender of household head Beni Mambasa Female Male HH head age 46.3 43.3 50.3 44.2 Married HH head 0.81 0.92 0.12 0.97 HH size 5.81 5.21 5.22 5.61 Primary education 0.86 0.85 0.74 0.87 Christian 0.98 0.90 0.98 0.94 Time in village (yrs) 14.4 13.5 13.8 14.1 In village <12 mo. 0.18 0.30 0.29 0.22 In village <3 mo. 0.13 0.28 0.22 0.18 Plot area (hectares) 0.54 1.14 0.39 0.85 Consumer assets 11.3 9.38 9.05 10.8 Productive assets 3.75 4.41 3.01 4.18 Livestock assets 4.15 2.81 2.64 3.75 N 1788 Table X.ii. Community characteristics by territory and gender of household head Territory Sex Beni Mambasa Female Male Electricity supply 0.28 0 0.24 0.15 Radio signal 0.97 0.85 0.94 0.92 Phone service 0.90 0.56 0.82 0.75 Time to town 57.8 304.7 107.0 168.6 Village population 20892.9 4609.1 18880.4 13388.9 Primary school 0.88 0.61 0.80 0.76 Formal safety nets 0.73 0.35 0.66 0.56 Humanitarian support 0.12 0.099 0.12 0.11 Peace structure 0.63 0.33 0.58 0.49 All weather access 0.38 0.70 0.43 0.53 Time to market 21.7 44.1 25.0 32.0 Time to water 16.2 22.5 16.5 19.2 Early warning systems 0.72 0.29 0.62 0.53 N 1788 86 | PARTNERSHIP FOR THE DEVELOPMENT OF EASTERN CONGO IMPACT EVALUATION BASELINE REPORT USAID.GOV Table X.iii. Livelihoods by territory and gender of household head Territory Gender Beni Mambasa Female Male Crop production 0.57 0.75 0.54 0.66 Livestock production 0.25 0.23 0.17 0.25 Agricultural wage 0.54 0.47 0.45 0.53 Mining wage 0.028 0.25 0.012 0.14 Wage other 0.37 0.43 0.33 0.40 Formal employment 0.11 0.14 0.089 0.13 Natural resources 0.12 0.21 0.089 0.17 Petty trading 0.57 0.72 0.55 0.64 Rentals 0.091 0.11 0.085 0.10 Remittances 0.21 0.28 0.19 0.25 N 1788 Table X.iv. Components of resilience by territory and gender of household head Territory Gender Component (scale 0 to 10) Beni Mambasa Female Male Cash savings 1.35 0.49 0.69 1.04 Formal safety nets 7.29 3.48 6.59 5.57 Humanitarian aid 1.20 0.99 1.22 1.10 Consumption Assets 0.98 0.81 0.78 0.93 Productive Assets 1.21 1.42 0.97 1.35 Livestock 0.48 0.32 0.30 0.43 Bonding Social Capital 5.48 5.47 5.23 5.51 Bridging Social Capital 4.16 4.90 4.02 4.54 Education and Training 5.75 5.72 4.93 5.87 Gender equitable decision making 5.64 5.45 8.25 5.13 Local gov’t responsiveness 7.01 4.08 6.34 5.71 N 1788 Table X.v. Impact indicators by territory and gender of household head Territory Sex Beni Mambasa Female Male Household Resilience Capacity (0/110) 40.5 33.1 39.3 37.2 High potential value chains (0/5) 1.62 2.37 1.52 1.99 Livelihood diversification index (0/15) 4.13 5.09 3.66 4.66 Social cohesion (outside) (0/16) 10.0 12.0 10.4 10.9 87 | Page Social cohesion(internal) (0/16) 12.1 13.4 12.4 12.7 LG responsiveness (0/7) 4.95 4.34 4.81 4.68 N 1788 U.S. Agency for International Development 198 Isiro Avenue Kinshasa, Gombe Democratic Republic of the Congo Phone: +243 81 700 5701