DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. FINAL REPORT MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID/Philippines Collaborating, Learning, and Adapting for Improved Development FEBRUARY 2023 1 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV TABLE OF CONTENTS ABSTRACT 3 ACRONYMS 4 ANNEXES 6 LIST OF TABLES 7 LIST OF FIGURES 7 EXECUTIVE SUMMARY 8 METHODOLOGY 8 FINDINGS 8 MRP INTERVENTIONS’ RELEVANCE 8 CONTRIBUTION TO IDPS’ AND HCMS’ ECONOMIC CONDITIONS 9 IMPROVEMENT IN BENEFICIARIES’ SOCIAL COHESION 9 REDUCTION OF SOCIAL GAPS 10 CONTRIBUTION TO SELF-RELIANCE AND DURABLE SOLUTIONS 10 MECHANISMS TO SUSTAIN PROJECT OUTCOMES 10 CONCLUSIONS 11 RECOMMENDATIONS 11 PLANNING, DESIGN, AND IMPLEMENTATION OF FUTURE DEVELOPMENT INTERVENTIONS 11 ENSURING OUTCOME SUSTAINABILITY 12 CONDUCTING FUTURE EVALUATION, RESEARCH, AND LEARNING 12 EVALUATION PURPOSE AND EVALUATION QUESTIONS 13 EVALUATION PURPOSE 13 EVALUATION QUESTIONS 13 BACKGROUND 14 DESCRIPTION OF THE PROBLEM AND CONTEXT 14 PROJECT DESCRIPTION 15 DESCRIPTION OF THE INTERVENTION AND THEORY OF CHANGE 16 EVALUATION METHODOLOGY 18 EVALUATION DESIGN 18 SAMPLING DESIGN 18 DATA COLLECTION 18 DATA ANALYSIS 19 METHODOLOGICAL LIMITATIONS 19 FINDINGS 20 RELEVANCE 20 RELEVANCE TO IDP AND HCM NEEDS 20 ADAPTATIVE MANAGEMENT 20 COLLABORATION WITH LOCAL PARTNERS 21 BENEFICIARY STORY OF RELEVANCE 22 RELEVANCE TO SOCIAL INCLUSION ISSUES 22 COMMUNITY SOLIDARITY GROUPS’ IMPORTANCE 24 COMPLEMENTATION OF PHILIPPINE GOVERNMENT RESPONSE PRIORITIES 25 RELEVANCE TO USAID’S DEVELOPMENT STRATEGY 27 EFFECTIVENESS 28 IMPROVING IDPS‘ AND HCMS’ ECONOMIC CONDITIONS 28 STRENGTHENING SOCIAL COHESION OF IDPS AND HCMS 31 USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 2 CONTRIBUTION TO PERCEPTIONS ABOUT POLARIZATION 32 CONTRIBUTION TO PUBLIC PARTICIPATION 34 REDUCING SOCIAL GAPS 36 GENDER ANALYSIS 36 IDP/HCM ANALYSIS 37 RURAL-URBAN ANALYSIS 38 CONTRIBUTION TO IDPS’ AND HCMS’ SELF-RELIANCE 38 BASELINE-ENDLINE ANALYSIS 38 DURABLE SOLUTION FOR IDPS 40 SUSTAINABILITY 41 FORGED PARTNERSHIPS AND SYNERGIES AMONG ACTIVITY STAKEHOLDERS 42 ESTABLISHED POLICIES AND STRATEGIES FOR SUSTAINABILITY 44 VALUE CHAIN APPROACH (VCA) 45 EMPOWERED COMMUNITY SOLIDARITY GROUPS 46 IMPLEMENTED INTENTIONAL SOCIAL INCLUSION APPROACH 48 LESSONS LEARNED 48 CONCLUSIONS 50 RELEVANCE 50 EFFECTIVENESS 50 SUSTAINABILITY 50 RECOMMENDATIONS 52 INTERVENTION PLANNING, DESIGN, AND IMPLEMENTATION 52 ENSURING SUSTAINABILITY OF ACTIVITY OUTCOMES 52 CONDUCTING FUTURE RESEARCH 53 3 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV ABSTRACT The United States Agency for International Development (USAID) commissioned a third-party performance evaluation of the Marawi Response Project (MRP). MRP is USAID’s development assistance response to address the long-term rehabilitation needs of communities affected by the Marawi siege in March–October 2017. The evaluation covers the following areas: relevance, effectiveness, and sustainability. The evaluation used a convergent mixed-methods design. Data were generated through an endline survey, focus group discussions, key informant interviews, and document review. The evaluation team gathered data from individuals, communities, and organizations in the twenty-two municipalities and two cities within Lanao del Sur and Lanao del Norte. The evaluation team analyzed the quantitative data using descriptive and inferential statistical tools and the qualitative data using content analysis. The evaluation team found that MRP’s relevance to the needs of the internally displaced persons (IDPs) and host community members (HCMs) is anchored in its application of USAID’s guidance on co-creation processes, adaptive management, and local partnerships. The activity has demonstrated its effectiveness by stimulating the restoration of individual and community capacities to engage in economic activities; improving beneficiaries’ perceptions of polarization and public participation; reducing gender and other social gaps; and offering opportunities for IDPs to integrate locally. The team found that MRP has established concrete sustainability mechanisms that are essential to sustaining the activity’s outcomes. These mechanisms focused on harnessing partnerships and synergies with activity stakeholders; laying the foundation for the community solidarity groups (CSGs) to qualify for future government support and be linked to potential markets; empowering the CSGs to continue to operate, serve their members, and even scale up their livelihoods after MRP ends; and establishing a social fabric for the harmonious involvement, participation, and interaction of IDPs and HCMs as they scale up their activities. Based on the findings, the evaluation team forwards recommendations that USAID, local government units, and other development actors can consider for planning, designing, and implementing future interventions; ensuring sustainability of activity gains; and conducting future evaluation and research endeavors. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 4 ACRONYMS AMELP Activity Monitoring, Evaluation, and Learning Plan BARMM Bangsamoro Autonomous Region of Muslim Mindanao BLGU Barangay Local Government Unit BRG Business Recovery Grant CDCS Country Development Cooperation Strategy CLAimDev Collaborating, Learning, and Adapting for Improved Development COP Chief of Party CSG Community Solidarity Group CSO Civil Society Organization DILG Department of Interior and Local Government DNH Do No Harm DOLE Department of Labor and Employment DTI Department of Trade and Industry ECOWEB Ecosystem Work for Essential Benefits FGD Focus Group Discussion GBV Gender Based Violence HCM Host Community Members IKG In-Kind Grant IR Intermediate Result IDP Internally Displaced Person KII Key Informant Interview LCP Local Consortium Partner LGU Local Government Unit MARADECA Maranao People Development Center MRP Marawi Response Project MSC Most Significant Change NGA National Government Agency NGO Non-Governmental Organization PDP Philippines Development Plan PSA Philippines Statistics Authority RF Results Framework SOW Statement of Work 5 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV STTA Short Term Technical Assistance TFBM Task Force Bangon Marawi TESDA Technical Education and Skills Development Agency USAID United States Agency for International Development VEO Violent Extremist Group VCA Value Chain Approach USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 6 ANNEXES Annex 1 Statement of Work Annex 2 Evaluation Design and Workplan (Inception Report) Annex 3 Sample Selection Criteria and Process Annex 4 Evaluation Participants Annex 5 Enumerators’ Training Design Annex 6 Pilot Test Design and Administration Guide Annex 7 Endline Survey Tool Annex 8 FGD and KII Tools Annex 9 Informed Consent Form Annex 10 Activities that Contributed to MRP’s Adaptability and Responsiveness Annex 11 Community Engagement and Grant Application Cycle Annex 12 Collaborative Activities Conducted by MRP Annex 13 MRP Performance in Relation to the Project Indicators Annex 14 Coordination, Partnership, and Collaboration Activities of MRP Annex 15 List of Sustainability Commitments from Various LGUs Annex 16 Quantitative Analysis Results Annex 17 FGD and KII Transcripts Annex 18 Process and Script for Validating Survey Respondents Annex 19 Suggested Revisions Based on Pilot-test Observation Annex 20 Non-Disclosure and Conflict of Interest Agreement 7 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV LIST OF TABLES Table 1 Summary Information Table 2 MRP Project Areas Table 3 Sample Activities That Contributed to mrp’s Adaptability and Responsiveness Table 4 Women’s Participation in MRP Activities Table 5 Sample of Collaborative Activities That Mrp Conducted Table 6 Sample Coordination, Partnership, and Collaboration Activities Table 7 Sample of LGU Commitments LIST OF FIGURES Figure 1 MRP’s Results Framework Figure 2 Percentage of Intervention and Comparison Participants with Savings and Sufficient Income Figure 3 Percentage of Intervention and Comparison Participants Across Sources of Household Income Figure 4 Percentage of Intervention and Comparison Participants Across Employment￾Related Variables Figure 5 Level of Polarization of MRP Beneficiaries During Baseline and Endline Figure 6 Level of Trust of MRP Beneficiaries During Baseline and Endline Figure 7 Improvement in Trust Indicators Among MRP Beneficiaries from Baseline to Endline Figure 8 Levels of Public Participation in Intervention and Comparison Groups Figure 9 Improvement in Public Participation Indicators Among MRP Beneficiaries from Baseline to Endline Figure 10 Comparison of Women’s and Men’s Self-Reliance, Polarization, and Public Participation Figure 11 Comparison of IDPs’ and HCMs’ Self-Reliance, Polarization, and Public Participation Figure 12 Comparison of Rural and Urban Participants’ Self-Reliance, Polarization, and Public Participation Figure 13 Level of Self-reliance of MRP Beneficiaries During Baseline and Endline Figure 14 Level of Self-reliance of Intervention and Comparison Participants During Endline Figure 15 Durable Solution Preference of Intervention and Comparison Participants Figure 16 MRP Grant Implementation Process USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 8 EXECUTIVE SUMMARY The siege of Marawi City by ISIS-inspired armed groups in March–October 2017 displaced around 369,196 individuals, who took refuge in emergency shelters or with relatives or friends in surrounding areas. The siege worsened affected communities’ socioeconomic conditions by destroying property, lifelines, and businesses. USAID’s Marawi Response Project (MRP) was designed to address these internally displaced persons’ (IDPs’) early recovery needs and the long-term rehabilitation needs of communities and local governments affected by the siege. This third-party performance evaluation aims to gauge MRP’s performance in accomplishing its outcomes. The evaluation addresses the following questions: Relevance: Were MRP’s development interventions relevant to IDP and host community needs and adaptive to the complex and changing external environment in Mindanao, especially in the project areas? Effectiveness: Did MRP accomplish the project’s expected outcomes of improving self￾reliance and strengthening social cohesion among IDPs and host community members (HCMs)? Sustainability: Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen social cohesion between IDPs and HCMs? The evaluation’s target audiences include USAID/Philippines, USAID/Regional Development Mission for Asia, USAID/Washington, Plan International (MRP’s implementing partner), national government agencies, local government units (LGUs), nongovernmental organizations (NGOs), community solidarity groups (CSGs), and other donor agencies. METHODOLOGY The evaluation team employed a convergent mixed-methods design. The quantitative aspect involved an endline survey (n=684) that generated beneficiary-level data on socioeconomic indicators and activity outcome variables—namely, self-perceived self-reliance, self-perceived level of polarization, and self-perceived level of public participation. The qualitative aspect involved document review, focus group discussions (FGDs; n=87), and key informant interviews (KIIs; n=37). The endline survey and the FGDs were conducted in person; the KIIs were carried out using both in-person and online modalities. The team used descriptive and inferential statistical tools to analyze the quantitative data and performed descriptive analyses (e.g., means, standard deviation, frequency and percentage distribution) of pertinent survey variables. The team used panel data analysis to determine improvement in self-reliance, polarization, and public participation among MRP beneficiaries and T￾test for independent samples to compare MRP beneficiary and comparison group data. The team triangulated results with qualitative data from document reviews and FGD and KII audio and video recording transcriptions. FINDINGS MRP INTERVENTIONS’ RELEVANCE The results revealed that MRP’s interventions were relevant to IDP needs and adaptive to the activity area’s complex environment. The team attributes this finding to MRP’s application of USAID 9 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV guidance on co-creation, adaptive management, and local partnerships. The co-creation process was especially helpful in ensuring that the interventions were relevant to IDP and HCM needs. MRP’s flexibility in programming and packaging interventions reflects USAID’s principle of adaptive management. By being flexible in framing and setting targets, for example, MRP could reconfigure its interventions to meet beneficiary needs at a given juncture of implementation. Its adaptive community-based approaches were relevant in improving IDP and HCM self-reliance, narrowing social gaps, and providing enabling conditions for IDPs’ local integration. The team found that the community solidarity groups (CSGs), MRP’s primary mechanism for soliciting grant proposals, have been pivotal in addressing beneficiary needs. CSG members’ role in identifying and prioritizing community needs, suggesting opportunities for collaboration to address these needs, and pointing out potential community tensions is essential in ensuring that interventions address IDP and HCM needs. MRP has adapted intervention planning and implementation by regularly engaging government agencies, LGUs, and traditional and religious leaders. It has also endeavored to mainstream its efforts in collaboration with Task Force Bangon Marawi and other government agencies. The project has thus made relevant contributions to the ongoing efforts to rehabilitate Marawi. CONTRIBUTION TO IDPS’ AND HCMS’ ECONOMIC CONDITIONS MRP’s IR1, the improvement of IDPs’ and HCMs’ economic conditions, has the following outcome indicators: (1) number of displaced business owners with new or restarted businesses and (2) percentage of trained IDPs/HCMs gainfully employed. To attain these outcomes, MRP provided business recovery grants, livelihood microgrants, and workforce development training workshops. These interventions alleviated IDPs’ and HCMs’ economic conditions. In particular, the team found that the proportion of endline survey participants who reported having sufficient income to meet their families’ basic needs as well as having savings was greater among MRP beneficiaries than among the comparison group. The proportion of beneficiaries engaged in income-generating activities such as crop and livestock farming, non-food retailing, food retailing/sari-sari store, and paid work was also greater than in the comparison group. Inferential analysis results suggest that the proportion of beneficiaries engaged in paid work/livelihood and having the right skills to find a job was significantly higher than in the comparison group. Interview data corroborate these findings. During the interviews, beneficiaries shared that MRP’s interventions have helped meet their families’ needs—for example, food and school materials for their children—especially during the height of the COVID-19 pandemic. These findings imply that MRP’s business and livelihood interventions have stimulated economic activity among beneficiaries amid the economic impact of the COVID-19 pandemic. In particular, the in-kind grants and the training that MRP provided improved IDPs’ and HCMs’ economic conditions. IMPROVEMENT IN BENEFICIARIES’ SOCIAL COHESION MRP’s IR2 focuses on strengthening IDPs’ and HCMs’ social cohesion. To achieve this result, MRP provided social cohesion grants designed to foster community improvement and civic engagement. The team found improvement in several social cohesion indicators. For example, MRP beneficiaries’ overall perception of polarization had significantly improved at the endline. This finding implies that the beneficiaries are perceiving significantly less polarization or fragmentation in their communities. The beneficiaries’ level of trust had also significantly increased at the endline and was significantly higher than in the comparison group. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 10 The endline survey data also revealed that MRP beneficiaries have a high level of public participation, another social cohesion indicator, whereas the non-MRP respondents had low participation levels. Further analysis confirmed that MRP beneficiaries’ level of participation was significantly higher than that of the comparison group. The team also found improvement in indicators related to communication with CSG and barangay leaders about community problems through writing or personally talking to their community leaders about problems they have observed in the community. There is also improvement in indicators related to participation in community decisions and training, which involves attending employment training and community meetings. REDUCTION OF SOCIAL GAPS The evaluation team also investigated MRP’s contribution to reducing social gaps. The endline survey revealed that women and men beneficiaries had statistically comparable levels of self-reliance, polarization, and public representation. The survey data also revealed that IDP and HCM beneficiaries had statistically comparable perceptions of their levels of polarization and public representation at endline. The gap reduction was driven by improvement in IDPs’ perceptions of these social cohesion outcomes—for example, an increase in IDPs’ public participation. Finally, beneficiaries from urban and rural locations had comparable perceptions of the level of polarization in their communities at endline. Beneficiaries from urban and rural areas both agree/strongly agree that polarization does not exist in the communities where they are residing. CONTRIBUTION TO SELF-RELIANCE AND DURABLE SOLUTIONS MRP’s results framework indicates improvement in IDPs’ and HCMs’ self-reliance as the project’s manageable impact. Self-reliance is defined as an individual’s or community’s social and economic ability to meet its essential needs in a sustainable manner. Endline survey data show that MRP beneficiaries’ self-reliance remained high despite COVID-19’s adverse effects on Mindanao household livelihoods and economic conditions. KII and FGD qualitative data corroborate these findings. For example, one beneficiary attributed the improvement in her family’s economic condition to her gains from the handicraft business she is operating out of MRP’s business recovery grant. Before receiving the grant, she did not have the means to send her sons to school. Another beneficiary expressed that their income from the livelihood grant enabled them to meet their needs at the height of the COVID-19 pandemic. The livelihood grants and training provided by MRP are one of the factors that may have influenced some IDPs’ decision to integrate locally. In addition to economic opportunities in their local communities, the grants and training provided them with platforms for interacting and working with other IDPs and HCMs. The activities MRP conducted on voters’ education and registration also formalized IDPs’ membership in their local communities. MECHANISMS TO SUSTAIN PROJECT OUTCOMES MRP has established concrete mechanisms that will likely sustain the activity’s economic and social cohesion outcomes. MRP’s sustainability strategy follows USAID’s Local Systems Framework, which highlights engagements among beneficiaries, various LGUs, government agencies, civil society organizations (CSOs), and the private sector. MRP used this model in several ways, including forging partnerships among relevant stakeholders that strengthened their buy-in to the activity; establishing sustainability policies and strategies that support grantees, qualify them for future government 11 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV support, and link them to markets; building, empowering, and formalizing CSGs to continue to serve their members after MRP ends; and promoting socially inclusive processes for both IDPs and HCMs in intervention implementation. CONCLUSIONS Relevance. The interventions that MRP implemented are relevant to IDPs’ needs and have been adaptive to the activity area’s complex environment. MRP’s application of USAID’s guidance on co￾creation processes, adaptive management, and local partnerships throughout implementation contributed to the interventions’ relevance and adaptability to beneficiary needs. The activity has significantly contributed to the ongoing efforts to rehabilitate Marawi by effectively collaborating with other actors in the activity areas and by mainstreaming efforts and activities. Effectiveness. MRP cushioned beneficiaries’ self-reliance from the adverse effects of external challenges, particularly from the economic challenges brought about by COVID-19. The business grants, community microgrants, and training received by the beneficiaries allowed them to generate income by starting businesses or engaging in gainful employment. Thus, MRP stimulated individual and community capacities to engage in economic activities. MRP’s social cohesion grants and training fostered civic engagement and community improvement among the IDPs and HCMs. It has contributed to improvements in the beneficiaries’ perceptions of social relations, connectedness, and orientation toward the common good. It has also improved IDPs’ and HCMs’ public participation. MRP’s interventions and processes offered opportunities for IDPs to integrate locally as a durable solution to their displacement. Hence, MRP accomplished the activity outcomes set forth at the outset. Sustainability. MRP has established concrete sustainability mechanisms that are essential to sustaining the activity outcomes. These mechanisms align with USAID’s Local Systems Framework. Guided by the framework’s tenets, MRP strengthened the buy-in of strategic stakeholders by harnessing partnerships among them, which resulted in institutional complementation. It laid the foundation for the CSGs to qualify for future government support and linked them to potential markets for their products. MRP laid building blocks for CSGs to continue to operate, serve their members, and even scale up their livelihoods after MRP’s conclusion by empowering and formalizing them into cooperatives and CSOs. MRP also established a social fabric for IDPs’ and HCMs’ harmonious involvement, participation, and interaction as they scale up their ventures by targeting both groups in the activity’s interventions. With these mechanisms, it is likely that the activity’s outcomes will be sustained after MRP ends. RECOMMENDATIONS The team puts forward the following recommendations based on the findings and conclusions: PLANNING, DESIGN, AND IMPLEMENTATION OF FUTURE DEVELOPMENT INTERVENTIONS 1. Future USAID development interventions, especially for complex environments, should consider using a co-creation process. Future interventions could benefit from the multidisciplinary perspectives that would be generated during the exercise. 2. Future development interventions for complex environments such as MRP could be configured with a much longer timeframe. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 12 3. The team recommends selecting the right performance indicators and clear measurements and tools to strengthen the validity of performance data and outcomes. Moreover, the targeting for each indicator across the activity’s life needs to be rationalized based on baseline assessment information, updated project implementation data, and other relevant data sources. 4. The team recommends a review of the Grant Management Manual to facilitate the quick delivery of quality interventions to beneficiaries. In complex operating environments, delays could trigger tensions among beneficiaries and other stakeholders. The evaluation team reaffirms the importance of setting up activity accountability measures that foster speedy delivery of relevant and timely interventions, especially in highly complex environments. ENSURING OUTCOME SUSTAINABILITY 1. The LGUs should continue the partnerships with the implementing partners and other CSOs in undertaking peace and development efforts in communities covered by MRP. To build on the positive results achieved by the CSGs, the LGU-CSO partnership remains critical in designing and implementing joint efforts. 2. LGUs need to follow through on their commitments to foster sustainability of economic and social cohesion outcomes. It is essential that the commitments the LGUs made during the rounds of consultation that MRP conducted be translated into binding mechanisms. Among those are commitments related to sustaining the value chain and gender-based violence initiatives. CONDUCTING FUTURE EVALUATION, RESEARCH, AND LEARNING Considering the fluid nature of the IDPs’ durable solution preference at the time of the evaluation, other development actors such as academe could research and document this durable solution and the MRP interventions’ impact on decisions. Results from these studies could help inform future interventions by USAID and other donor agencies. 13 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The evaluation examined MRP’s performance in accomplishing its outcomes and intermediate results (IRs) from September 2018 to March 2022. It also generated evidence-based learning that can help improve USAID’s design, implementation, management, monitoring, and evaluation of its development interventions by: 1. Measuring MRP’s performance in terms of relevance, effectiveness, and sustainability in achieving its target outputs and outcomes; and 2. Learning from MRP’s experiences to provide valuable recommendations for USAID/Philippines and other stakeholders in managing development projects in similar environments. The evaluation’s target audiences comprise USAID/Philippines, USAID/Regional Development Mission for Asia, USAID/Washington, Plan International, local communities, local government units (LGUs), Marawi stakeholders, local community solidarity groups (CSGs) of internally displaced persons (IDPs) and host communities, nongovernmental organizations, donor agencies (local and international), and national government agencies. EVALUATION QUESTIONS The evaluation answered the following key questions on the project’s relevance, effectiveness, and sustainability: 1. Were MRP’s development interventions relevant to IDP and host community needs and adaptive to the complex and changing external environment in Mindanao, especially in the project areas? 2. Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic activities among IDPs and host community members (HCMs)? 3. Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs? Refer to Annex 1 for the sub-evaluation questions that guided the evaluation team in gathering the data. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 14 BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT The 2017 Marawi Siege was a prolonged conflict that displaced an estimated 369,196 individuals1 who found refuge in emergency shelters or who lived with relatives or friends in surrounding communities. The siege, carried out by both national and international ISIS-inspired armed groups, worsened communities’ socioeconomic conditions by destroying property, livelihoods, and businesses. The Lanao provinces have long been areas of armed conflict among multiple groups and clans. When MRP started, these provinces were among the poorest in the country.2 The challenges faced by IDPs and HCMs are numerous and varied. These include escalating social vulnerabilities, continuing economic weaknesses, and cultural conflicts and susceptibilities. These challenges are exacerbated when IDPs and their host communities lack confidence in the local government’s ability to address the crisis. When local governments and national government agencies do not address these crises sufficiently, tensions between IDPs and HCMs may evolve into internal conflicts that violent extremist organizations (VEOs) can take advantage of to further destabilize the area. Given the historical conflicts and the continuous presence of armed groups (especially VEOs) in Mindanao, the fears and tensions among IDPs and their host communities heightened during and after the Marawi siege. The United Nations Office for the Coordination of Humanitarian Affairs (OCHA) identified the following challenges the IDPs and their host communities face in the emergency shelters and host families and communities: 3 ● High vulnerability to and risk of illnesses and malnutrition because of inadequate shelter, food, and health services. ● Increased number of out-of-school youth and children. ● Low income and high unemployment because of lost jobs and livelihoods. ● Social conflicts/tensions between host communities and IDPs. ● Lack of resources and support systems from families, neighbors, and communities. ● Limited access to livelihood opportunities and capital. ● Dependence on humanitarian assistance and host community resources. The 2019 MRP baseline study revealed significant differences in perceptions of self-reliance, polarization, and public representation between IDPs and host communities. The study also revealed significant perception differences between genders and geographical classifications (rural/urban). The baseline study highlighted the following key findings: ● Women have lower perceived self-reliance than men. ● IDPs have lower perceived self-reliance than HCMs. ● Urban residents express lower levels of trust than rural residents. ● Urban residents score public representation lower than do rural residents. ● IDPs score public representation lower than HCMs. 1 Based on estimated IDPs from 2018. Bangon Marawi Comprehensive Rehabilitation and Recovery Program. Manila, and Philippines: Humanitarian Bulletin, Issue No. 10, November 2018, Official United Nations Office for the Coordination of Humanitarian Affairs. The figure for the remaining IDPs is an estimate. 2 Philippine Statistics Authority (PSA) 2018. 3 Based on estimated IDPs from 2018. Bangon Marawi Comprehensive Rehabilitation and Recovery Program. Manila, and Philippines: Humanitarian Bulletin, Issue No. 10, November 2018, Official United Nations Office for the Coordination of Humanitarian Affairs. The figure for the remaining IDPs is an estimate. 15 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV Ending displacement is a solution to the IDPs’ situation. While waiting for this, MRP aimed to assist the IDPs and the host communities in finding ways to normalize their situations and take initial steps to prepare them to improve their lives and sustain them when they return to their homes. PROJECT DESCRIPTION The response of the USAID/Philippines Mission to the Marawi crisis began as part of USAID’s medium-term strategic goal to support the reintegration or return of IDPs through 2020, particularly for those whose likelihood of returning to Marawi seemed challenging. The following are the plan’s objectives: Phase 1: Address the early recovery needs of individuals displaced by the Marawi conflict. Phase 2: Transition IDPs to social and economic stability and establish the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs. MRP is designed to become a part of the second phase. The project became USAID’s primary Marawi response effort, integrating and building on USAID/Philippines’ initial work in responding to the Marawi crisis. Table 1 shows MRP’s summary information. As of September 2021, MRP covers the provinces of Lanao del Norte and Lanao del Sur, the cities of Iligan and Marawi, the 20 municipalities of Lanao del Sur, and the four municipalities of Lanao del Norte (Table 2). TABLE 1. SUMMARY INFORMATION ACTIVITY NAME MARAWI RESPONSE PROJECT (MRP) IMPLEMENTING PARTNER PLAN INTERNATIONAL COOPERATIVE AGREEMENT NUMBER 72049218CA00007 TOTAL ESTIMATED COST (TEC) USD$ 25,000,000 PERIOD OF PERFORMANCE SEPTEMBER 2018–SEPTEMBER 2021 (EXTENDED TO MARCH 2022) ACTIVE GEOGRAPHIC REGIONS LANAO DEL SUR, MARAWI CITY, LANAO DEL NORTE, AND ILIGAN CITY COUNTRY DEVELOPMENT COOPERATION STRATEGY CDCS 2016–2019 CDCS 2020–2024 DEVELOPMENT OBJECTIVE DO2: IMPROVED PEACE AND STABILITY IN CONFLICT-AFFECTED AREAS OF THE PHILIPPINES, PARTICULARLY MINDANAO USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 16 The municipalities of Lumbatan and Molando are additional sites reflected in the Year 2 and Year 3 annual reports, respectively. Tubaran is one of the original sites but not reflected in the list of project sites in the annual reports from Years 1–3. DESCRIPTION OF THE INTERVENTION AND THEORY OF CHANGE MRP aimed to solidify a “social contract” between the constituents and the local government to enhance social cohesion between IDPs and the HCMs. MRP’s development interventions should therefore help alleviate latent social conflict and promote strong social bonds among the constituents in the activity areas.4 Considering the historical context of armed conflicts in the area, MRP used a conflict-sensitive approach in implementing its interventions without weakening the activity’s ability to assist, empower, and unite the IDPs and host communities to work together in identifying their problems and implementing courses of action that could usher them toward more durable solutions. The project’s theory of change (TOC) states that: If IDPs, returnees, and host populations actively participate in and contribute to the social and economic development of their communities in a manner that reduces inter- and intracommunity tensions and fosters resilience, then they will be more self-reliant and able to advance solutions to their displacement. MRP’s TOC is operationalized through the results framework (RF) shown in Figure 1. 4 Based on MRP’s Indicators Reference Sheet. TABLE 2. MRP PROJECT AREAS PROVINCE CITY/MUNICIPALITY Lanao del Norte (4) Iligan City, Baloi, Pantao Ragat, Pantar Lanao del Sur (20) Balindong, Buadiposo Buntong, Bubong, Butig, Ditsaan Ramain, Kapai, Lumba Bayabao, Lumbaca Unayan, Lumbatan, Lumbayanague, Madalum, Marantao, Marawi City, Masiu, Molando, Poona Bayabao, Piagapo, Saguiaran, Tamparan, Tugaya, 17 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV Figure 1. MRP’s Results Framework MRP’s RF points to the durable solution, defined as IDPs’ local integration as its high-level impact. The activity’s manageable impact is the “self-reliance of IDPs and HCMs improved.” For this manageable impact to be realized, two mutually reinforcing IRs are identified: economic conditions of IDPs and HCMs improved (IR1) and social cohesion of IDPs and HCMs strengthened (IR2). The RF also articulates the indicators and metrics by which the activity’s performance is measured. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 18 EVALUATION METHODOLOGY EVALUATION DESIGN This evaluation employed a convergent mixed-methods approach. 5 This method facilitated the simultaneous generation of quantitative and qualitative data that are essential to the evaluation questions on relevance, effectiveness, and sustainability. It also allowed for data triangulation. The quantitative aspect involved an endline survey that generated beneficiary-level data on (1) socioeconomic indicators and (2) changes in the activity’s impact or outcome variables—namely, self-perceived self-reliance, self-perceived level of polarization, and self-perceived level of public participation. The qualitative aspect involved document review, focus group discussions (FGDs), and key informant interviews (KIIs). These methods generated primary data from beneficiaries and other relevant stakeholders. SAMPLING DESIGN The sampling frame covered the IDPs and HCMs who received MRP interventions in the 22 municipalities and two cities of Lanao del Sur and Lanao del Norte provinces. The endline survey gathered data from a total of 684 individuals from both the intervention group (n=437), which was based on approximately 17,222 CSG members, and the comparison group (n=247).6 The sample size included 10 percent allowance for possible attrition. Power analysis simulation indicated that this sample size would be able to detect at the minimum moderate effect size (0.25) at α=0.05 and 1- β=0.95 using common group comparison tests involving two groups. Moreover, the sample size was within a 95 percent confidence interval and 5 percent margin of error. A total of 124 beneficiaries and stakeholders participated in the FGDs (n=87) and KIIs (n=37). The participants were purposively selected with implementing partners’ assistance. Annex 3 shows the sample selection criteria and process. Annex 4 shows the list of participants. DATA COLLECTION After coordinating with the LGUs and barangay local government units (BLGUs) of the data gathering sites, the fieldwork commenced. The endline survey was conducted with the involvement of 18 field enumerators who had prior data gathering experience and were familiar with the activity’s geographical scope and context. The enumerators were trained to conduct the endline survey. Annex 5 shows the training design the team employed. The tool is composed primarily of the survey items fielded during the baseline study. The evaluation team added questions to gather data about beneficiaries’ perceptions of MRP’s relevance and sustainability and IDPs’ perceptions about durable solutions to their displacement. The tool was pilot tested before the actual survey. Annex 6 contains the pilot test design and administration guide. The team used KOBO toolbox in administering and managing the survey process. Refer to Annex 7 for the endline survey tool used for data gathering. The evaluation team developed FGD and KII guides to capture narrative data on MRP interventions’ relevance, effectiveness, and sustainability. The guides also included questions on lessons and 5 Creswell, J. (2012). Research Design: Qualitative, Quantitative, and Mixed-Methods Approaches. Los Angeles: Sage Publications, Inc. 6 This sample size is an expanded version of the baseline which had n(intervention)=354 and n(comparison)=44. 19 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV recommendations relevant to MRP’s implementation. Team members conducted the FGDs and KIIs using a blended approach. Those involving beneficiaries were done face to face. Personnel from the implementing partners helped gather the participants onsite, and selected enumerators assisted the evaluation team in documenting the discussion and interview proceedings. The KIIs with other stakeholders were conducted online using the Google Meet platform. On average, the discussions and the interviews lasted for about 1.25 hours. Refer to Annex 8 for the FGD and KII tools and Annex 9 for the consent forms. DATA ANALYSIS The evaluation team analyzed the quantitative data using descriptive and inferential statistical tools. The means, standard deviations, frequency distribution, and percentage distribution of the pertinent survey variables were generated. The team conducted panel data analysis to determine the improvement in perceived self-reliance, polarization, and public participation among MRP beneficiaries. The team used T-test for independent samples to compare the data from MRP beneficiaries with data from the comparison group and performed graphical renditions of quantitative results for selected activity outcomes. The evaluation team used the JASP statistical package7 and Microsoft Excel in performing the quantitative analyses. The team used qualitative approaches in analyzing the data from FGDs, KIIs, and document reviews. The FGD and KII audio and video recordings were first transcribed. The team then content analyzed the transcriptions and the activity documents. The main response categories or codes related to the (1) evaluation criteria—namely, effectiveness, relevance, and sustainability and (2) activity impact/outcome—namely, socioeconomic, and social cohesion. METHODOLOGICAL LIMITATIONS The team had difficulty involving all the baseline respondents for the end-line survey due to contamination, difficulty establishing contact, and refusal to participate. These challenges significantly affected the baseline comparison group’s already small sample size.8 As a result, the team found it methodologically not feasible to perform statistical analyses involving the original comparison group. Some of the online KIIs with MRP stakeholders did not push through because of difficulty in establishing contact, difficulty in coming up with a common schedule, and failure to join during the scheduled online interview. Moreover, the team encountered difficulty in transcribing some portions of the audio and video recordings of selected KIIs because internet connection instability made some participants’ contributions unintelligible. The team also encountered constraints in probing the activity’s effectiveness given that its evaluation scope did not include the efficiency criterion. Hence, effectiveness concerns that involved facets of efficiency were not probed methodologically in this evaluation. 7 JASP was downloaded free from https://jasp-stats.org/download/ 8 The original baseline sample size for the comparison was 44. Out of these, only 11 were traced during the endline. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 20 FINDINGS The evaluation findings are presented and discussed according to the three evaluation criteria. Whenever relevant, the findings under each criterion are grouped according to the impact and outcomes defined in the MRP RF presented in Figure 1. RELEVANCE RELEVANCE TO IDP AND HCM NEEDS Co-creation process MRP is the first USAID/Philippines development intervention that underwent a co-creation process. USAID’s principle of co-creation as a design approach aims to bring people together to collectively produce a mutually valued outcome that involves a participatory process assuming some degree of shared power and decision making. 9 This is affirmed by the following KII narratives: The co-creation process brought together various stakeholders with multidisciplinary expertise and experience. Most importantly, ECOWEB and MARADECA, who are our implementing partners, were part of the process. Their experience in implementing development programs in the area was captured in the process. (KII, COP, Plan International) I found the exercise useful in terms of understanding the interventions that Plan International set out to do to address the problem set for MRP. On the other side, the expectations of USAID as a partner of Plan International and their cooperative agreement. So, it was a very useful exercise to really define the scope of work of MRP. (KII, Ria Orca, USAID) The co-creation process served as a foundation for ensuring that interventions were relevant to IDP and HCM needs. The process resulted in the RF shown in Figure 1. Moreover, interview data indicate that the co-creation process influenced the activity design decision of targeting both IDPs and HCMs. ADAPTATIVE MANAGEMENT MRP’s flexibility in programming and packaging interventions reflects USAID’s principle of adaptive management especially when operating in an unstable and complex area. Its programming shows that it functions within an “enabling environment that encourages the design of more flexible programs, promotes intentional learning, minimizes obstacles to modifying programs, and creates incentives for learning and adaptive management.”10 The following KII response highlights MRP’s flexibility and adaptability: For this particular assistance, adaptive management was essential. For example, we recognize the color of money available. If the money comes from the stream of conflict mitigation, we develop assistance along that stream. It influences target setting. But, year on year it’s demonstrable that the targets are met. (KII, MERL Lead, Plan International) By being flexible in framing and setting targets, MRP could reconfigure its interventions to meet beneficiary needs at given junctures of its implementation. For example, Indicator 10, which was originally framed in the activity monitoring, evaluation, and learning plan (AMELP) as the “number of displaced/HCMs who benefited from social cohesion grants,” was expanded in the succeeding years 9 USAID, Co-Creation: An Interactive Guide, p. 2. 10 USAID, Program Cycle Operational Policy, p. 13. 21 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV to include additional sub-indicators. By so doing, MRP tapped additional funds from available funding streams from USAID. An example of this is the additional funding for MRP’s COVID Quick Response. This funding contributed to the activity’s continuing accomplishments even at the height of the pandemic. This aspect of the activity is illustrated by the following KII response: In 2020, soon after the pandemic erupted, we saw that municipalities hosting larger numbers of IDP families have higher rates of COVID-19. So, we made a case for $1.2 million COVID￾19 response funds. That is another color of money adaptation. (KII, Ria Orca, USAID) Table 3 shows a sample of activities that MRP conducted to make its programming more adaptive and responsive to IDP/HCM needs. Refer to Annex 10 for an expanded list. COLLABORATION WITH LOCAL PARTNERS MRP adapted intervention planning and implementation by regularly engaging local political, traditional, and religious leaders to resolve issues identified during implementation. The following interview excerpts from MRP stakeholders demonstrate this: What’s good about MRP is that from the start we already talked. In fact, during the early stages, they always give me updates. Until now, we consider Plan International as a significant partner especially on business and livelihood. (KII, Field Office Manager, Task Force Bangon Marawi) TABLE 3. SAMPLE ACTIVITIES THAT CONTRIBUTED TO MRP’S ADAPTABILITY AND RESPONSIVENESS ACTIVITY YEAR ECOWEB, MARADECA SAMPLE OUTPUTS Pause-and-reflect session 2019 BARMM Youth Identification of key challenges related to the grant process Assessment of out-of-school youth (OSY) in BARMM 2020 ECOWEB, MARADECA, Graduate students from Johns Hopkins’ University Youth’s perceptions of workforce opportunities and knowledge of CSO activities for OSY Conflict assessment study 2021 118 CSGs Recommendation of social cohesion activities to limit any further social cleavages Policy formulation and advocacy planning sessions 2022 LGUs of Marantao, Baloi, Iligan, Butig, Balindong, Pantao Ragat, Madalum, Tubaran, Pantar, Lumba Bayabao, Ditsaan Ramain, Poona Bayabao, Buadiposo Buntong, Tugaya Finalization of CSGs’ organizational vision, mission, and advocacy agenda; Sample policy instruments like advocacy plans MLGU consultation meetings on project sustainability 2021- 2022 ECOWEB, MARADECA Agreements with LGUs to sustain the project outcomes USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 22 MRP has helped a lot in providing programs for the IDPs and HCMs. They did a great job of addressing the needs of the beneficiaries. They worked with us closely on the implementation of the programs. (KII Buadiposo Buntong, LGU MPDC) MRP is unique in its interventions as it provided non-cash assistance to the activity beneficiaries. KII data suggest that one of the inputs that contributed to this assistance emanated from a prominent government leader who was consulted at the outset. The evaluation team deems that the foregoing findings helped address the baseline recommendation, which encouraged MRP to engage the LGUs during the pre-entry stage of social cohesion activities to strengthen the relationships among IDPs, HCMs, and barangays. Box 1 (on the right) contains a story shared by a municipal administrator about how MRP harnessed its relationship with their LGU. BENEFICIARY STORY OF RELEVANCE MRP beneficiaries shared stories about how the interventions they received from MRP ameliorated their situation and helped meet their needs, especially during the height of the pandemic. The following KII excerpts portray this: The in-kind grant from MRP was really a big help to my pharmacy business. It was also timely for the sanitary needs of my customers during the time of the COVID-19. (KII, business recovery grant [BRG] Grantee, Marawi City) For me, the grant that MRP provided to us was appropriate because it also suited our ability as women. In particular, the sewing machine that they gave us helped women in our area to earn income. (KII, IDP Leader, Ditsaan Ramain) RELEVANCE TO SOCIAL INCLUSION ISSUES MRP has implemented mechanisms to address social inclusion issues. One of the key mechanisms that MRP employed to address barriers and opportunities to women’s participation in economic and social cohesion activities was the establishment of gender specific CSGs. Around 278 all-women CSGs have been established. These CSGs played significant roles in community conversations, prioritization workshops, and participatory conflict assessments, which informed MRP activity design and implementation. BOX 1. HARNESSING LGU PARTNERSHIP FOR RELEVANCE, EFFECTIVENESS, AND SUSTAINABILITY The Municipality of Bubong, Lanao del Sur, is a second-class municipality with a population of 26,514. At the height of the Marawi Siege, Nabillah R.H. Abdulhakim served as the Municipal Administrator and worked closely with MRP’s implementing partner, MARADECA, which has been a partner of the local government in peace and development efforts for many years. She recognized as important MRP’s approach in targeting both IDPs and HCMs. Likewise, she lauded how MRP went down to the communities and asked people about their priority needs. “MRP asked people about their needs. The women expressed they needed support for dressmaking to earn income. During the graduation ceremony, I saw the products of women beneficiaries. The women worked hard and committed to supporting their families.” She strongly believes in the importance of sustaining the gains from MRP by strengthening the CSGs and increasing their participation and voice in LGU processes. “We are helping the youth and women federations in their accreditation in the municipality. They already have by-laws and policies which are the requirements for accreditation. They can become members of our local special bodies. The women can be registered as cooperatives and the youth can be accredited by National Youth Commission. This is important for their representation.” She has only great words for MRP and to Maradeca and looks forward to continuing the partnerships in implementing other relevant interventions and in providing more durable solutions. both to the IDPs and host communities in the municipality. 23 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV Another mechanism was the development of community champions from among the youth, women, and religious leaders. The community-based champions’ efforts helped shape the decision making and programming of the target barangays in addressing child protection and gender-based violence (GBV) issues.11 In total, MRP trained 82 CSG champions who also organized GBV community rollout sessions reaching 879 CSG members. Furthermore, document review data suggest that 51 percent of beneficiaries in Year 3 are women. Currently, 43 percent of the CSGs are all-women and 59 percent are led by women. These figures show that women have been taking the lead in most of MRP’s major activities. Although participation in activities does not immediately translate into women’s empowerment, such involvement forms an essential initial step in confidence building. Table 4 shows the activity’s performance on gender￾related accomplishments for Year 3.12 Despite high rates of participation by women in MRP, the conditions women struggle with persist. Document review data indicate that women still confront the double burden of working to earn a living and performing domestic work. GBV also persists. Moreover, project reports indicate that the gender equality champions MRP developed still face the challenge of a prevailing culture of silence and pressure for women to go along and accept amicable settlements.13 However, MRP’s continued implementation of GBV-related interventions during the pandemic addressed the noted rise of GBV during this period. This rise has been highlighted by the UN Women organization (2021) and explained as follows: Gender-based violence, already a global crisis before the pandemic, has intensified since the outbreak of COVID-19. Lockdowns and other mobility restrictions have left many women trapped with their abusers, isolated from social contact and support networks. Increased economic precarity has further limited many women’s ability to leave abusive situations.14 The following narratives attest to MRP’s efforts to address women’s needs, including those related to GBV: Before MRP, only my husband had a source of income. It was hard. With the livelihood support I got from MRP I am earning Php 300.00-500.00. I used it for medicine and food. 11 January-March 2022 Quarterly Report, p. 45. 12 Year 3 Annual Report, p. 47. 13 Year 3 Annual Report, p. 70. 14 UN Women Organization (2021). Gender-Based Violence: Women and Girls at Risk. Retrieved from https://www.unwomen.org/en/hq-complex-page/covid-19-rebuilding-for-resilience/gender-based-violence) TABLE 4. WOMEN’S PARTICIPATION IN MRP ACTIVITIES GENDER INDICATORS NUMBER OF WOMEN PARTICIPANTS Women trained in life and technical skills 972 Women business recovery grant recipients 1,032 IDPs/HCMs trained on civic engagement, peace narratives or peace building, community score card, and gender training 2,953 USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 24 This seems not that much, but it has helped a lot. (FGD, Women CSG Member, Buadiposo Buntong)" In the last two years of MRP, we ramped up social cohesion, COVID response, and GBV. GBV funds came very timely because of COVID, and we saw an uptick in GBV. The pandemic restrictions affected the GBV rates negatively. So, GBV increased. It was very timely for us because we have convinced the local government partners that, other than the health risks, one of the harmful consequences of COVID is the rising cases of gender-based violence. So, it came very timely for us. We have been opportunistic, Plan International and us. (KII, Ria Orca, USAID) The story of an all-women CSG member in Box 2 shows how the grant they received from MRP empowered them to meet their families’ economic needs. COMMUNITY SOLIDARITY GROUPS’ IMPORTANCE The team found that the CSGs, the primary mechanism through which the MRP solicits grant proposals, were pivotal in addressing IDP and HCM needs.15 CSG members’ role in identifying and prioritizing community needs, suggesting opportunities for collaborating to address these needs, and pointing out potential community tensions is essential. The MRP team together with the local consortium partners (LCPs) identified required activities from pre-community entry to grant proposal submission. 16 The participatory processes the LCPs facilitated served as opportunities to strengthen CSG member relationships. The CSGs are thus helping build social cohesion and a sense of common understanding among members. From September 2018 to March 2022, MRP organized 665 CSGs. Of the total, 552 are intact and 113 have dissolved. (Refer to Annex 11for the community engagement and grant application cycle.) The evaluation team identified models of successful and sustainable CSGs as follows: • Active CSGs that have received grants and have successfully implemented their projects. 15 Year 1 Annual Report, p. 22. 16 Sept 1–Dec 31 Quarterly Report, p. 28. BOX 2. FROM STRUGGLE TO EMPOWERMENT: STORY OF WOMEN’S RESILIENCE Rainidah S. Macadato evacuated in 2017 to nearby Bagoinged Ditsaan Ramain, Lanao del Sur, to flee the bitter clashes in Barangay Marinaut West, Marawi City. She remembers that it was extremely difficult at first. They lived in the gym with other displaced families they hardly knew. Buy they needed to find ways to survive and relied on the help of the government and other groups. In September 2019, Rainidah joined the all-women community solidarity group, Newtonians Women’s Dressmaking, that the local implementing partner, Maradeca, had organized in their barangay. She attended many organizational meetings in preparation for the community microgrant. She was designated CSG Treasurer. In December 2020, they were happy to receive their livelihood support from MRP, which included 33 sewing machines and fabric. “For me, the grant that MRP provided to us was appropriate because it also suited our ability as women. In particular, the sewing machine that they gave us helped women in our area to earn income.” “The hijab we made from the fabric earned us Php19,950.00. So far, our profit has reached Php40,000.00. All members received their share. We are now on our fifth cycle.” While Rainidah and other members of the women’s CSG have been thankful to MRP, they also recognize the importance of setting up policies to ensure the sustainability of their operations. “We develop policies on how to manage our investment and income. We set some rules on the percentage sharing of income between those who are involved and the rest of the CSG members. Twenty percent of the income will go back to our capital investment, and 10 percent is also shared with all members even if they are not working on the sewing machines. This way they continue to participate in our CSG. We believe we need to do this to sustain what we received from MRP. Through this, we hope to increase our operations.” 25 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV • Active CSGs that have received grants even if they were not successful in implementing their projects. • Active CSGs that are yet to receive their grants but remain hopeful to eventually receive them. • Active CSGs even with disapproved grants and with no chance to receive them. The factors contributing to these models include established community leadership, established social relationships, level of group maturity, socially cohesive local environments, and inherent and acquired skills in managing their projects. Examples of unsuccessful CSGs include the following: • Dissolved CSGs even after receiving grants and successfully implementing projects, which happened mostly in urban areas. • Dissolved and frustrated CSGs that have not received any grants. • Dissolved CSGs were not able to successfully implement projects for various reasons, including lack of management skills. Document review data show several reasons for CSG dissolution: presence of rido (feud between families/clans) or conflict in the area; CSG members’/leaders’ loss of interest and desire to discontinue their participation; CSG declaring itself dissolved after dividing grants equally and no longer meeting as a group; CSG members returning to Marawi and/or transferring to other places for livelihood/work reasons; or inability to contact/trace CSG members.17 The narratives below highlight the value of the CSGs based on the perspective of the selected stakeholders: I like MRP’s approach to CSG organizing. They included both IDPs and the members of the host communities. I believe that such an organization is appropriate for socioeconomic intervention. (KII, Asst. Regional Director, DTI, Lanao del Norte) "We did not experience conflicts in our group. The members of our CSG are also enjoying our aquaculture project because, like here, every time we clean the fishpond, we have a picnic, and we cook. So, our CSG project also served as avenue for everyone to have bonding. (FGD, CSG Member, Balindong)" COMPLEMENTATION OF PHILIPPINE GOVERNMENT RESPONSE PRIORITIES MRP’s interventions complemented the functions of Task Force Bangon Marawi (TFBM). The interventions are particularly relevant to TFBM’s role in providing an environment conducive to the revival of business and livelihood activities, as stated in Administrative Order No.3 s.2017.18 MRP has been pivotal in fostering economic recovery by taking an active role as a member of TFBM’s Subcommittee on Business and Livelihood. Moreover, MRP contributed to the formulation and implementation of the TFBM’s Sustainability Plan for Scaling Up Business and Livelihood of Marawi IDPs. This plan serves as a platform to scale up 17 January–March 2022 Quarterly Report, p. 42. 18 Official Gazette (2017). Administrative Order No. 03, s. 2017. Creating an Inter-Agency Task Force for the Recovery, Reconstruction and Rehabilitation of the City of Marawi and Other Affected Localities, downloaded July 2022 from https://www.officialgazette.gov.ph/downloads/2017/06jun/20170628-AO-3-RRD.pdf USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 26 interventions relating to business and livelihood and ensure their sustainability19. The following KII narratives provide insights on how MRP complemented government programs: The first convergence of DTI with MRP was on the legitimization of the business that includes registration of business names, input/lectures from DTI onwards to securing business permits. (KII, Asst. Regional Director, DTI, Lanao del Norte) I always say to the people that infrastructure assistance from the government will end. You [IDPs and HCMs] are provided with livelihood assistance so you will not depend on government support. It fits the context because Maranaos are business oriented. (KII, Field Office Manager, Task Force Bangon Marawi) Moreover, MRP’s business and livelihood grants, which benefited several cooperatives in Lanao del Sur and Norte with IDP and HCM members, are in harmony with the government project dubbed “Marawi Rehabilitation through Cooperativism Project.” The Cooperative Development Authority (CDA) under the aegis of TFBM implemented the project in 2018. The team found that MRP contributed to this project’s goal of reactivating the cooperatives in Marawi City to provide business and livelihood opportunities to the residents20. MRP also helped various LGUs address COVID-19 through its COVID Quick Response initiative at the pandemic’s onset. It provided COVID-19 essentials (e.g., PPE, masks, thermal scanners, and disinfectants) and conducted information, education, and communication activities and capacity￾building and technical training (e.g., treatment facility management and specimen collection and handling)21. These contributions are corroborated by the KII narrative shown below: From the start of the pandemic, MRP actively coordinated with the Inter Agency Task Forces (IATF) of the different LGUs. Specific attention was given to participation in the Lanao del Sur (LDS) IATF as it covered the majority of MRP project areas. MRP was invited and participated in a number of the LDS IATF meetings. (KII, COP Plan International; AR 2020, p.32) Table 5 shows a sample of the collaborative activities between government agencies and MRP. Annex 12 presents an expanded list. 19 TFBM Subcommittee on Business and Livelihood (2021). Sustainability Plan for Scaling Up Business and Livelihood of Marawi IDPs. 20 CDA-TFBM Magazine (2018). Rising from the Ashes: The Rise of Coops in Marawi After the War. 21 Based on Annual Report 2020, pp.33–36. TABLE 5. SAMPLE OF COLLABORATIVE ACTIVITIES THAT MRP CONDUCTED AGENCY INVOLVED NATURE OF COLLABORATION YEAR Task Force Bangon Marawi (TFBM) TFBM and DTI furnished MRP with the official list of businesses located in the MAA prior to the siege. The information greatly facilitated the processing of BRGs. 2019 Technical Education and Skills Development Agency (TESDA) TESDA provided MRP with the list of recommended Technical Vocational Institutes (TVIs) and assisted in providing technical training to livelihood grant recipients. 2019 27 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV RELEVANCE TO USAID’S DEVELOPMENT STRATEGY USAID’s 2013–2018 Country Development Cooperation Strategy (CDCS) Development Objective 1: Broad-Based and Inclusive Growth Accelerated and Sustained Development Objective 2: Peace and Stability in Conflict-Affected Areas in Mindanao Improved Development Objective 3: Environment Resilience Improved MRP is relevant to USAID’s 2013–2018 (extended through 2019) Country Development Cooperation Strategy (CDCS). It is especially relevant to the objective of improving peace and stability in conflict-affected areas, particularly in Mindanao (Development Objective 2). MRP helped address the objectives of transitioning IDPs to social and economic stability (Objective 2) and establishing the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs (Objective 3). USAID’s 2019–2024 Country Development Cooperation Strategy (CDCS) Development Objective 1: Democratic Governance Strengthened Development Objective 2: Inclusive, Market-Driven Growth Expanded Development Objective 3: Environmental and Community Resilience Enhanced The positive changes in the social cohesion indicators (i.e., polarization and public participation) as shown in the endline survey results helped achieve the objective of strengthening democratic governance (Development Objective 1) in the 2019–2024 CDCS. In particular, the team found that MRP’s interventions are relevant to the realization of building more responsive local governance (IR 1.4) considering that “USAID anticipates that during this CDCS, civil society will become more engaged, better networked, and more inclusive as they connect across regions and sectors.” 22 The noted positive changes in social cohesion support the enhancement of environmental and community resilience (Development Objective 3), particularly in increasing social connectedness (IR 3.3.2). These findings support USAID’s belief and approach that “increasing social connectedness and social cohesion is one of a set of responses to strengthen community resilience. USAID will work through civil society to build formal and informal networks that include populations vulnerable to such threats and tie together areas affected by shocks and disasters, including violence and conflict.” 22 USAID PHILIPPINES CDCS 2020–2024 Draft September 18, 2019 – revised October 28, 2019. Lanao and Regional Cluster of the Joint Child Protection and Gender-Based Violence (JCPGBV) Working Group MRP can highlight the project’s gender integration efforts to BARMM and coordinate with other members of the working group for possible collaboration and complementation with JCPGBV. 2020 Department of Trade and Industry (DTI), Bureau of Fisheries & Aquatic Resources (BFAR), Provincial Agriculture and Provincial Technology & Livelihood Development Offices These agencies helped increase the number of BRG referrals. With this, MRP expanded its roster of BRG recipients to include agricultural cooperatives, women’s cooperatives, and agribusiness enterprises. 2020 M/BLGU representatives and religious leaders from the municipalities of Balindong and Lumba Bayabao, and Pantar Municipal and barangay LGU representatives and religious leaders were included in the TOT for community engagement and the CSC process in its workforce readiness interventions. 2021 Department of Interior and Local Government (DILG), BARMM Commission on Elections (COMELEC) The DILG and BARMM Commission on Elections served as resource persons during the MRP-YouthLed Voter’s Education Training of Trainers. 2021 USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 28 The activity’s economic outcomes are essential for expanding inclusive, market-driven growth (Development Objective 2), particularly in improving private enterprise development (IR 2.4.1). MRP’s approach of tapping cooperatives in its business and livelihood grants is also consistent with USAID’s perspective: “Cooperatives play a critical role in addressing livelihood challenges, especially in rural areas, and benefit host communities through various social initiatives. A huge potential exists for small and medium-sized enterprises as development partners.” EFFECTIVENESS IMPROVING IDPS‘ AND HCMS’ ECONOMIC CONDITIONS Document review data on MRP’s performance on IR1 indicators suggest that the activity has helped 1,963 business owners start a new business or restart their previous businesses. A total of 3,117 individuals benefited from BRGs. MRP has provided livelihood opportunities to 8,350 IDPs and HCMs through its livelihood microgrants. Moreover, 1,336 business owners benefited from enterprise management training and workshops. 23 The team examined income- and employment-related variables in the endline survey to understand how MRP’s outcomes and output indicators translate into improvement of beneficiaries’ economic conditions. Income-related variables The team found that the proportion of endline survey participants who reported having sufficient income to meet their families’ basic needs (e.g., food, medicine, children’s education) was higher among participants who received MRP interventions (44.5 percent) than among participants from the comparison group (38.0 percent) (Figure 2). In the same manner, the proportion of those who reported having savings was higher among survey participants who received MRP interventions (15.3 percent) than among the comparison group (11.3 percent). Figure 2. Percentage of Intervention and Comparison Participants with Savings and Sufficient Income The team further found that the proportion of MRP beneficiaries who indicated having sufficient income (see Figure 2) was higher than among the general population in the Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) (19.0 percent). 24 Moreover, MRP beneficiaries’ economic conditions, as discussed above, provide a positive picture compared with that of the regional economic landscape during the pandemic. Government reports suggest that the number of 23 Refer to Annex 13 for the performance of MRP in relation to the indicators. 24 Ishikawa, I. (2021). Impact of COVID-19 on Peace and Stability in Mindanao: Nested Protection and Downside Risks. JICA Ogata Sadako Research Institute for Peace and Development. 29 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV households receiving salaries had declined and that the incomes from farms and businesses in the region had plummeted. 25 The evaluation team also looked at MRP beneficiaries’ and the comparison group’s sources of household income (Figure 3). More beneficiaries than non-beneficiaries were engaged in income￾generating activities such as crop farming (64.5 percent) and livestock farming (56.5 percent), non￾food retailing such as RTW and textile (76.0 percent), food retailing or sari-sari store (73.5 percent), and paid work (61.2 percent). Figure 3. Percentage of Intervention and Comparison Participants Across Sources of Household Income Box 3 contains the story of a couple whose dressmaking business was devastated by the Marawi siege. Their story shows how MRP’s grant, which provided them with sewing machines, helped their family surmount the challenges brought about by the pandemic. 25 Mindanao Development Authority (2022). COVID-19 Socioeconomic Impact Assessment: Strengthening Recovery Towards Peace and Development in the Six Regions. 35.5% 43.5% 24.0% 38.8% 33.3% 26.5% 44.4% 16.7% 64.5% 56.5% 76.0% 61.2% 66.7% 73.5% 55.6% 83.3% Farming crops Farming livestock/fishing Non-food retailing (e.g., RTW, textile) Paid work (including wages and pay for… Remittance Sari-sari store/food retailing 4Ps Donor assistance (e.g., from an aid agency, a… Sources of Household Income Comparison Intervention BOX 3. SURVIVING THE AFTERMATH OF THE MARAWI SIEGE PUNCTUATED BY THE COVID-19 PANDEMIC. (During the Marawi siege, this Christian IDP couple from Marawi City evacuated to Iligan City. As entrepreneurs, master cutters, and makers/tailors of wedding gowns and suits, they had a space near Padian Market in Marawi City and have Meranaos customers. Now in Iligan City they continue their trade in a much smaller space and continue to have Meranao customers from Marawi City.) We grew up in Marawi. Before the siege, we had 12 dressmakers and tailors, and we were renting a two-story building and installed 12 heavy-duty sewing machines and two zig-zaggers. In Marawi, we both were just cutting the patterns and our hired dressmakers and tailors were the ones doing the sewing. Business was very good with our Meranao customers who preferred our services as they kept having repeat orders. We prefer to live in Marawi since we have adjusted well there. It took a year when we were allowed to go back to Marawi. All our machines were destroyed, clothing materials scattered, and the P300,000 cash we kept at our shop was gone. Everything was wasted by the siege. That was not easy for us to accept. The government gave us cash assistance of P73,000 for the industrial sewing machine that we bought and used now along with the zig-zagger and clothing materials. Meanwhile, as members of CSG, MRP gave me and my daughter sewing machines, clothing materials, and other sewing paraphernalia. These are now part of our present business. MRP’s assistance was good timing: at one point during the pandemic, the sales from the face masks we were sewing was our main income. Our income kept our lives going during that very challenging time. In addition, many of our long￾time customers, our contemporaries when we were children, also assisted us with cash to help us go on with our sewing business, something we never expected they would do. The building owner in Marawi City where we had our sewing business asked [whether] we are intending to go back, and we said yes. Thus, we are waiting for the time and the opportunity so we can continue our sewing business in the same location in Marawi, where we grew up. We hope the time is soon. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 30 Employment-related variables The endline survey data shown in Figure 4 suggest that more MRP beneficiaries than comparison group members indicated that (1) they are engaged in work/livelihood that pays them money (55 percent), (2) they have the right skills to find a job (80 percent), and (3) they are confident in finding a job in case they would lose their present one (59 percent). An inferential analysis suggests that the proportion of MRP beneficiaries is significantly higher in terms of engagement in paid work/livelihood (p=0.024) and having the right skills to find a job (p=0.000). These findings imply that MRP’s interventions in the form of business/livelihood grants and training workshops have expanded IDP and HCM employment opportunities. Figure 4. Percentage of Intervention and Comparison Participants Across Employment-Related Variables The Mindanao Development Authority’s assessment of COVID-19’s impact, particularly the findings on BARMM, indicated that “BARMM has geographically dispersed provinces that makes the movement of people and goods time-consuming, difficult, and costly. The limited movements of people and goods that ensued have ultimately prevented the growth of the regional economies in Mindanao.” 26 The following narratives from MRP beneficiaries highlight the economic relief the activity has afforded: "It's different Sir, because before we operated the bakery, we didn't have much income. Then when we got a bakery, it helped us meet the needs of our family. During special events like graduation a lot of people order from us. We are sharing his profit. At first, we were able to divide P500 per member. We have increased our capacity to buy food for our family. (KII, CSG Member, Baloi)" I have seen during beneficial monitoring that beneficiaries have improved their houses. Those who engaged in selling groceries have available inventory in their sari-sari stores. (KII, MERL, Plan International) MRP’s economic outcomes are important considering that the COVID-19 pandemic has resulted not only in a health and life crisis but also a deep economic recession. In fact, the Philippines experienced a −8.3 percent gross domestic product growth rate in the fourth quarter of 2020 and a −9.5 percent rate for the entire year. 27 The employment-related outcomes discussed above indicate that MRP 26 Mindanao Development Authority (2022). COVID-19 Socioeconomic Impact Assessment: Strengthening Recovery Towards Peace and Development in the Six Regions. 27 Philippine Statistics Authority (2021). Philippine GDP posts -8.3% in the fourth quarter of 2020; -9.5% for full year of 2020. Retrieved October 2022 from https://psa.gov.ph/content/philippine-gdp-posts-83-percent-fourth-quarter-2020-95- percent-full-year-2020 31 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV addressed to some extent beneficiaries’ socioeconomic needs brought about by the Marawi crisis as well as the pandemic. This aligns with the JICA (2020) findings that jobs and resumption of agricultural activities are among the potential COVID-19 recovery measures that BARMM constituents indicated.28 The discussions above imply that MRP’s business and livelihood interventions have stimulated economic activity among the beneficiaries amid the pandemic’s economic impact. In particular, the in-kind grants and training that MRP provided its beneficiaries have improved their economic conditions. Considering that COVID-19 resulted in widespread loss of livelihood and rise in poverty levels among displaced populations,29 these results show that MRP provided substantial economic relief to the beneficiaries. The following KII excerpts indicate how the trainings and in-kind grants helped beneficiaries: The members of our CSG have a chance to benefit from the projects we received from MRP. And they received training that helped them earn a living. (KII, HCM leader, Women CSG, Buadiposo Buntong) MRP has helped us through the provisions of sewing machines and training. The income we derived from our products have supported our families’ daily needs, especially for the school expenses of our children. (FGD, Women CSG Members, Marantao) Box 4 shows the experience of a family who became IDPs because of the siege. They traveled as far as Cebu to seek recovery. Their sojourn back to Iligan proved to be beneficial as they received assistance from the government and from the MRP. It helped them eventually commence their journey toward recovery. STRENGTHENING SOCIAL COHESION OF IDPS AND HCMS MRP’s IR2 focuses on strengthening IDPs’ and HCMs’ social cohesion. MRP’s AMELP indicates that social cohesion refers to two intertwined features of society: absence of latent social conflict and presence of strong social bonds. MRP focuses on areas where there is a significant concentration of 28 JICA (2020). https://www.jica.go.jp/philippine/english/office/topics/news/200907.html 29 Khunt and Schuttler (2020). Economic Integration Into Host Communities in Times of Crisis: How to Ease the Impact of COVID-19 on Displaced Populations in Low- and Middle-Income Countries. German Institute for Development Evaluation. BOX 4. CONFRONTING BUSINESS RECOVERY AS IDPS (The Marawi siege took place three days before the start of Ramadan. As a practice, a 30-day food provision was stocked at every Muslim household since unnecessary travel is discouraged. On the business side, Meranao entrepreneurs stock their merchandise and material inventories in preparation for the Eid al-Fit’r feast, which also brings days of “buying sprees” among customers.) We are a wholesaler, so we usually keep our inventory level every three days. Our wholesale mobile phone business was doing very good. At the time of the siege, our family’s business inventory in Marawi was about PHP4 million. We evacuated to our hometown in Mulondo to spend the Ramadan. We tried our luck in Cebu City. We engaged in the night market selling mobile phones and gadgets. We gave it up after three months as the income was not enough for our daily provisions and to pay the rentals. We also had a problem with the landlord who [would] not accept IDPs as apartment renters. We left Cebu and settled in Iligan City, where we received cash assistance of P73,000 from the government. Later, my textile businessman uncle and I jointly received P200,000 worth of textile from the MRP. We chose textile because the release was faster. MRP found difficulties in procuring mobile phones and gadgets for my business. I eventually sold my textile share of inventory to my uncle and continued with my mobile phone business. Despite the general dislike of the Iligan residents to the Marawi IDPs, my business continues to expand from one stall (2m×3m), I now have the adjacent three stalls. Our income is now covering our family’s needs. We have five children. I got married at 13 and I now have a college-graduating daughter. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 32 IDPs from the Marawi siege and intends to provide venues to reduce if not eliminate polarization between the displaced and their host communities. CONTRIBUTION TO PERCEPTIONS ABOUT POLARIZATION In the context of MRP, polarization is defined as community fragmentation in terms of trust (or lack thereof) and attitude in working together, with specific attention to relations between IDPs and HCMs. 30 Level of polarization in MRP is measured using a 4-point Likert-type scale consisting of 15 items. Overall, high scores on the polarization scale suggest positive views that the community is not polarized or fragmented. As Figure 5 shows, the team found that MRP beneficiaries’ overall perception of polarization had significantly improved at the endline (p=0.020). The significant rise in scores from the baseline (Mean=2.86) to the endline (Mean=2.99) suggests that the beneficiaries are perceiving significantly less polarization or fragmentation in the communities they are living in. This positive outcome on beneficiaries’ perceptions of polarization indicates formation of social cohesion. Moreover, several polarization indicators (e.g., differences in tribes, differences in religion) were associated with socially divisive influences, the absence of which contributes to desirable development outcomes such as growth, poverty reduction, stability, peace, and conflict resolution).31 The team observed that polarization, particularly the “IDP-HCM lens,” may need to be rethought in the context of the Marawi siege considering that most, if not all, of the IDPs took refuge among HCMs who are their relatives. The data on polarization would have been more contextual if this nuance had been captured in the survey tool. Contribution to perceptions of trust Trust among citizens correlates strongly with economic growth.32 It holds a prominent place in economics literature, where it is considered to reduce transaction costs. The evaluation team examined trust-related indicators in the polarization tool. The analysis results revealed that MRP beneficiaries’ level of trust had risen significantly at endline (p=0.046) and was significantly higher than that of the comparison group (p<0.001) (Figure 6). The effect size generated in the analysis 30 Based on MRP AMELP. 31 OECD (2011). The Social Cohesion and Policies for Enhancing Civic Participation. 32 Larsen, C.A. (2014). Social cohesion: Definition, Measurement, and Developments. Retrieved from https://www.un.org/esa/socdev/egms/docs/2014/LarsenDevelopmentinsocialcohesion.pdf Figure 5. Level of Polarization of MRP Beneficiaries During Baseline and Endline Figure 6. Level of Trust of MRP Beneficiaries During Endline 33 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV (d=0.481) suggests that MRP interventions had a medium or average contribution to the improvement in the beneficiaries’ trust level. The following trust indicators had improved at endline (Figure 7): (1) trust in people and neighbors, (2) perception of existence of differences in the community between people and tribes, and (3) trust in police and military. Figure 7. Improvement in Trust Indicators Among MRP Beneficiaries from Baseline to Endline These trust indicators relate to horizontal trust between citizens, 33 which enables societies to overcome basic challenges of collective action. 34 The improvement in these indicators is thus crucial as collective action is needed for CSGs to advocate for themselves and pursue their vision and aspirations even after MRP ends. MRP beneficiaries rated the indicator suggesting willingness to work with IDPs/HCMs on implementing projects (Item 16) relatively higher, perhaps reflecting a seed of the collective action beneficiaries need to manifest to sustain the gains from MRP interventions. The findings that MRP beneficiaries’ improved trust in the police and military supports the idea that the way to deal with risks in the society is to trust other people and the roles people play in society, like police officers, social workers, etc. 35 Trust is important for institutions such as the market, democracy, and even the state to function36. Improved trust, particularly in the police and military, is important considering the complex political and economic environment of MRP. MRP’s interventions brought IDPs and HCMs together and helped reestablish social relationships and networks. MRP's Baseline Assessment Report has also established at the onset that IDPs and HCMs have good relationships and debunked the idea about potential tensions between HCMs and IDPs. The following excerpts from KIIs with beneficiaries and stakeholders show key factors that furthered trust among IDPs and HCMs in MRP areas: We accept IDPs because they are Muslims, and Maranaos. They are IDPs now, but we experienced their situation in the past. (KII, Barangay Captain, Ditsaan Ramain) 33 Ibid. 34 Ibid. 35 Beck 1992; as cited in Larsen 2014 36 Larsen 2014 3.16 3.23 3.20 2.88 3.12 2.75 2.86 3.02 2.41 2.25 2.92 2.22 0.00 2.00 4.00 I trust the military. I trust the police. Different ethnic tribes do not divide the people… Differences do not exist between people living… Most people in the neighborhood can be… It is not naïve to trust people. Improved Trust Indicators Baseline Endline USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 34 We have no issues with the IDPs here because they are our relatives. Some of them have old houses here. (FGD, Women CSG Member, Buadiposo Buntong) Based on our experience, we don’t have conflicts between HCMs and IDPs. They work together in our community. (KII, LGU MPDC, Buadiposo Buntong) CONTRIBUTION TO PUBLIC PARTICIPATION Another component of social cohesion involves IDP/HCM civic engagement, which, regardless of the kind of political system in place, is essential for growth. 37 Providing space for dissenting voices is fundamental to the creation of a sustainable, socially cohesive society. In the context of MRP, public representation includes participation and involvement (i.e., as member or officer) of IDPs/HCMs in CSGs, local special bodies (e.g., health boards, school boards, environment boards), and local government.38 This participation was measured using a 4-point Likert-type scale that gathered data on participation and involvement in community affairs such as CSG meetings and barangay activities. Figure 8 shows that based on the endline survey, the team found that the MRP beneficiaries had a high level of participation (Mean=2.64), whereas the non￾MRP respondents had a low level (Mean=2.04). To note, the perceived level of participation scores that fall within the 2.50-3.24 range is described as “high”; scores that fall within the 1.75-2.49 range are described as “low”. This is based on the baseline protocol of score interpretation which is also used by Plan in target setting. The item-level analysis shown in Figure 9 reveals improvement in indicators related to communication with CSG and barangay leaders about community problems. This communication involved writing (Mean=1.81) or personally talking (Mean=2.44) to community leaders about problems observed in the community. There is also improvement in indicators related to participation in community decisions and training, which involved attending trainings for employment and meetings to address community problems (Mean=2.85) and community decision making (Mean=2.74). 37 OECD (2011). The Social Cohesion and Policies for Enhancing Civic Participation. 38 Based on MRP AMELP. Figure 8. Levels of Public Participation in Intervention and Comparison Groups 35 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV Figure 9. Improvement in Public Participation Indicators Among MRP Beneficiaries from Baseline to Endline These results suggest that MRP beneficiaries have improved their level of participation in community affairs, including CSG-related activities. They have also improved the extent to which they communicate with authorities and community leaders, including their CSG leaders and BLGU officers. These improvements are likely attributable to the social cohesion interventions provided by MRP to the CSGs, such as the community improvement and civic engagement grants. Through these grants CSGs received training on how to develop their advocacy agenda and how to communicate their agenda to pertinent local bodies. The following narratives corroborate the survey results above. The excerpts show that the CSGs and their members can now represent themselves in local bodies and engage government officials in their projects: The CSGs have made requests from the municipal LGUs. They start to coordinate directly with them. Compared to before, they handle this now on their own. The relationship between LGUs and CSGs is open. This paves the way for greater LGU support. (FGD, MARADECA) When there are activities, for example when the projects are received, the chairwoman and the barangay councilors are there as witnesses. The relationship is close because the whole barangay is planning, talking as a group. (KII Buadiposo Buntong, HCM leader, CSG women) In the following excerpt, a beneficiary depicts MRP’s role in promoting social cohesion among beneficiaries by fostering participation in activity implementation: I have observed that apart from the economic and financial benefit, the social interaction among us has also improved. Because without the MRP, the people here would not meet since there’s no reason for them to do so. The interaction with other people has been intensified because of the project activities. (KII, HCM leader, Women CSG, Buadiposo Buntong) The findings above could serve as an impetus for formulating effective public policies to address beneficiary (particularly IDP) needs and aspirations. On the one hand, the IDPs and HCMs must lobby for their needs and aspirations, which would require them to effectively communicate and actively participate in local bodies. On the other hand, the LGUs need to provide space to hear their USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 36 voices. This is consistent with OECD’s development perspective39: “If the society integrates minorities, has a relatively strong sense of belonging, and provides opportunities for upward mobility, the effectiveness of its public policies will obviously be greater than in socially fragmented societies.” Box 5 contains a story of a youth CSG that showed enhanced public participation by becoming part of the municipal and provincial youth federation. The CSG made a meaningful contribution to their community’s development. REDUCING SOCIAL GAPS The team also examined how MRP’s interventions have contributed to the reduction of social gaps. GENDER ANALYSIS First, the team examined interventions’ impact on women, particularly on their level of self-reliance, polarization, and public participation. Comparative analysis of survey data shown in Figure 10 suggests that the women beneficiaries had statistically comparable levels of self-reliance (p=0.205), polarization (p=0.868), and public representation (p=0.094) as did men. This means, for example, that the women were as self-reliant as the men at endline. This is an improvement from the baseline when women had significantly lower self-reliance than men. The KII narrative below tells how the livelihood grant that an all-women CSG received from MRP allowed a mother to augment her family’s income: "Personally, if I didn't join MRP, maybe I was just at home and didn't have a job. It's really a bit difficult, you know, the motor driving of my husband, you can’t always earn a lot, especially now that there are so many motors, a lot of competition. When MRP came, it was already there, as long as personally, it really helped me a lot. (KII, IDP Leader, Ditsaan Ramain)" 39 OECD (2011) Figure 10. Comparison of Women’s and Men’s Self-reliance, Polarization, and Public Participation BOX 5. COMMUNITY ENGAGEMENTS TOWARD BUILDING YOUTH FORMATION AND INCREASING PUBLIC PARTICIPATION “Our meetings served as opportunities to establish relationships between and among youth IDPs and host communities,” says Asniah Sarip, the Chairperson of Masakaw Youth, composed of IDPs and HCMs with Sangguniang Kabataan (SK) members, one of the community solidarity groups in Barangay Basingan, Bubong, Lanao del Sur. Formed in January 2020, the all-youth group CSG ventured into chicken and duck production as its livelihood project. Asniah and the members expressed gratitude for the community grant and the other interventions they received from MRP. The various CSG processes allowed them to work together to address issues affecting them and provided them with additional skills, especially in livestock and organizational management. They viewed MRP’’s assistance and support as the main catalyst for their formation. Ashniah maintains that “MRP has redirected them to acquire new sense of purpose as young people for their own welfare and towards contributing positively to their community.” The Masakaw Youth is now a member of the municipal and provincial youth federation. Asniah looks forward to more meaningful public participation of youth where they can be formally represented and their issues are heard and addressed. 37 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV The reduction in the gap between women and men in MRP outcome measures is a positive development considering empirical data suggesting that women tended to absorb the shocks brought about by the pandemic. This is pointed out by the UN Women organization: Women are being hit hard by the fallout of the pandemic. Because they typically earn less, have fewer savings, and hold less secure jobs to begin with, women are particularly susceptible to economic shocks in general. The pandemic has devastated feminized sectors like hospitality, tourism, and retail, depriving many women of their livelihoods. Across all regions, women have been more likely to drop out of the labor force during the pandemic.40 IDP/HCM ANALYSIS The team also compared the endline survey data on IDPs and HCMs. The data showed that IDP and HCM beneficiaries had statistically comparable self-perceptions of their levels of polarization (p=0.303) and public representation (p=0.450) at the endline (Figure 11). The reduction in the gap from the baseline was driven by improvement in IDPs’ perceptions. For example, the results revealed an increase in IDP public participation. This finding suggests that MRP has addressed the public participation disparity between IDPs and HCMs observed during the baseline assessment. This finding also implies that MRP has addressed essential needs of both IDPs and HCMs. This provides support to USAID’s Do No Harm approach, whereby it recognizes that “while the activity is focused on providing durable solutions for IDPs, as part of a DNH approach, the implementer must ensure that this is not achieved at the expense or exclusion of other, non-IDP members of the community.” This parallels USAID’s recognition of the sensitivities surrounding activity implementation and of the need to avoid interventions that may be deemed culturally inappropriate or counterproductive.41 Activity stakeholders have also noted the importance of MRP’s beneficiary targeting. Many stakeholders have expressed their appreciation for including both IDPs and HCMs in the activity, as the following narratives exemplify: The beauty of MRP is that not only the IDPs from Marawi were targeted but also the host communities. They made sure that the host community and IDPs could work together. I think this approach is important – that they both benefit from the projects. (KII, Municipal Administrator, Bubong) 40 UN Women Organization (2021). Gender-Based Violence: Women and Girls at Risk. Retrieved from https://www.unwomen.org/en/hq-complex-page/covid-19-rebuilding-for-resilience/gender-based-violence 41 USAID (2018). Project Notice of Funding Opportunity for MRP Response Project. Figure 11. Comparison of IDPs’ and HCMs’ Self￾Reliance, Polarization, and Public Participation USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 38 After being displaced, we never thought we would have the opportunity to be part of a community again, let alone vote for our chosen candidates in any election. (Youth leader, Lumbatan, MRP Success Story, AR 2021, p. 70) RURAL-URBAN ANALYSIS The team also compared how MRP benefits had been felt among beneficiaries from both rural and urban areas. The analysis revealed that beneficiaries from urban and rural locations had comparable self-perceptions about the level of polarization (p=0.102) in their communities at endline (Figure 12). Beneficiaries from urban and rural areas both agreed/strongly agreed that polarization did not exist in the community where they were residing. This finding indicates a shift from the baseline result, which showed that beneficiaries from urban locations were more likely to feel that polarization and fragmentation existed in their communities. Moreover, the analysis revealed a reduction of the gap in public representation between beneficiaries from urban and rural locations at the endline (p=0.053). During the baseline assessment, participants from rural areas were noted to have significantly better public participation self-perceptions. Hence, the social cohesion interventions that MRP implemented fostered improvement in public participation, especially among IDPs.42 CONTRIBUTION TO IDPS’ AND HCMS’ SELF-RELIANCE MRP’s RF (see Figure 1) indicates improvement in IDPs’ and HCMs’ self-reliance as the activity’s manageable impact. Self-reliance is defined as an individual’s or community’s social and economic ability to meet its essential needs in a sustainable manner. This is based on the activity’s two IRs: (1) improved economic conditions and (2) strengthened social cohesion of IDPs and HCMs. 43 The evaluation team utilized the same tool used during the baseline assessment to measure IDPs’ and HCMs’ self-reliance. To examine the activity’s contribution to beneficiaries’ self-reliance, the team performed baseline-endline analysis and intervention-comparison analysis of self-reliance survey data. BASELINE-ENDLINE ANALYSIS A descriptive analysis of the endline survey shows that MRP beneficiaries’ self-reliance remained high (Mean=2.50) (Figure 13). This outcome was achieved despite COVID-19’s adverse effects on the economic conditions of Mindanao households and their livelihoods due to mobility restrictions and decline in financial security (based on 2021 data; n=1,805). 44 42 Annex 16 contains the quantitative analysis results. 43 Based on the MRP’s AMELP. 44 Mindanao Development Authority (2022). COVID-19 Socioeconomic Impact Assessment: Strengthening Recovery Towards Peace and Development in the Six Regions. Figure 12. Comparison of Rural and Urban Participants’ Self-Reliance, Polarization, and Public Participation 39 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV Considering that most of the survey’s self-reliance indicators are expressed in economic terms (e.g., The income I earn from my job/livelihood is sufficient to cover the basic needs of my family), the pandemic’s impact would have reduced beneficiaries’ self-reliance. However, although beneficiaries’ level of self-reliance declined at endline, as shown in the figure, the magnitude is negligible or not statistically significant (p=0.231). The non-significance of the decrease implies that MRP’s interventions (e.g., livelihood grants) may have “cushioned” the beneficiaries from the economic shocks brought about by COVID-19. The qualitative data from KIIs with MRP beneficiaries corroborate these findings. One beneficiary attributed the improvement in her family’s economic condition to her gains from the handicraft business she is operating out of MRP’s BRG. Prior to the grant, she did not have the means to send her sons to school: If we compare our situation before and now, now is better. For example, my two sons who stopped schooling for two years are now enrolled. The income we have from the handicrafts that we make helped in buying the needs for their studies. (KII, Business Recovery Grant Recipient, Marawi City) Another beneficiary expressed how the livelihood grant their CSG received sustained them during the pandemic. The beneficiary even stated that their income from the livelihood grant was their means of meeting their family’s needs during the height of the pandemic: At one point during the pandemic, the sales from the face masks that we made using the sewing machines granted to us was our main income. Our income kept our lives going during that very challenging time. (KII, CSG Member, Barangay Cabili, Iligan City) Intervention-comparison group analysis The team performed a comparative analysis of MRP beneficiaries’ perceived self-reliance and that of the comparison group. This was done to examine beneficiaries’ self-reliance appraisals in relation to other individuals who also experienced the same external factors. The results indicate that MRP beneficiaries had slightly higher self-reliance at endline (Figure 14). Although the groups’ self￾reliance levels do not differ significantly (p=0.102), the result suggests that MRP’s interventions contributed to the beneficiaries’ higher appraisals. The KII excerpt below, which tells how the income from a BRG helped a father meet his family’s needs, further demonstrates MRP’s contribution to beneficiary self-reliance: Figure 14. Level of Self-reliance of Intervention and Comparison Participants During Endline Figure 13. Level of Self-reliance of MRP Beneficiaries During Baseline and Endline USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 40 MRP’s assistance really helped my business. I was able to buy a “bongo” (delivery truck) and a mini truck. As a result, I was able to send my children to school and meet my family needs. (KII, BRG Recipient, Lumba Bayabao) DURABLE SOLUTION FOR IDPS MRP’s AMELP indicates that IDPs’ durable solution is the “high level” of the project. Moreover, the activity conceives local integration of IDPs as the durable solution. Although this evaluation does not delve into the activity’s impact in the technical sense, the team explored how the interventions have facilitated IDPs’ durable solution. Figure 15 shows that most IDP beneficiaries prefer to return to Marawi (68.8 percent). The proportion of those who prefer local integration is a bit higher among MRP beneficiaries (18.5 percent) than the non￾MRP participants (17.8 percent). A good proportion of the beneficiaries were not yet fully decided at the time of the survey (12.7 percent). The team’s findings are consistent with UNCHR’s 45 finding that almost all IDPs surveyed (95 percent) expressed desire to go back to Marawi. The present findings also parallel the finding46 that most participants (70 percent) indicated their desire to return. 47 The following narratives show perspectives on IDPs’ durable solution: We still have a plan to go back to Marawi because we have a place there. My parents’ house was in my name, and then there was still a place. (KII, IDP leader/farmer, Balindong) I think that is one reason [inclusion of HCMs] why some IDPs feel comfortable in the local community, and they have an on-going livelihood. But they will eventually return to Marawi because they are from that place. (KII, Field Office Manager, Task Force Bangon Marawi) The preference among some IDPs for local integration may have been influenced by the livelihood grants and trainings they received from MRP. The grants and trainings provided them with economic opportunities in the local communities as well as platforms for interacting and working with other IDPs and HCMs. 45 UNCHR (2017). Profiling Internally Displaced Persons of the Marawi Conflict. Retrieved from http://www.protectionclusterphilippines.org/wp-content/uploads/2019/01/Profiling-Internally-Displaced-Persons-of-the￾Marawi-Conflict-2018-2.pdf 46 Collado (2019) 47 Collado. Z. (2019). Determinants of Return Intentions Among Internally Displaced Persons (IDPs) of Marawi City, Philippines. https://doi.org/10.1080/09614524.2019.1673320 Back to Marawi Local Integration Undecided Figure 15. Durable Solution Preference of Intervention and Comparison Participants 41 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV The following KII responses illustrate MRP’s intention to facilitate IDPs’ local integration. They also show observations of key stakeholders on how IDPs and HCMs interact with each other: When we ask them, they really want economic support. That’s the anchor for the recovery. But we know that we must invest in social cohesion because they will stay in their new communities for an extended time. So, in a way we’re trying to facilitate the integration of IDPs in their new communities in Lanao. (KII, Ria Orca, USAID) They [MRP] made sure that the host community and IDPs can work together. IDPs who evacuated here have already settled in our place, and they are already members of the community. (KII, MLGU, Bubong) There are instances in other projects where the sharing of resources to IDPs became an issue. With MRP, the host communities are included. I think that is one reason why some IDPs feel comfortable in the local community, and they have an on-going livelihood. (KII, Field Office Manager, Task Force Bangon Marawi) IDPs who preferred local integration as a durable solution may also have been influenced by the activities MRP conducted on voters’ education and registration. The team found that MRP’s substantial efforts in conducting voter education and registration activities resulted in the formalization of IDP membership in the local communities. For instance, document review data suggests that several LGUs facilitated special voter registration activities that helped formalize IDPs’ integration into the communities they are now residing in. As a result of these activities, 532 youth IDPs and HCMs from the 25 barangays of the towns of Butig, Ditsaan Ramain, Lumbaca Unayan, Lumbatan, Marantao, and Poona Bayabao—all in Lanao del Sur Province—participated in the community voter education and youth engagement workshops facilitated by the completers of voter education training. The workshops resulted in the registration of 242 new voters and transferees. 48 The data on the durable solution paints a different picture from that of the survey conducted by Plan International. Plan’s survey showed that most IDP participants preferred to integrate locally. The changing data on IDPs’ durable solution based on the endline survey, MRP’s durable solution survey, and literature sources suggest that the durable solution is still a “fluid” construct among IDPs at the time of the evaluation. Its fluidity is influenced by a host of factors beyond MRP’s scope. SUSTAINABILITY Sustainability refers to a local system’s ability to produce desired outcomes over time. Discrete projects contribute to sustainability when they strengthen the system’s ability to produce valued results and its ability to be both resilient and adaptive in the face of changing circumstances.49 MRP’s sustainability strategy followed USAID’s Local Systems Framework, 50 which facilitates collaborative engagements among project beneficiaries, LGUs, government agencies, CSOs, and the private sector. The framework defines how collaborative delivery of program assistance though local systems can increase the sustainability of development benefits. MRP operationalized this model in several ways, including the following: • Forging partnerships among relevant activity stakeholders that strengthened their buy-in to 48 MRP Annual Report 2021. 49 United States Agency for International Development (USAID), “Local Systems: A Framework for Supporting Sustained Development”, April 2014. 50 Ibid. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 42 the project. • Establishing sustainability policies and strategies that supported grantees in ways that qualify them for future government support and link them to markets. • Building, empowering, and formalizing CSGs so that they can continue to serve their members after MRP ends. • Promoting socially inclusive processes in intervention implementation, which included both IDPs and HCMs. These strategies are discussed in further detail in the section that follows. FORGED PARTNERSHIPS AND SYNERGIES AMONG ACTIVITY STAKEHOLDERS MRP forged partnerships and synergies among relevant activity stakeholders. MRP’s engagement with various strategic actors in implementing its interventions even during the activity’s early stages is crucial in sustaining its outcomes and strengthening stakeholders’ buy-in to the activity. Table 6 contains a sample of the coordination, partnership, and collaboration activities MRP forged with various stakeholders. Refer to Annex 14 for an expanded list. TABLE 6. SAMPLE COORDINATION, PARTNERSHIP, AND COLLABORATION ACTIVITIES ORGANIZATION/AGENCY CONTRIBUTION TO MRP 2019 Task Force Bangon Marawi (TFBM) Furnished the list of businesses in the MAA prior to the siege through DTI, which facilitated processing of BRG proposals. Department of Labor and Employment (DOLE) Assisted with job fairs, advertising skills training, and employment programs; facilitated the approval of CSG applications for workers’ associations. PLGU of LDS Helped launch MRP activities through its various offices. 2020 BARMM Served as a link in the implementation of MRP activities such as BARMM-Health on COVID-19 response. Department of Trade and Industry (DTI) Provided technical expertise for the Business Development and Enterprise Development Course. Municipal and Barangay LGUs Assisted in coordinating the delivery, distribution, and deployment of assistance needed to contain the pandemic. 2021 Bangsamoro Electoral Office (BEO) Served as resource persons on the Training of Trainers on Voter’s Education and Youth Engagement Workshops; helped register IDPs in new locations. Ministry of Trade, Investment, and Tourism (MTIT) Served as resource persons and mentors of the MRP-organized Enterprise Development Training and Mentorship program for the CSGs; supported business registration of CSGs in Lanao del Sur. 43 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV The close engagement and buy-in from stakeholders are essential factors for outcome sustainability. These factors facilitated alignment of priorities, strategies, and resource sharing. The following narratives illustrate how MRP has established synergy with other strategic actors in the rehabilitation of Marawi: TFBM and MRP have collaborated in facilitating activities. An example is the series of trade fairs. In these fairs, we really select the beneficiaries so that they will have the opportunity to sell their products. We also have the business forum, Lanao del Sur Business Forum. This is for providing the beneficiaries with market linkages. We have a series of activities to help those who receive assistance. We are collaborating to expand the market, possibly outside Marawi. (KII, Field Office Manager, Task Force Bangon Marawi) For the BRG distribution of in-kind inventories, DTI complemented by training the individual entrepreneurs along the areas of business and financial management, marketing – pricing, costing. (KII, Asst. Regional Director, DTI, Lanao del Norte) The team found that MRP’s intentionality in engaging with LGUs resulted in sustainability commitments. The LGUs have expressed support and commitment to help sustain the activity’s gains in their respective areas. Table 7 outlines a sample of the sustainability commitments MRP has generated from various LGUs. Annex 15 shows an expanded list.51 51 MRP Annual Report 2021, pp.74–82, MRP Quarterly Report Year 4 Quarter 2 2022, p. 25. Equal Access International (EAI) Partnered in capacity development on community peacebuilding, project design thinking for innovative solutions, project implementation and management, and development of media and advocacy campaigns targeting youth leaders. TABLE 7. SAMPLE OF LGU COMMITMENTS LGU COMMITMENTS Iligan City To pursue its collaboration in establishing demonstration farms for horticulture and aquaculture for the IDP and HCM beneficiaries with the goal of also benefiting non-direct beneficiaries in the long term To provide a potential site for a market hub through its Economic Enterprise Development and Management Office (EEDMO) To support interventions on Gender Development and GBV prevention Baloi To complement MRP in its Fish Hatchery and Horticulture Nursery projects To jointly conduct hands-on training, entrepreneur mentorship program, and other civic engagement trainings activities To support interventions on Gender Development and GBV prevention Butig To provide a 4- to 5-hectare area of land with authority to use for the establishment of a horticulture demonstration for a cluster of CSGs in Butig, LDS To create a TWG with representatives from the community organizations to work on the establishment of a horticulture demonstration farm. To facilitate in federating the 52 CSGs from 6 Barangays to formally become CSOs to represent the Special Bodies USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 44 ESTABLISHED POLICIES AND STRATEGIES FOR SUSTAINABILITY Grant-related policies (e.g., business registration, sustainability plan, training) MRP has established internal mechanisms that are essential in promoting the sustainability of activity gains. It has stipulated that sustainability principles be incorporated into grant administration. MRP’s Grant Management Manual illustrates the processes and procedures from the preparation of concept notes and grant proposals to endorsement and approval of grant proposals and kickoff with the beneficiaries. The process is illustrated in Figure 16: Figure 16. MRP Grant Implementation Process Part of the grant administration (third phase) is for prospective grantees of BRGs and community microgrants to have their business/livelihood duly registered with the government. The team found that this policy is essential for sustainability as it qualifies the beneficiaries to avail themselves of possible government support when MRP ends. In addition, grantees must sign a sustainability agreement, part of which is the sustainability plan that they need to develop. The implementing partners, MARADECA and ECOWEB, made necessary follow-ups relating to continuity and sustainability, as well as business improvement training and organizational capacity building. Monitoring is done through FLUXX. 52 As of March 2022, there were 394 BRGs and 368 community microgrants. Almost all BRGs are still in operation. 52 FLUXX is the industry-leading solution for data-driven grantmaking that maximizes impact. https://www.fluxx.io/ Poona Bayabao To collaborate in clustering CSGs to implement community learning facilities to improve social cohesion. To accredit qualified CSGs as CSOs To support interventions on Gender Development and GBV prevention Buadiposo Buntong To support MRP projects and agricultural and fishery development priorities, especially those that support or can be integrated with the Municipality’s primary crops/commodities (e.g., as rice, corn, vegetables) To collaborate in strengthening community-based organizations with LGU commitment to accredit MRP-supported CSGs as CSOs To support interventions on Gender Development and GBV prevention Tugaya To collaborate in systematizing mass production of the woodwork To collaborate on VAWC and GBV awareness raising and improving mechanisms for case reporting To provide GAD space 45 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV VALUE CHAIN APPROACH (VCA) MRP’s value chain prioritization is a potential sustainability mechanism as it mainstreams promising (nascent) products in view of a demand-driven market. Among MRP’s value chain priorities are aquaculture, horticulture, and weaving products. Figure 17 illustrates the framework of MRP’s overall VCA. The framework depicts four work streams of MRP intervention and activity, which are indicated in blue circles.53 MRP has implemented activities that are relevant to the four streams identified in the framework. Moreover, LGUs have supported the value chain priorities by establishing demonstration farms, providing market hubs, integrating farmers’ produce into municipalities’ primary crop commodities, and providing technical expertise. Figure 17. MRP Value Chain Framework However, unless a real commitment from buyers in national and international markets is reached, the trade expos that were conducted may not be enough to scale up the grantees’ ventures. This observation correlates with some of the key challenges already identified in the MRP VCA concept notes—namely, scaling production and delivery, bringing commercial partners into the system, and unleashing the meso-space (Schneider 2020). Because the value chain was introduced only in the second year and selected CSGs were identified and given value chain grant support only in the third year, success could not be measured at the time of evaluation. Value chain development and promotion take time. As MRP exits and the CSGs evolve into more functional and purposive business organizations, the CSGs will need further government and nongovernment interventions to be competitive in a market-driven economy. Although the team supports value chain priorities as well-thought-out, market-driven interventions, beneficiary ownership is low as the CSG clusters are still learning the technology (e.g., with aquaculture, particularly freshwater fish tilapia) while simultaneously undergoing organizational 53 The four streams include (1) Core Activities: Continuation of Recovery Grants, Technical Assistance, and Workforce Development Initiatives designed to restore livelihoods and small businesses and improve social cohesion; (2) Private Sector Engagement: Initiation of outreach, facilitation, and if possible risk reduction schemes designed to establish buyer-led commercial market systems that are supplied by beneficiaries of MRP grants and technical support; (3) Adaptive Learning: Bringing a robust CLA lens to the project focused on enabling MRP and its partners to discover how to best organize and manage scaled production for commercial systems that are supplied by businesses, entrepreneurs, and collections of both from Marawi/LDS; and (4) Meso Development: Efforts to map, network, engage, and capacity build an effective Meso Space to sustainably connect micro- and macro-level interests in Marawi and help it navigate its journey toward self-reliance (Schneider, 2020). USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 46 transformation (e.g., being a cooperative). Thus, measuring value chain progress and success goes beyond the activity life of MRP. Developing and tapping associations and cooperatives As another sustainability measure, MRP’s CSGs are trying to develop their businesses as cooperatives or associations. Several CSGs are now registered as associations with the Department of Labor and Employment (DOLE), a stepping-stone toward formal business engagement. CSGs are now talking about forming cooperatives as they develop and start to harvest in aquaculture and horticulture production. This is a step toward real enterprise development and market mainstreaming. Of the 28 CSGs from eight municipalities engaged in horticulture, food retail, and weaving, more than half have registered or are registering as associations with DOLE54. This will also be true with the recently organized clusters of associations (registered or in the process) for aquaculture in the municipality of Balindong, which is registering as a cooperative. With MRP’s exit, concerned LGUs through the Committee on Enterprise Development (or similar body) together with government line agencies could help these CSGs develop into formally organized associations or cooperatives. DOLE could assist the associations, and the community development associations could help the cooperatives. EMPOWERED COMMUNITY SOLIDARITY GROUPS MRP has devoted substantial institutional capacity development efforts to empower the CSGs. These efforts included organizing them as recognized community associations and training them on civic engagement for effective public representation as well as on community scorecards to identify their priority needs and plan and implement projects to address them. MRP also consolidated CSGs to become federations and CSOs and supported them to become social enterprises. CSG capacity building MRP has devoted substantial efforts to building CSG capacities. The effect of these efforts can be seen in the following: • Most of the CSGs (83 percent) are active (552 of 665 CSGs), based on the MRP CSG database. • Most of the CSGs (71 percent, or 139 of the 193 assessed) have organizational capacities that are considered functional to effective based on the Rapid Organizational Capacity Assessment that MRP conducted. These CSGs have the capacity to manage their current grants. 55 • The financial capacities of CSGs and their members involved with livelihood projects have improved. Toward becoming CSOs Transforming the CSGs into formal organizations (e.g., CSOs) is vital in sustaining the activity’s outcomes. The team observed the following: 54 MRP Quarterly Report FY 2022, Jan 1-Mar 31, 2022 : (Table 7. List of MRP-supported Livelihood Clusters as of FY 2022, Q2) 55 MRP Rapid Organizational Capacity Assessment (AR 2021, p. 64). 47 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV ● MRP provided technical guidance and capacity-building support through the local implementing partners and external consultants in forming the CSGs into sectoral advocacy groups and providing them with learning sessions on policy formulation and advocacy planning. These efforts resulted in 118 CSGs composed of 2,720 members federating themselves to form 17 sectoral CSGs (women, youth, and workers/farmers). 56 These CSGs have evolved from being purely a mechanism for delivering community economic and social cohesion microgrants to becoming sectoral advocacy groups. ● The CSG federations developed their organizational vision and mission as grassroots-based organizations and their organizational values as social movements of women, youth, farmers, or workers in their respective localities. MRP also supported them in securing their accreditation and recognition by the LGU and national agencies such as DOLE. ● The CSG sectoral federations identified and prioritized their advocacy agendas to advance priority issues of their respective communities, which focus on GBV prevention and women’s empowerment, youth education and employment, farmers’ access to capital and markets, conflict transformation, and workers’ rights. Finally, MRP provided them with technical guidance in drafting policy instruments such as resolutions and policy advocacy plans. They will use these advocacy plans to engage relevant government bodies—for example, the Local Youth Development Council, the Local Committee on Anti-Trafficking and Prevention/Response to VAWC, the Local Council for the Protection of Children, and the Local Development Council. Toward becoming social enterprises Endline survey data shown in Figure 18 indicate that most (73 percent) of the beneficiaries agreed/strongly agreed that MRP provided strategies to sustain their businesses/livelihoods. Moreover, a significant number of them (88 percent) expressed that they will continue to be active in their CSGs even after MRP ends. This is essential not only in achieving financial gains for their businesses but in maximizing their social impact through the realization of their organizational aspirations. Figure 18. MRP Beneficiaries’ Agreement on MRP’s Provision of Sustainability Strategies and on Sustaining Participation After the Activity Ends The following narratives illustrate the beneficiaries’ commitment to continue and sustain the gains achieved through MRP’s interventions: The intention of forming the group [CSG] is not for it to become a formal organization immediately. It’s really to simplify and make it more efficient for grants to be delivered. As it turns out, because the groups are composed of people, some have evolved into more formal 56 MRP Quarterly Report Year 4 Quarter 2 2022, p. 19. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 48 groups. They now have a formal platform to pursue their aspirations. The grant served as an anchor for that design. (KII, MERL Lead, Plan International) Up to the present, our members continued to earn profits and increased the inventories and sales of our individual sari-sari stores. Our CSG still exists, and we continue to meet. (FGD, Women CSG Members, Butig) As a women’s community solidarity group, we committed at the start to sustain what MRP gave to us. We knew that what they gave would really help us. (KII, IDP Leader, Ditsaan Ramain) IMPLEMENTED INTENTIONAL SOCIAL INCLUSION APPROACH MRP endeavored to be socially inclusive in implementing its interventions. MRP’s activities provided opportunities and spaces for greater engagement and collaboration between and among IDPs and HCMs. In particular, the requirement for IDPs and HCMs to participate in joint analysis and planning sessions to access community grants and collectively benefit from them helped address tensions over limited resources and reinforced their social cohesion. This core strategy is key in sustaining MRP’s gains and in scaling up project outcomes. This strategy has started to gain traction among development agencies and “the results have been positive and widespread, appreciated by communities that would otherwise have suffered greater deprivation and stress. Not only do such programs promote greater community harmony and lessen the risk of conflict over resources, but they can also be cost-effective life-saving mechanisms to vulnerable individuals.”57 Although this strategy has leveraged positively the collaboration of IDPs and HCMs to work together in addressing their collectively identified priority issues, it remains important to ensure that IDPs’ needs and interests, given their greater vulnerability, are prioritized. The needs of IDPs and HCMs are wide and complex. In most cases, MRP’s interventions addressed their shared needs through community grants and other relevant project processes. Addressing beneficiaries’ distinct needs remains complex and requires more resources and time. The value of targeting IDPs/HCMs as beneficiaries can be gleaned from the following narratives generated from the interviews: The beauty of MRP is that not only the IDPs from Marawi were targeted, but also the host communities. They made sure that the host community and IDPs could work together. I think this approach is important – that they both benefit from the projects. (KII, Municipal Administrator, Bubong) There are instances where the sharing of resources to IDPs became an issue. With MRP, the host communities are included. I think that is one reason why some IDPs feel comfortable in the local community because they have already assimilated there, and they have an ongoing livelihood. (KII, Field Office Manager, Task Force Bangon Marawi) LESSONS LEARNED The evaluation team shares the following reflections and lessons learned from this evaluation: 1. Intentionally targeting IDPs and HCMs provided the beneficiaries with myriad opportunities to work together, harness relationships, and address common issues. Moreover, the participatory 57 United Nations High Commissioner for Refugees, IDPs in Host Families and Host Communities: Assistance for Hosting Arrangements, April 2012. 49 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV processes carried out by MRP forged effective engagement of both LGUs and CSGs. On the one hand, these mechanisms increased IDPs’ and HCMs’ sense of ownership and commitment to engage in project activities. On the other hand, the LGUs have become more open in allowing spaces for greater civic engagement by IDPs and HCMs in the barangay and municipal governance structures. 2. Mutually reinforcing economic development and social cohesion outcomes are integral to addressing IDP/HCM needs in highly complex environments. MRP’s experience showed the importance of intentionally incorporating these two outcomes to address economic development gaps and sociocultural and political dynamics. 3. Efficiency in delivering assistance for emergency situations should inform the development of project protocols and requirements. In line with the principles of adaptive management, there needs to be a balance between accountability and timely delivery of appropriate responses to beneficiaries. In highly complex situations, being efficient in delivering interventions to vulnerable and often frustrated beneficiaries could lead them to actively engage in activity processes and increase their sense of hope for durable solutions. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 50 CONCLUSIONS The evaluation team forwards the following conclusions based on the evaluation findings. RELEVANCE EQ1. Were MRP’s development interventions relevant to IDP and host community needs and adaptive to the complex and changing external environment in Mindanao, especially in the project areas? The interventions MRP implemented were relevant to IDP and HCM needs and adaptive to the activity area’s complex environment. MRP’s application of USAID’s guidance on co-creation processes, adaptive management, and local partnerships throughout implementation contributed to the interventions’ relevance and adaptability to beneficiary needs. MRP’s business recovery and community microgrants alongside its business and workforce development trainings addressed beneficiaries’ economic recovery needs. Moreover, the social cohesion grants and trainings the activity provided are relevant to easing possible tensions in the operating environment. The activity has significantly contributed to the ongoing efforts to rehabilitate Marawi by effectively collaborating with other actors in the activity areas and by mainstreaming its efforts and activities. Its adaptive community-based approaches contributed to improving self-reliance among IDPs and HCMs, narrowing social gaps at the community level, and providing enabling conditions for IDPs’ local integration. EFFECTIVENESS EQ2. Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic activities among IDPs and HCMs? MRP cushioned beneficiaries’ self-reliance from the adverse effects of external challenges, particularly those brought about by COVID-19. The BRGs, community microgrants, and trainings received by the beneficiaries allowed them to generate income by starting businesses and engaging in gainful employment. Thus, MRP stimulated a restoration of individual and community capacities to engage in economic activities. Moreover, the activity’s interventions reduced gender and social gaps, particularly those noted during the baseline. MRP’s social cohesion grants and trainings fostered civic engagement and community improvement among IDPs and HCMs. Consequently, it has improved beneficiaries’ perceptions of social relations, connectedness, and orientation toward the common good. It has improved IDPs’ and HCMs’ perceptions of polarization in their communities and their own public participation in civic activities. MRP’s interventions and processes offered opportunities for IDPs to integrate locally as a durable solution to their displacement. Although IDPs’ durable solution preference appears to be a “fluid” subject now, the activity has contributed to the preference for local integration through the business and employment opportunities it has provided to IDPs during their displacement. SUSTAINABILITY 51 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV EQ3. Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs? MRP has established concrete sustainability mechanisms that are essential to sustaining the activity’s outcomes. These mechanisms align with USAID’s Local Systems Framework. Guided by the tenets of the framework, MRP harnessed partnerships between and among strategic stakeholders to strengthen their buy-in, which resulted in institutional complementation. It laid the foundation for the CSGs to qualify for future government support and linked them to potential markets for their products through the sustainability policies and strategies it implemented. MRP developed building blocks for CSGs to continue to operate, serve their members, and even scale up their livelihoods after MRP ends by empowering and formalizing them into cooperatives and CSOs. MRP also established a social fabric for the harmonious involvement, participation, and interaction of IDPs and HCMs as they scale up their ventures by targeting both groups in activities and interventions. The capacity- and partnership-building mechanisms that MRP carried out provided an impetus that could sustain the activity’s social and economic outcomes. Moreover, MRP’s contribution through the business and livelihood grants and the workforce development training will allow the IDPs and HCMs to take advantage of opportunities brought about by the resumption of business and livelihood activities in the localities where they reside. The strengthening, consolidation, and social network expansion of the CSGs transformed them into advocacy/sectoral organizations with a strong foundation for sustaining economic and social cohesion gains. With these mechanisms, it is likely that the activity’s outcomes will be sustained after MRP ends. USAID.GOV MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION | 52 RECOMMENDATIONS The evaluation team recommends the following points for consideration by USAID, government agencies, and other development actors. INTERVENTION PLANNING, DESIGN, AND IMPLEMENTATION 1. MRP is the first USAID intervention that was designed through a co-creation process. Given its contribution to a groundwork for better adaptive management, the team recommends that future USAID interventions, especially for complex environments, consider using this co￾creation process. Future interventions could benefit from the multidisciplinary perspectives that would be generated during the exercise. 2. Future interventions designed for complex environments such as that of MRP could be configured with a much longer timeframe. Although MRP achieved its designed outputs and outcomes on a year-to-year basis, realizing the IDPs’ durable solution requires time beyond the life of the activity. 3. The team recommends selecting the right performance indicators and clear measurements and tools to strengthen the validity of activity performance data and outcomes. Moreover, the targeting for each indicator across the activity’s life needs to be rationalized based on baseline assessment information, updated implementation data, and other relevant data sources. Guided by the principles of adaptive management and learning, the targeting could be more pragmatic, adaptive, and relevant to the changes in the complex operational context. 4. The team recommends reviewing the Grant Management Manual to facilitate the quick delivery of quality interventions to project beneficiaries. In the case of MRP, delays in grant delivery affected project implementation. In complex operating environments, delays could trigger tensions among beneficiaries and other stakeholders. The evaluation team thus reaffirms the importance of setting up project accountability measures that foster speedy delivery of relevant and timely interventions, especially in highly complex environments. ENSURING SUSTAINABILITY OF ACTIVITY OUTCOMES 1. The LGUs should continue the partnerships with the implementing partners and other CSOs in undertaking peace and development efforts in communities covered by MRP. To build on the positive results achieved by the CSGs, the LGU-CSO partnership remains critical in designing and implementing joint efforts. LGUs’ commitment to provide continuing support, accompaniment, and greater space for public participation will lead to CSGs’ organizational development as social formations and as effective partners in implementing community projects for the collective benefit of IDPs and HCMs. 2. LGUs need to follow through on their commitments to foster sustainability of the activity’s economic and social cohesion outcomes. It is essential that the commitments the LGUs gave during the rounds of consultation that MRP conducted be translated into binding mechanisms. For example, the LGUs could integrate their commitments (such as those related to the value chain and GBV initiatives) into their respective local governments’ legislative agenda. The realization of the commitments would thus not be subject to leadership changes in the concerned agencies and offices. 53 | MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID.GOV CONDUCTING FUTURE RESEARCH Considering the fluid nature of IDPs’ durable solution preferences at the time of evaluation, other development actors, such as academe, could conduct research to document the durable solution and MRP interventions’ impact on decisions. The durable solution model for the Marawi siege IDPs could be explored using econometric and psychosocial approaches. Results from these studies could provide context for USAID and other donor agencies to consider when designing future interventions. Annexes MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION REPORT LIST OF ANNEXES Annex 1 Statement of Work 01 Annex 2 Evaluation Design and Workplan (Inception Report) 26 Annex 3 Sample Selection Criteria and Process 138 Annex 4 Evaluation Participants 146 Annex 5 Enumerators’ Training Design 152 Annex 6 Pilot Test Design and Administration Guide 155 Annex 7 Endline Survey Tool 164 Annex 8 FGD and KII Tools 203 Annex 9 Informed Consent Form 224 Annex 10 Activities that Contributed to MRP’s Adaptability and Responsiveness 227 Annex 11 Community Engagement and Grant Application Cycle 230 Annex 12 Collaborative Activities Conducted by MRP 232 Annex 13 MRP Performance in Relation to the Project Indicators 237 Annex 14 Coordination, Partnership, and Collaboration Activities of MRP 243 Annex 15 List of Sustainability Commitments from Various LGUs 248 Annex 16 Quantitative Analysis Results 253 Annex 17 FGD and KII Transcripts 262 Annex 18 Process and Script for Validating Survey Respondents 274 Annex 19 Suggested Revisions Based on Pilot-test Observations 277 Annex 20 Non-Disclosure and Conflict of Interest Agreement 284 (MRP External Evaluation Annexes) Page 1 | 286 Annex 1 STATEMENT OF WORK Marawi Response Project External Performance Evaluation Report (MRP External Evaluation Annexes) Page 2 | 286 STATEMENT OF WORK EXTERNAL EVALUATION OF THE MARAWI RESPONSE PROJECT (MRP) PHOTOCREDIT: SUNSTAR PHILIPPINES (MRP External Evaluation Annexes) Page 3 | 286 DISCLAIMER: This report is made possible by the support of the American People through the United States Agency for International Development (USAID). Its contents are the sole responsibility of CLAimDev and do not necessarily reflect the views of USAID or the United States Government. CONTENTS ACRONYMS IV ACTIVITY SUMMARY 1 BACKGROUND 1 CHALLENGE 2 RESPONSE 2 THEORY OF CHANGE 2 GOAL 3 RESULTS FRAMEWORK 3 EVALUATION DESIGN 4 PURPOSE 4 AUDIENCE 4 FRAMEWORK 4 EVALUATION QUESTIONS 7 RELEVANCE 7 EFFECTIVENESS 7 SUSTAINABILITY 7 CONTEXT 7 APPROACHES AND GUIDING PRINCIPLES 9 PERFORMANCE EVALUATION 9 (MRP External Evaluation Annexes) Page 4 | 286 MRP AS A TRANSITIONAL ASSISTANCE INITIATIVE 9 COMPLEXITY-AWARE EVALUATION 10 GENDER EQUALITY AND SOCIAL INCLUSIVITY 10 LOCATION AND GEOGRAPHIC SCOPE 10 DELIVERABLES AND REPORTING REQUIREMENTS 10 EVALUATION TEAM 11 EXTERNAL EVALUATION TEAM 11 EXTERNAL EVALUATION TEAM QUALIFICATIONS 13 CLAIMDEV EVALUATION TEAM 14 EVALUATION SCHEDULE 15 FINAL REPORT 16 FORMAT 16 QUALITY ASSURANCE CRITERIA 17 OTHER REQUIREMENTS 18 (MRP External Evaluation Annexes) Page 5 | 286 ACRONYMS AMELP Activity Monitoring, Evaluation and Learning Plan AMT Activity Management Team AOR Agreement Officer Representative CAME Complexity Aware Monitoring and Evaluation CDCS Country Development Cooperation Strategy CLA Collaboration, Learning and Adaptation CLAimDev Collaboration, Learning and Adaptation for Improved Development COP Chief of Party COR Cooperative Agreement Representative DCOP Deputy Chief of Party DDL Development Data Library DEC Development Experience Clearinghouse DO Development Objectives ECOWEB Ecosystems Work for Essential Benefits FGD Focus Group Discussion IDP Internally Displaced Persons HCM Host Community Members IP Implementing Partners KII Key Informant Interviews LOE Level of Effort MARADECA Maranao People Development Center MRP Marawi Response Project MSME Medium and Small Micro-Enterprise (MRP External Evaluation Annexes) Page 6 | 286 NGO Non-Government Organization OEDG Office of Economic and Democratic Governance PI Principal Investigator PIRS Performance Indicator Reference Sheet PRM Office of Program Resource Management RF Results Framework SOW Statement of Work STTA Short-Term Technical Assistance SURGE Strengthening Urban Resilience for Growth with Equity ToC Theory of Change USAID United States Agency for International Development VEO Violent Extremist Organizations (MRP External Evaluation Annexes) Page 7 | 286 ACTIVITY SUMMARY TABLE 1. SUMMARY INFORMATION Activity Name Marawi Response Project (MRP) Implementing Partner PLAN International Cooperative Agreement number 72049218CA00007 Total Estimated Cost (TEC) US$25,000,000 Period of Performance September 2018–September 2021 Active Geographic Regions Lanao del Sur, Marawi City, Lanao del Norte, and Iligan City Country Development Cooperation Strategy CDCS 2016-2019 Development Objective DO2: Improved peace and stability in conflict-affected areas of the Philippines, particularly Mindanao Evaluation Type External performance evaluation BACKGROUND The USAID-funded Marawi Response Project (MRP) supports people and families who were internally displaced by the 2017 Marawi siege,58 resulting from armed conflicts between the Armed Forces of the Philippines and ISIS-inspired militants in Marawi (Lanao del Sur), Philippines. USAID selected PLAN International to implement the activity, which lasts from September 2018 to September 2021. USAID/Philippines designed MRP in line with its medium-term strategic goal to support the reintegration or return of internally displaced persons (IDPs), particularly those with an uncertain likelihood of returning to Marawi. Work toward USAID/Philippine’s medium-term strategic goal has been conducted in two phases. Completed in mid-2019 through the joint efforts of various partners, the first phase focused on addressing the early recovery needs of displaced individuals (Objective 1) and their transition to social and economic stability (Objective 2). The second phase focused on continuing the pursuit of transitioning to social and economic stability (Objective 2) and establishing the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs (Objective 3) through the Marawi Response Project. MRP thus became USAID’s primary Marawi response effort, integrating and building upon various new and ongoing USAID/Philippine’s activities 58 Also referred to as the Battle of Marawi or the Marawi crisis. (MRP External Evaluation Annexes) Page 8 | 286 operating in and surrounding Marawi. The lessons learned from MRP are essential to guiding USAID’s interventions in similar contexts. CHALLENGE MRP assists IDP families who are in the early emergency and transitional reintegration phases of displacement.59 MRP is responding to the numerous challenges that both IDPs and their host communities face such as having lost their homes, property, and livelihoods. Additional challenges include a mutual feeling of being marginalized and neglected, and tension between IDPs and host communities. When IDPs and host communities lack confidence in the local government’s ability to address these challenges, tensions may increase, and the challenges may become crises. Unaddressed crises or crises that are not addressed sufficiently have the potential to become conflicts of their own. Moreover, violent extremist organizations (VEO) can manipulate such tensions to further destabilize the area. RESPONSE MRP works to improve economic conditions and increase social cohesion among IDPs and their host communities in Marawi and nearby municipalities. The activity conducts interventions to revive economic livelihoods, support business recovery, and provide employment opportunities. It has also applied community capacity development approaches such as advancing leadership opportunities, strengthening organizations, and ensuring active civic participation among marginalized groups. MRP partners with local government units, government agencies, civil society groups, and the private sector. MRP additionally works with local partners, including Ecosystems Work for Essential Benefits (ECOWEB) and Maranao People Development Center (MARADECA). MRP’s experiences operating in complex and frequently high-threat environments in Mindanao provide valuable lessons to USAID/Philippines for evolving appropriate implementation approaches and managing activities in similar environments. The activity’s COVID-19 adaptations provide additional lessons. THEORY OF CHANGE MRP’s theory of change states: If… IDPs, returnees, and host populations actively participate in and contribute to the social and economic development of their communities in a manner that reduces inter- and intra-community tensions and fosters resilience, 59 According to USAID Assistance to Internally Displaced Persons Policy Implementation Guidelines, there are four phases of displacement: (1) pre-emergency preparedness, (2) early emergency, (3) transitional reintegration, and (4) long-term development. (MRP External Evaluation Annexes) Page 9 | 286 then… they will be more self-reliant and better enabled to advance solutions to their displacement. GOAL MRP’s goal is to support IDPs and their host communities in becoming more self-reliant and finding long-term solutions to displacement. This is based on the perspective of transitioning from displacement and is consistent with USAID’s approach to operating in dynamic environments. Thus, the activity’s high-level impact is advancing long-term and durable solutions to support local integration of IDPs. RESULTS FRAMEWORK Table 2 details MRP’s results framework. TABLE 2. MRP RESULTS FRAMEWORK RESULTS LOGIC RESULTS STATEMENTS High-Level Impact A durable solution: IDPs integrated into local communities Manageable Impact Self-reliance of IDPs and host community members improved Intermediate Results and Outputs IR 1: Economic conditions of IDPs and host communities improved. Output 1.1: Business recovery opportunities expanded. Output 1.2: Employment opportunities expanded. Output 1.3: Livelihood opportunities expanded IR 2: Social cohesion of IDPs and host communities strengthened. Output 2.1: Social cohesion grants provided to host and displaced communities. Output 2.2: Social cohesion training provided to host and displaced communities (MRP External Evaluation Annexes) Page 10 | 286 Performance Indicators ● Indicator 1: Level of self-perceived self-reliance of assisted displaced and host community members. ● Indicator 2: Number of displaced business owners with new or re-started businesses ● Indicator 3: Percentage of trained internally displaced persons/host community members who are gainfully employed. ● Indicator 4: Number of displaced business owners who benefitted from business recovery micro-grants. ● Indicator 5: Number of displaced business owners trained in enterprise management. ● Indicator 6: Number of displaced persons/host community members trained in workforce readiness and life skills. ● Indicator 7: Number of displaced/host community members who benefitted from micro grants. ● Indicator 8: Level of polarization between host and displaced communities ● Indicator 9: Level of self-perception of public representation ● Indicator 10: Number of displaced/host community members who benefitted from social cohesion grants. ● Indicator 11: Number of displaced/host community members trained EVALUATION DESIGN PURPOSE The objectives of this performance evaluation are to: (1) assess MRP’s performance in achieving its target outcomes and outputs through the activity’s planned strategies and interventions; (2) identify and learn from the factors that influenced implementation and results; and (3) gather insights and recommendations to inform successful implementation of other activities in similarly complex environments. AUDIENCE The target evaluation audience includes the following groups and individuals: USAID/Philippines, USAID/Regional Development Mission for Asia, USAID/Washington, Marawi rehabilitation (MRP External Evaluation Annexes) Page 11 | 286 stakeholders, local communities, NGOs, local government units, and Philippines government agencies. In collaboration with USAID/Philippines, the evaluation team, will develop an evaluation dissemination and learning plan to reach the target audience. FRAMEWORK Figure 1 illustrates the scope of the evaluation and identifies the key activity elements that require data collection and in-depth analysis. The framework also suggests analyzing the interrelationships between the different key elements to better understand the factors contributing to MRP’s outputs and outcomes. Each element of Figure 1 is first explained directly below. GENERAL ENVIRONMENT There is an array of external factors that influence the way MRP is implemented and the results it achieves. These include the local environment, technology, economic conditions, demographics, sociocultural forces, and political or legal factors. SPECIFIC ENVIRONMENT The specific environment refers to the beneficiaries, families, communities, organizations, leaders, business sector, local government units, government organizations, NGOs, and other important actors who are directly and indirectly involved in implementing MRP. PURPOSE The overall intention of the activity includes the theory of change/development hypotheses and all other information related to institutional mandates that guide the activity’s implementation. OUTPUTS AND OUTCOMES Outputs and outcomes refer to information about MRP’s intended and unintended results. This evaluation will thus consider and integrate the three outcome indicators that MRP’s baseline study highlights, namely: level of self-reliance, level of polarization between host and displaced communities, and level of participation (i.e., the voice or level of representation of IDPs in their new locality). The units of analysis for outputs and outcomes are individual (i.e., IDPs and host community members) and groups (e.g., micro, small, and medium-sized enterprises (MSME) and community groups). INPUTS Inputs are information related to the materials, and financial, human, and other resources used to implement the activity. OPERATIONS (MRP External Evaluation Annexes) Page 12 | 286 Operations refers to how the activity was implemented and managed, including service delivery systems and processes. This will also include the management structure, leadership styles, organizational culture, and other internal management mechanisms that describe activity operations. (MRP External Evaluation Annexes) Page 13 | 286 Figure SEQ Figure \* ARABIC 1. MRP Evaluation Framework (MRP External Evaluation Annexes) Page 14 | 286 EVALUATION QUESTIONS The overarching questions that this external performance evaluation aims to answer are: 1. In what ways did MRP contribute to IDPs and their host communities becoming more self￾reliant and finding long-term solutions to displacement? 2. How did the complex environment in Marawi affect the implementation of MRP? The evaluation will focus on three evaluation criteria: the relevance, effectiveness, and sustainability of MRP interventions. The evaluation will contextualize its findings, conclusions, and recommendations based on changes in the social environment in Marawi and in host communities. RELEVANCE ● In what ways has MRP contributed to integrating IDPs into host communities as a durable solution for addressing their needs? ● In what ways is MRP aligned with USAID's approach to operating in complex environments? ● In what ways is MRP aligned with the Government of the Philippines’ approach to transitioning displaced families to safer communities and restoring livelihood and income opportunities? EFFECTIVENESS ● To what extent has MRP achieved its intended outcomes in improving the self-reliance of IDP communities? ● To what extent was MRP able to improve economic conditions for IDPs and host community members? ● To what extent has MRP been able to strengthen the social cohesion of IDPs and host communities? SUSTAINABILITY What mechanisms have evolved out of MRP interventions to sustain and further improve economic conditions for beneficiaries and social cohesion between IDPs and host communities? CONTEXT What changes in the social environment enhanced or inhibited MRP’s implementation and contributed to the status of IDPs and host communities? (MRP External Evaluation Annexes) Page 15 | 286 TABLE 3. EVALUATIONDESIGN MATRIX QUESTIONS SUGGESTED DATA SOURCES SUGGESTED DATA COLLECTION METHODS SUGGESTED DATA ANALYSIS METHODS Relevance ● In what ways has MRP contributed to integrating IDPs into host communities as a durable solution for addressing their needs? ● In general, in what ways is MRP aligned with USAID's approach to operating in complex environments? ● In what ways is MRP aligned with the Government of the Philippines’ approach to transitioning displaced families to safer communities and restoring livelihood and income opportunities? ● Activity documents and reports ● IPs and key stakeholders ● USAID personnel ● Document review ● End-line survey on beneficiaries’ perceptions and their economic conditions ● Stakeholder feedback (survey, key informant interviews [KII], and focus group discussions [FGD]) ● Qualitative analysis ● Quantitative analysis Effectiveness ● To what extent has MRP achieved its intended objectives relative to improving the self￾reliance of IDP communities? ● To what extent was MRP able to improve economic conditions for IDPs and host community members? ● To what extent has MRP been able to strengthen the social cohesion of IDPs and host communities? ● Activity documents and reports ● IPs and key stakeholders ● Document review ● End-line survey on beneficiaries’ perceptions and their economic conditions ● Stakeholder feedback (survey, KII, FGD) ● Qualitative analysis ● Quantitative analysis ● Metadata analysis ● Impact Analysis (using treatment and comparison groups, randomized sampling methods, etc.) Sustainability ● What mechanisms have evolved out of MRP interventions to sustain and further improve economic conditions for ● Activity documents and reports ● Document review ● Qualitative analysis (MRP External Evaluation Annexes) Page 16 | 286 TABLE 3. EVALUATIONDESIGN MATRIX QUESTIONS SUGGESTED DATA SOURCES SUGGESTED DATA COLLECTION METHODS SUGGESTED DATA ANALYSIS METHODS beneficiaries and social cohesion between IDPs and host communities? ● IPs and key stakeholders ● Stakeholder feedback (survey, KII, FGD) ● Most significant change and/or outcome harvesting methods (FGD, KII) ● Quantitative analysis Context ● What changes in the social environment enhanced or inhibited activity implementation and have contributed to the status of IDPs and host communities? ● Activity documents and reports ● IPs and key stakeholders ● Document review ● KII ● FGDs ● Context evaluation tools (e.g., sentinel indicators) using USAID and MRP context indicators ● Qualitative analysis ● Quantitative analysis APPROACHES AND GUIDING PRINCIPLES PERFORMANCE EVALUATION This external performance evaluation will assess the ways and extent to which MRP met its goal and objectives. It will draw on MRP’s 2019 baseline study results and conduct an end line survey to measure the project’s effect on three key outcome indicators: self-reliance, polarization, and public participation of the IDPs in host communities. In addition to considering MRP’s quantitative and qualitative accomplishments (actual versus target), the evaluation will also measure how well MRP has complied with the data quality standards that were defined in the performance indicator reference sheet (PIRS) of the activity’s approved activity monitoring, evaluation, and learning plan (AMELP). This evaluation will also determine MRP’s relevance in terms of its contribution to and potential influence on the broader goals and strategies of the Mission, especially with respect to managing activities in complex situations. MRP AS A TRANSITIONAL ASSISTANCE INITIATIVE (MRP External Evaluation Annexes) Page 17 | 286 MRP is USAID’s special activity to provide humanitarian and developmental assistance to IDPs that are affected by armed conflict in Mindanao. The external evaluation will assess this activity using the framework in the USAID Assistance to Internally Displaced Persons Policy. Furthermore, it will consider the important guidance raised in the USAID Office of Transition Initiatives’ document, Lessons Learned: Monitoring and Evaluation in Complex and High-Threat Environments. In addition, the external evaluation will carefully consider using the IDP tools and frameworks that are provided in the UN Guiding Principles on Internal Displacement. USAID supports the goals of these principles and encourages its partners and host governments to use them as a practical reference. COMPLEXITY-AWARE EVALUATION Due to the complexities of the local environment and the onset of the COVID-19 pandemic during MRP implementation, the external evaluation will carefully consider USAID’s complexity-aware monitoring and evaluation (CAME) approaches. GENDER EQUALITY AND SOCIAL INCLUSIVITY Gender equality and social inclusion are important dimensions to be considered in measuring the success of MRP. The evaluation will thus consider whether and how MRP reduced gender disparities and other social disparities, empowered women and members of traditionally disadvantage groups, and mitigated the incidence of gender-based violence. Furthermore, the evaluation will reflect awareness of and respect for the local culture in terms of their values, traditions, language, communication mode, and style. LOCATION AND GEOGRAPHIC SCOPE The evaluation will gather information at the individual, community, and organization level in the 19 municipalities and 2 cities located within the two provinces of Lanao. Since many of the assisted IDPs have migrated to other locations, this evaluation will also attempt to trace and reach out to these individuals during data collection. TABLE 4. MRP INTERVENTION AREA PROVINCE CITY/MUNICIPALITY Lanao del Norte Iligan City, Baloi, Pantao Ragat, Pantar Lanao del Sur Marawi City, Balindong, Buadiposo Buntong, Bubong, Butig, Ditsaan, Ramain, Kapai, Lumba Bayabao, Lumbaca Unayan, Lumbayanague, Madalum, Marantao, Masiu, Poona Bayabao, Piagapo, Saguiaran, Tubaran DELIVERABLES AND REPORTING REQUIREMENTS The following are the deliverables under this evaluation contract: (MRP External Evaluation Annexes) Page 18 | 286 Table 5. Deliverables and Requirements TABLE 5.DELIVERABLES AND REQUIREMENTS DELIVERABLES REQUIREMENTS Inception Report The report will include the following: ● Evaluation design: detailed matrix, data collection instruments, potential informants/respondents/sites, and known limitations. ● Work plan: schedule, logistical arrangements, key stakeholders, and areas to be visited. ● Evaluation dissemination and learning plan: a description of the learning products and learning events that will be developed to disseminate the evaluation to the target audience Briefing/Learning Events The evaluation team will conduct the following briefings: ● Inception briefing ● Mid-term briefing and interim meetings to regularly update USAID/Philippines on the evaluation’s implementation status. ● Out-briefing to present the draft report to USAID/Philippines ● Learning events (at least two) for key stakeholders to ensure utilization of the evaluation’s findings and recommendations Draft Evaluation Report ● Submit first as a draft to USAID/Philippines for comment. Final Evaluation Report The final report and submission will: ● Address and incorporate all comments raised by USAID/Philippines on the draft report. ● Meet USAID quality standards for evaluation reports. ● Include complete data and records that are properly documented, and provided in an electronic format that is easily readable and well-organized. ● Be formatted using USAID branding and templates and submitted electronically as a PDF file. (MRP External Evaluation Annexes) Page 19 | 286 EVALUATION TEAM EXTERNAL EVALUATION TEAM The external evaluation team should be composed of experts with in-depth knowledge and understanding of evaluation and/or survey design and methodology, social cohesion, socio￾economic interventions, humanitarian assistance in highly complex situations (especially in the context of Mindanao cultures), and political conflict dynamics. At least one member of the team should have expertise in gender issues. To reduce bias, evaluation team members shall not have been involved in the implementation of MRP in any recent, on-going or substantial way with the implementation of MRP. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest. The evaluation team shall demonstrate familiarity with USAID’s Evaluation Policy. The composition of the external evaluation team, and corresponding roles and responsibilities are as follows: EVALUATION TEAM LEADER (1 STTA) The evaluation team leader will oversee all aspects of the evaluation. They will manage the evaluation team’s activities and ensure the work plan is implemented in a timely manner. They will also have primary responsibility for drafting and revising all evaluation deliverables. TECHNICAL SPECIALISTS (2 STTA) Two technical specialists will serve as co-evaluators to work closely with the evaluation team leader in implementing the work plan and analyzing data. The two specialists are expected to actively participate in formulating the data analysis frameworks, sampling design, and data collection tools. Each specialist is expected to conduct in-depth data analysis and write draft evaluation findings related to their respective subject matter and submit them to the evaluation team leader for inclusion in the final evaluation report. QUALITATIVE DATA ANALYST (1 STTA) AND QUANTITATIVE DATA ANALYST (1 STTA) The quantitative and qualitative data analysts are tasked with developing the evaluation’s data collection and consolidation platforms. The data analysts will implement quality control mechanisms to ensure data validity and accuracy. They will also analyze the field data using SPSS (for quantitative data) and NVivo (for qualitative data). They will additionally be assigned to develop data infographics to supplement the evaluation reports. FIELD EVALUATION ASSISTANTS (3 STTA) (MRP External Evaluation Annexes) Page 20 | 286 Three field evaluation assistants will conduct key data collection at the field level. Two will be assigned in Lanao del Norte and one will be assigned in Lanao del Sur. The field evaluation assistants will ensure all data collection activities are completed in their respective areas following standard protocols and data quality requirements. The field assistants will guide the survey data collectors, who will administer the face-to-face end￾line survey to identify respondents in their respective areas. They will also supervise the enumerators in conducting online tracking and reaching out to other beneficiaries/respondents who are no longer found in their respective activity locations. PROJECT ASSISTANT (1 STTA) The project assistant will assist the evaluation team with all logistical, travel, documentation, and administrative needs. They will work closely with the evaluation team leader to determine the evaluation’s administrative requirements. They will also work closely and be supervised by CLAimDev’s administrative unit when conducting procurement and making other logistical arrangements. EXTERNAL EVALUATION TEAM QUALIFICATIONS The ideal candidates for the external evaluation team should possess the combination of skills and qualifications described below: EVALUATION TEAM LEADER ● Evaluation professional with at least 10 years of experience conducting research or evaluation studies, having served as a team leader or in an equivalent position on USAID evaluations or large research projects. ● In-depth knowledge and understanding of evaluation, survey design and methodology, and/or social cohesion, socio-economic interventions, humanitarian assistance in highly complex situations (especially in the context of Mindanao cultures), and political conflict dynamics. ● A master’s degree in the social sciences or related disciplines is required. A Ph.D. is preferred. ● Ability to write message-driven evaluation reports in English. ● Proven ability to lead and manage project evaluation teams. ● Willingness and ability to work as part of a team. ● Excellent English communication skills, both written and oral. Evaluation reports drafted by candidates may be requested. ● Fluency in Tagalog and local dialects spoken in Mindanao is a plus. EVALUATION TECHNICAL SPECIALISTS (2) ● Expertise in social cohesion, socio-economic interventions, humanitarian assistance in highly complex situations (especially in the context of Mindanao cultures), and political conflict (MRP External Evaluation Annexes) Page 21 | 286 dynamics, and/or in-depth knowledge and understanding of evaluation, survey design and methodology. One of the technical specialists should have a strong background in gender and inclusive development. ● Willingness and ability to work as part of a team. ● Demonstrated knowledge of monitoring and evaluation systems. ● Evaluation experience is strongly preferred. ● At least seven years of experience in a relevant discipline is preferred. ● A master’s degree in social sciences or a related discipline is required. A Ph.D. or doctoral studies is preferred. ● Excellent English communication skills, both written and oral. Sample studies or published work may be requested from candidates. ● Fluency in Tagalog and local dialects spoken in Mindanao is a plus. QUALITATIVE AND QUANTITATIVE DATA ANALYSTS ● Expertise in analyzing qualitative and/or quantitative data. ● Willingness and ability to work as part of a team. ● Demonstrated knowledge and experience in organizing and cleaning qualitative or quantitative datasets, and in analyzing qualitative data using MAXDQA or SPSS to analyze quantitative data. ● Demonstrated knowledge and experience in organizing and displaying data in graphs and tables, including creating data visualizations and infographics. ● At least five years of experience in data analysis. ● A bachelor’s degree in social sciences or a related discipline is required. A master’s degree or studies is preferred. ● Excellent English communication skills, both written and oral. Sample studies or published work may be requested from candidates. ● Fluency in Tagalog and local dialects spoken in Mindanao is a plus. FIELD REGIONAL RESEARCH ASSISTANTS ● At least five years of experience in monitoring and evaluation, social cohesion, socio-economic interventions, humanitarian assistance in highly complex situations (especially in the context of Mindanao cultures), or political conflict dynamics. ● A master’s degree in the social sciences or a related discipline is required. (MRP External Evaluation Annexes) Page 22 | 286 ● Willingness and ability to work as part of a team. ● Excellent English communication skills, both written and oral. Sample studies or published work may be requested from candidates. ● Fluency in Tagalog and local dialects spoken in Mindanao is required. PROJECT ASSISTANT ● Experience providing support services, preferably for evaluations, including formatting interview guides and survey questionnaires; using remote data collection tools; supporting document management and data processing; and scheduling and arranging consultation meetings, travel, and venues for learning events. ● A bachelor’s degree in the social sciences or a related discipline is required. ● Excellent English communication skills, both written and oral. ● Fluency in Tagalog and local dialects spoken in Mindanao is a plus. CLAIMDEV EVALUATION TEAM CLAimDev will have the ultimate responsibility for the integrity, quality, and management of the external evaluation team. CHIEF OF PARTY (1 CLAIMDEV) The CLAimDev chief of party will have the primary responsibility for quality control. She will supervise and manage the work of the principal investigator and team leader. PRINCIPAL INVESTIGATOR (1 CLAIMDEV) The principal investigator will be the CLAimDev senior monitoring, evaluation and learning specialist. He will have primary responsibility for all aspects of the evaluation, including design, implementation, and reporting. He will also be responsible for managing, monitoring, and ensuring the integrity of all collaborative relationships with USAID and its partners, stakeholders, and beneficiaries. He will additionally supervise the work of all evaluation STTA consultants as well as the CLAimDev evaluation specialist. EVALUATION SPECIALIST (1 CLAIMDEV) The evaluation specialist will assist the principal investigator/CLAimDev senior monitoring, evaluation and learning specialist as part of continuity measures and integrating CLA approaches into the CLAimDev work processes. The evaluation specialist will also assist the principal investigator in ensuring effective coordination between the evaluation team and the contracted communications groups during development of learning materials and implementation of learning events. MEETINGS AND EVENT COORDINATOR (1 CLAIMDEV) (MRP External Evaluation Annexes) Page 23 | 286 The meetings and event coordinator will provide logistics, meeting, and event coordination support to the external evaluation team. EVALUATION SCHEDULE The evaluation will be conducted during an eight-month period, estimated to run from August 2021 through March 2022. Divided into three phases, the list of evaluation activities and their estimated duration are shown in Table 6. TABLE 6. ESTIMATED DURATIONAND SCHEDULE TASK NUMBER TASKNAME ESTIMATED DURATION (BUSINESS DAYS) Phase 1 – Evaluation design (August - September 2021) 1 Hold evaluation team planning meeting 3 2 Conduct document review 10 3 Hold initial consultations with IPs and major stakeholders 5 4 Prepare inception report that includes evaluation design, methodology, tools, and schedule 10 5 Submit draft inception report and receive USAID comments 5 6 Revise inception report 8 7 Submit revised inception report to USAID/Philippines 0 8 Hold inception briefing 2 Phase 2 – Evaluation field research (September - December 2021) 9 Pilot and finalize tools, and plan field logistics 10 10 Train end-line field enumerators 3 11 Coordinate field visits and make other logistical preparations 5 12 Conduct document review and analysis 10 13 Collect data 30 14 Process and analyze data 20 15 Hold USAID out-briefing 2 (MRP External Evaluation Annexes) Page 24 | 286 TABLE 6. ESTIMATED DURATIONAND SCHEDULE TASK NUMBER TASKNAME ESTIMATED DURATION (BUSINESS DAYS) Phase 3 – Draft and finalize evaluation report and conduct learning events (January – March 2022) 16 Draft final report 30 17 Submit final report draft to USAID/Philippines for comment 0 18 Revise final report draft based on USAID comments 5 19 Conduct first learning event: present findings, conclusions, and recommendations to USAID, IPs and major stakeholders 5 20 Revise final report based on learning event feedback 5 21 Submit final report to USAID/Philippines 0 22 Conduct second learning event: share findings and lessons learned with a wider audience of stakeholders, beneficiaries, and the interested public 5 23 Obtain final report approval from USAID/Philippines 0 24 Submit final report to the DEC and upload data to the DDL 0 FINAL REPORT FORMAT The final report will be a message-oriented document that will cover the nine sequential elements in USAID’s evaluation report template: 1. Abstract 2. Executive summary 3. Evaluation purpose 4. Background on the context and the strategies/projects/activities being evaluated. 5. Evaluation questions 6. Methodology 7. Limitations to the evaluation (MRP External Evaluation Annexes) Page 25 | 286 8. Findings, conclusions, and (if applicable) recommendations 9. Annexes The report should not exceed 40 pages, inclusive of a three-to-five-page executive summary. The evaluation methodology shall be explained in detail. Limitations to the evaluation will be disclosed, with particular attention to limitations associated with the evaluation methodology (e.g., selection bias and recall bias). The annexes to the report shall include the following: ● Evaluation SOW ● Evaluation design and work plan ● Statements of difference (if any) regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team ● All tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides. ● Sources of information (properly identified and listed) ● Disclosure of conflict-of-interest forms for all evaluation team members, either attesting to no conflicts of interest or describing existing conflicts of interest The principal investigator will ensure that the final evaluation report is publicly available through the USAID Development Experience Clearinghouse (DEC) within 90 calendar days of the official completion date listed in the evaluation contract. The approved evaluation report shall be uploaded to the DEC by the evaluators. Per ADS 579, all collated data shall be stored in electronically readable form and submitted to the Development Data Library (DDL). QUALITY ASSURANCE CRITERIA Per USAID ADS 201.3.5.17, draft evaluation reports must undergo a peer review organized by the office managing the evaluation. The following criteria will serve as the basis against which the report is reviewed: ● The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ● The evaluation report as a whole should be readily understood and should identify key points clearly, distinctly, and succinctly. ● The executive summary should present a concise and accurate statement of the most critical elements of the report. ● The evaluation report should adequately address all evaluation questions included in the SOW, or a revised version of the questions that resulted from documented consultation and agreement with USAID. (MRP External Evaluation Annexes) Page 26 | 286 ● The evaluation methodology should be explained in detail and sources of information properly identified. ● Limitations to the evaluation should be adequately disclosed in the report, with particular attention to limitations associated with the evaluation methodology (i.e., selection bias, recall bias, and unobservable differences between comparator groups). ● Evaluation findings should be presented as analyzed facts, evidence, and data. They should not be based on anecdotes or hearsay or should not simply be the compilation of people’s opinions. ● Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ● If recommendations are included, they should be supported by a specific set of findings, and should be action-oriented, practical, and specific. OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided via an electronic file in an easily readable format, as agreed upon with the contracting officer’s representative (COR). The data should be organized and fully documented for utilization by those who are not completely familiar with the activity or the evaluation. USAID will retain ownership of all datasets. USAID contractors must submit any dataset created or collected with USAID funding to the DDL in accordance with the terms and conditions of their awards. This is in keeping with Executive Order 13642 and the OMB Open Data Policy (M-13-13), which states that an agency’s “public data listing may also include, to the extent permitted by law and existing terms and conditions, datasets that were produced through agency-funded grants, contracts, and cooperative agreements.” (MRP External Evaluation Annexes) Page 27 | 286 Annex 2 EVALUATION DESIGN AND WORKPLAN (INCEPTION REPORT) Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 28 | 286 INCEPTION REPORT MARAWI RESPONSE PROJECT EXTERNAL PERFORMANCE EVALUATION USAID/Philippines Collaborating, Learning, and Adapting for Improved Development MAY 2022 (MRP External Evaluation Annexes) Page 29 | 286 ABSTRACT The Marawi Response Project (MRP) responded to the aftermath of the Marawi Siege that happened in March-October 2017. The siege, initiated by ISIS-inspired armed groups, displaced 369,196 people, and worsened the socio-economic conditions of the communities. It also destroyed properties, lifelines, and businesses. MRP is USAID’s development assistance response that addresses the long￾term rehabilitation need of the affected communities and the local governments in and around Marawi. The focus of the evaluation covers three (3) key questions to measure MRP's performance: Relevance, Effectiveness, and Sustainability. The evaluation will generate answers using a convergent mixed-methods design through a document review, qualitative FGDs and KIIs, and quantitative data through the end-line and MSME surveys using tablets with a pre-installed instrument. In addition, the evaluation findings will cover self-reliance, polarization, and public participation by IDPs and HCMs. The evaluation team will gather data from individuals, communities, and organizations in the twenty￾two municipalities and two (2) cities within Lanao Del Sur and Lanao Del Norte. The respondents come from various business locations, livelihoods, and professions in the affected communities. The external evaluation team is comprised of a pool of short-term technical experts and the MEL Team led by the CLAimDev Chief of Party. The USAID/Philippines and Panagora Home Office also oversee the conduct of the entire evaluation. (MRP External Evaluation Annexes) Page 30 | 286 INCEPTION REPORT Marawi Response Project External Evaluation May/2022 Contract No. 72049220D00004 Task Order 72049222F00002 (MRP External Evaluation Annexes) Page 31 | 286 CONTENTS ABSTRACT 29 ACRONYMS 33 EXECUTIVE SUMMARY 34 EVALUATION PURPOSE AND EVALUATION QUESTIONS 36 EVALUATION PURPOSE 36 EVALUATION QUESTIONS 36 BACKGROUND 39 DESCRIPTION OF THE PROBLEM AND CONTEXT 39 PROJECT DESCRIPTION 40 DESCRIPTION OF THE INTERVENTION AND THEORY OF CHANGE 41 MRP’S RESULTS FRAMEWORK 42 PERFORMANCE INFORMATION 43 EVALUATION METHODOLOGY 44 SCOPE OF THE EVALUATION 44 EVALUATION APPROACHES 45 PROJECT CYCLE PERSPECTIVE 45 KEY CONSIDERATIONS 46 GEOGRAPHICAL SCOPE AND LOCATION 47 UNITS OF ANALYSIS 47 SAMPLING METHODS 48 DATA COLLECTION METHODS 53 DATA COLLECTION TOOLS 54 DATA COLLECTION PROCEDURES 55 (MRP External Evaluation Annexes) Page 32 | 286 DATA ANALYSIS 56 ETHICAL CONSIDERATIONS 57 LIMITATIONS AND CONSTRAINTS 58 LEARNING AND DISSEMINATION PLAN 58 EVALUATION WORKPLAN 59 Inception Report Attachments 106 Attachment 1: CONSOLIDATED MEETING HIGHLIGHTS 108 Attachment 2: MRP EVALUATION DESIGN MATRIX 116 Attachment 3: SUMMARY OF MRP EVALUATION METHODOLOGY 120 Attachment 4: MRP ENDLINE SURVEY QUESTIONNAIRE 124 Attachment 5: DESIGN IN PRETESTING THE ENDLINE & MSME SURVEY QUESTIONNAIRES 142 Attachment 6: MICRO AND SMALL ENTREPRENEURS (MARAWI MSMEs) 146 Attachment 7: FARMERS, WOMEN AND YOUTH GROUPS 150 Attachment 8: TRAINING DESIGN FOR ENDLINE AND MSME SURVEYS 152 Attachment 9: KEY INFORMANT INTERVIEW (KII) 154 Attachment 10: INFORMED CONSENT (SURVEY) 158 Attachment 11: MRP EVALUATION WORKPLAN (Based on the Task Order and Milestone) 162 (MRP External Evaluation Annexes) Page 33 | 286 ACRONYMS BARMM Bangsamoro Autonomous Region of Muslim Mindanao CAME Complexity-Aware Monitoring and Evaluation CDCS Country Development Cooperation Strategy CLAimDev Collaboration, Learning, and Adaptation for Improved Development COP Chief of Party DNH Do No Harm ECOWEB Ecosystem Work for Essential Benefits FGD Focus Group Discussion HCM Host Community Members IKG In-Kind Grant IDP Internally Displaced Person KII Key informant Interview LGU Local Government Unit MARADECA Maranao People Development Center MRP Marawi Response Project MSC Most Significant Change MSME Micro, Small, and Medium Enterprises NGO Non-Government Organization PDP Philippines Development Plan PSA Philippines Statistics Authority PWD Persons with Disability SOW Statement of Work evaluation team Short Term Technical Assistance USAID United States Agency for International Development VEO Violent Extremist Organizations (MRP External Evaluation Annexes) Page 34 | 286 EXECUTIVE SUMMARY The Marawi Response Project (MRP) is part of USAID’s development assistance that supports the reintegration of IDPs by addressing the early recovery needs of displaced individuals and, more importantly, by addressing the long-term rehabilitation need of affected communities and the local governments in and around Marawi. The five-month Marawi Siege (March-October 2017), initiated by ISIS-inspired armed groups, displaced 369,196 individuals who found refuge in emergency shelters or with relatives or friends in surrounding areas. The siege worsened the socio-economic conditions of communities with the destruction of properties, lifelines, and businesses. The evaluation aims to gauge MRP’s performance in accomplishing its target outcomes by generating evidence-based learning from MRP’s experiences in the Philippines, especially in complex environments. The evaluation focuses on the essential lessons learned from MRP’s experience operating in complex, dynamic, and frequently high-threat environments. The target audiences of the evaluation are diverse groups and individuals: USAID/Philippines, USAID/Regional Development Mission for Asia, USAID/Washington, Plan International (MRP’s implementing Partner), local communities, local government units (LGUs), Marawi stakeholders, local community solidarity groups of Internally Displaced Persons (IDPs) and host communities, non￾governmental organizations, and donor agencies (both local and international), and Philippine national government agencies. The evaluation will focus on three (3) key evaluation questions to measure MRP’s performance. Relevance: Were MRP’s development interventions relevant to the needs of the IDPs and Host Communities and adaptive to the complex and changing external environment in Mindanao, especially in the project areas? Effectiveness: Did MRP accomplish the project’s expected outcomes of improving self￾reliance and strengthening social cohesion among IDPs and HCMs? Sustainability: Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen social cohesion between IDPs and HCMs? The evaluation will employ a convergent mixed-methods design. Focus group discussions (FGDs) and key informant interviews (KIIs) will be the foundation of primary qualitative data. The end-line and MSME surveys provide the foundation for quantitative data. Data from document reviews will supplement the primary data collected. The field enumerators will conduct the end-line and MSME surveys using tablets with pre-installed survey instruments. The evaluation team will code and thematically analyze data from FGDs, KIIs, and document reviews using qualitative content analysis conventions. The evaluation team will analyze findings from the mixed-methods data gathering approaches to formulate insights relevant to the evaluation questions related to relevance, effectiveness, and sustainability. In addition, the evaluation will underscore learnings from experiences covering self￾reliance, polarization, and public participation by IDPs and HCMs. The evaluation limitations include concerns on sampling, measurement, and non-conclusive issues. (MRP External Evaluation Annexes) Page 35 | 286 The evaluation will gather data from individuals, communities, and organizations in the twenty-one municipalities and two cities located within Lanao Del Sur and Lanao Del Norte, from a mix of business, livelihood, and employment opportunities pursued in the affected communities. The units of analysis of the evaluation will be the individuals disaggregated into gender, age, beneficiary type (IDP and HCM) and geography, organization/social group (women/youth/farmers), and business owners (MSME). The evaluation team has purposively selected respondents and their geographic locations given the evaluation’s limited time, logistics, and security considerations. This includes MRP partners – government and non-government, especially those implementing the micro￾grant component. The evaluation team will employ the most appropriate sampling methodologies in reaching the various respondents for this evaluation, given the limitations and constraints in time, resources, and security. Conducting two separate surveys, endline (following through the sample from the baseline survey amongst IDPs and HCMs) and MSME (business owners and traders in the most affected areas (MAA) in Marawi City), will measure the quantitative results of the interventions. Conducting FGDs and KIIs will validate the qualitative results of the same interventions. The evaluation duration will run for six months, beginning in April 2022, followed by the production of the learning materials and organization of the learning events after USAID accepts the final evaluation report. The evaluation team will disseminate the evaluation results at the learning events that CLAimDev organizes for USAID and other stakeholders. (MRP External Evaluation Annexes) Page 36 | 286 EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The evaluation will study MRP’s performance in accomplishing its outcomes and intermediate results covering the period of September 2018 to March 2022. It also aims to generate evidence-based learning from the study that will contribute to improving USAID’s development outcomes by: (1) Measuring MRP’s performance in terms of relevance, effectiveness, and sustainability in achieving its target outputs and outcomes; and (2) Learning from MRP’s experiences to provide valuable recommendations for USAID/Philippines and other key stakeholders in managing development projects in similar environments. The evaluation focuses on the important lessons from MRP’s experience operating in complex, dynamic, and frequently high-threat environments. These lessons will be valuable in guiding USAID/Philippines in evolving appropriate approaches for the design, management, monitoring, and evaluation activities implemented in contexts like those present in Mindanao during MRP’s implementation period. In collaboration with USAID/Philippines, CLAimDev will disseminate the evaluation results through a learning and dissemination event that a diverse target audience will attend. The target audiences of the evaluation are diverse groups and individuals such as the USAID/ Philippines, USAID/Regional Development Mission for Asia, USAID/Washington, Plan International, local communities, local government units (LGUs), Marawi stakeholders, local community solidarity groups of Internally Displaced Persons (IDPs) and host communities, non-governmental organizations, and donor agencies (local and international), and national government agencies. EVALUATION QUESTIONS The evaluation will answer the key questions on relevance, effectiveness, and sustainability. Table 1 illustrates the key evaluation and descriptive questions that the evaluation team will answer based on the following criteria.60 1. Were MRP’s development interventions relevant and adaptive to Mindanao's complex and changing external environment, especially in the project areas? 2. Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic activities among IDPs and HCMs? 3. Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs? 60 The evaluation team conducted a series of meetings with USAID, Plan International, and the Panagora Group to clarify points in the SOW and seek guidance on the evaluation context and processes. See Annex 2. (MRP External Evaluation Annexes) Page 37 | 286 TABLE 1. EVALUATION FOCUS CRITERIA KEY EVALUATION QUESTIONS DESCRIPTIVE QUESTIONS (FOCUS CRITERIA) Relevance: Were MRP’s development interventions relevant and adaptive to Mindanao's complex and changing external environment, especially in the project areas? • Sensitivity and Responsiveness to Gender and Social Inclusion Issues. In what ways have MRP’s development interventions been relevant and adaptive to the distinct needs of the vulnerable groups according to female/male and other social groups [e.g., Internally Displaced Persons (IDPs)/Host Community Members (HCMs), urban/rural, etc.]? • Being sensitive and responsive to context. In what ways have MRP’s objectives and design [Theory of Change (ToC), Results Framework (RF)] responded to the needs of the IDPs and host communities and in finding longer-term durable solutions to their displacement? • Adapting over time. In what ways were interventions being sensitive and adaptive to the complex and changing economic, environmental, social, political economy, and capacity conditions in the project’s environment? • Responding to policies and priorities. How aligned and coherent is MRP’s intervention with existing government priorities in responding to the Marawi Crisis and the USAID’s policy guidance in assisting Internally Displaced Persons, especially in transitioning them to safer communities and restoring livelihood and income opportunities? Effectiveness Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic • Sensitivity and Responsiveness to Gender and Social Inclusion Issues. In what ways do MRP’s actual outcomes demonstrate reduced gaps between gender groups and other social groups as identified in the baseline study? Did all the target IDPs and HCMs, including the most disadvantaged and vulnerable –women, children, and youth, benefit equally from the interventions? • Achieving the target outputs and outcomes. In what ways have MRP achieved its intended outputs and outcomes in improving the economic self￾reliance and social cohesion of IDPs and host communities? (MRP External Evaluation Annexes) Page 38 | 286 TABLE 1. EVALUATION FOCUS CRITERIA KEY EVALUATION QUESTIONS DESCRIPTIVE QUESTIONS (FOCUS CRITERIA) activities among IDPs and HCMs? • Significant Emerging Impact. Has MRP caused a significant change in the lives of the intended beneficiaries and their communities, such as improving economic self-reliance and strengthening social cohesion? Has MRP’s intervention transformed and created more enduring positive changes in the IDP’s perceptions and community norms/processes (whether intended or not) in responding to their situation? • Contribution to High-level effects. In what ways has MRP caused higher￾level effects, particularly in finding long-term and durable solutions in integrating the IDPs and host communities? Sustainability Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs? • Sensitivity and Responsiveness to Gender and Social Inclusion Issues. In what ways did MRP establish sustainability mechanisms that promote sustainable and equitable economic improvement and equal social participation among females and males and various social groups? • Continuity/Scalability of positive effects. In what ways has MRP established approaches to continue, scale-up, or replicate project outcomes on the IDPs/HCMs? • Building an enabling environment for sustainability of the project benefits. What mechanisms have evolved out of MRP interventions in creating the enabling environment to sustain and further improve economic conditions for beneficiaries and social cohesion between IDPs and HCMs? • Risks and potential trade-offs. To what extent do the various opportunities and risk factors enhance or threaten the sustainability of project benefits over time? (MRP External Evaluation Annexes) Page 39 | 286 BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT The Marawi Siege in 2017 was a prolonged conflict that resulted in the displacement of an estimated 369,196 individuals61 who found refuge in emergency shelters or who lived with relatives or friends in surrounding communities. The siege by the combined forces of both national and international ISIS￾inspired armed groups worsened the socio-economic conditions of communities, with the destruction of property, livelihoods, and businesses. The Lanao provinces have long been armed conflict areas between multiple groups and clans. When the project started, these provinces were among the poorest provinces in the country.62 The challenges faced by IDPs, and host communities are numerous and varied, ranging from increasing social vulnerabilities, continuing economic weaknesses, and cultural conflicts and susceptibilities. These challenges are exacerbated when IDPs and their host communities lack confidence in the local government’s ability to address the crisis. When local governments and national government agencies do not address these crises sufficiently, tensions between IDPs and their host community members (HCM) may evolve into internal conflicts that violent extremist organizations (VEO) can take advantage of to further destabilize the area. Given the historical conflicts and the continuous presence of armed groups (especially VEOs) in Mindanao, the fears and tensions among IDPs and their host communities increased during and after the Marawi siege. Various reports show numerous complexed challenges these IDPs and their host communities face in the emergency shelters and host families and communities63. These challenges are: • High vulnerability to and risk of illnesses and malnutrition because of inadequate shelter, food, and health services. • Increased number of out-of-school youth and children. • Low income and high unemployment because of lost jobs and livelihoods. • Social conflicts/tensions between host communities and IDPs. • Lack of resources and support systems from families, neighbors, and communities. • Limited access to livelihood opportunities and capital. Dependence on humanitarian assistance and host community resources. The 2019 MRP baseline study revealed significant differences in perceptions of self-reliance, polarization, and public representation between IDPs and host communities. The study also revealed significant perception differences between genders and geographical classifications (rural/urban). To summarize, the baseline study highlighted the following key findings: • Women have lower perceived self-reliance than men. 61 Based on estimated IDPs from 2018. Bangon Marawi Comprehensive Rehabilitation and Recovery Program. Manila, and Philippines: Humanitarian Bulletin, Issue No. 10, November 2018, Official United Nations Office for the Coordination of Humanitarian Affairs. The figure for the remaining IDPs is an estimate. 62 Philippine Statistics Authority (PSA) 2018. 63 Based on estimated IDPs from 2018. Bangon Marawi Comprehensive Rehabilitation and Recovery Program. Manila, and Philippines: Humanitarian Bulletin, Issue No. 10, November 2018, Official United Nations Office for the Coordination of Humanitarian Affairs. The figure for the remaining IDPs is an estimate. (MRP External Evaluation Annexes) Page 40 | 286 • IDPs have lower perceived self-reliance than HCMs. • Urban residents express lower levels of trust than rural residents. • Urban residents score public representation lower than do rural residents. • IDPs score public representation lower than do than HCMs. Ending displacement is a solution to the IDP’s situation. While waiting for this, USAID’s Marawi Response Project aimed to assist the IDPs and the host communities in finding ways to normalize their situations and take initial steps to prepare them to improve their lives and sustain these when they return to their homes. PROJECT DESCRIPTION The response of the USAID/PH Mission to the Marawi crisis began as part of USAID’s medium-term strategic goal to support the reintegration or return of Internally Displaced Persons (IDPs) through 2020, particularly those IDPs for whom the likelihood of returning to Marawi seems challenging. This plan had three objectives: a) address the early recovery needs of individuals displaced by the Marawi conflict, b) transition IDPs to social and economic stability; and c) establish the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs. MRP became a part of the second phase, contributing to the transition of IDPs to social and economic stability and establishing the enabling environment to address the long-term rehabilitation needs. After that, MRP became USAID’s primary Marawi response effort, integrating and building upon the initial work of USAID/PH in responding. The table below shows the summary information of the Marawi Response Project. TABLE 2. SUMMARY INFORMATION ACTIVITY NAME Marawi Response Project (MRP) IMPLEMENTING PARTNER PLAN International COOPERATIVE AGREEMENT NUMBER 72049218CA00007 TOTAL ESTIMATED COST (TEC) US$25,000,000 PERIOD OF PERFORMANCE September 2018–September 2021 (extended to March 2022) ACTIVE GEOGRAPHIC REGIONS Lanao del Sur, Marawi City, Lanao del Norte, and Iligan City COUNTRY DEVELOPMENT COOPERATION STRATEGY CDCS 2016-2019 CDCS 2020-2024 DEVELOPMENT OBJECTIVE DO2: Improved peace and stability in conflict-affected areas of the Philippines, particularly Mindanao (MRP External Evaluation Annexes) Page 41 | 286 As of September 2021, the MRP covers the provinces of Lanao del Norte and Lanao del Sur, the cities of Iligan and Marawi, the twenty municipalities of Lanao del Sur and the four municipalities of Lanao del Norte (Table 3). The municipalities of Lumbatan and Molando are additional sites reflected in the Y2 and Y3 annual reports, respectively. Tubaran is part of the original sites but not reflected in the list of project sites in Annual Reports from Year 1-3. TABLE 3. MRP PROJECT AREAS PROVINCE CITY/MUNICIPALITY Lanao del Norte (4) Iligan City, Baloi, Pantao Ragat, Pantar Lanao del Sur (20) Balindong, Buadiposo Buntong, Bubong, Butig, Ditsaan Ramain, Kapai, Lumba Bayabao, Lumbaca Unayan, Lumbatan, Lumbayanague, Madalum, Marantao, Marawi City, Masiu, Molando, Poona Bayabao, Piagapo, Saguiaran, Tamparan, Tugaya, DESCRIPTION OF THE INTERVENTION AND THEORY OF CHANGE MRP aims to solidify a “social contract” between the constituents and the local government and increase IDPs’ and host communities' “social cohesion.” Considering the historical context of armed conflicts in the area, MRP uses a conflict-sensitive approach in its implementation without weakening the activity’s ability to assist, empower and unite the IDPs and host communities to work together in identifying their problems and implementing solutions that will move them to the next desirable state. The Theory of Change/Development Hypothesis states: “If IDPs, returnees, and host populations actively participate in and contribute to the social and economic development of their communities in a manner that reduces inter and intra community tensions and fosters resilience, then they will be more self-reliant and able to advance solutions to their displacement.” (MRP External Evaluation Annexes) Page 42 | 286 The results framework of MRP points to a durable solution: local integration of IDPs as high-level impact. The manageable impact statement is “self-reliance of IDPs and host community members improved.” Figure 1 illustrates the MRP results framework, and Table 4 summarizes the framework. MRP’S RESULTS FRAMEWORK The table below shows the results framework illustrating MRP’s level of impact, results, outputs, and indicators. TABLE 4. MRP RESULTS FRAMEWORK RESULTS LOGIC RESULTS STATEMENTS HIGH-LEVEL IMPACT: Durable solution: local integration of IDPs MANAGEABLE IMPACT: Self-reliance of IDPs and host community members improved. INTERMEDIATE RESULTS AND OUTPUTS: IR 1: Economic conditions of IDPs and host communities improved. Output 1.1: Business recovery opportunities expanded. Output 1.2: Employment opportunities expanded. Figure 1: Illustration of MRP’s Results Framework (MRP External Evaluation Annexes) Page 43 | 286 TABLE 4. MRP RESULTS FRAMEWORK RESULTS LOGIC RESULTS STATEMENTS Output 1.3: Livelihood opportunities expanded IR 2: Social cohesion of IDPs and host communities strengthened. Output 2.1: Social cohesion grants to host and displaced communities. Output 2.2: Social Cohesion training PERFORMANCE INDICATORS: • Indicator 1: Level of self-perceived self-reliance of assisted displaced and host community members. • Indicator 2: Number of displaced business owners with new or re￾started businesses • Indicator 3: Percent of trained internally displaced persons/host community members gainfully employed. • Indicator 4: Number of displaced business owners who benefitted from business recovery micro-grants. • Indicator 5: Number of displaced business owners trained in enterprise management. • Indicator 6: Number of displaced persons/host community members trained in workforce readiness and life skills. • Indicator 7: Number of displaced/host community members who benefitted from the micro- grants. • Indicator 8: Level of polarization towards the other identity group • Indicator 9: Level of self-perception of public representation • Indicator 10: Number of displaced/host community members who benefitted from the social cohesion grants. • Indicator 11: Number of displaced/host community members trained. PERFORMANCE INFORMATION Table 5 lists some of MRP’s accomplishments through March 2022, based on the evaluation team’s initial document review. (MRP External Evaluation Annexes) Page 44 | 286 TABLE 5. PROJECT ACCOMPLISHMENTS AS OF MARCH 2022 1 2,797 displaced business owners who have benefitted from business recovery micro￾grants 2 845 displaced business owners trained in enterprise management 3 99% have completed USG-assisted workforce development programs 4 4,488 IDPs and HCMs have benefitted from livelihood micro-grants 5 52,140 IDPs and HCMs provided assistance 6 5,401 IDPs trained on civic engagement and other social cohesion-related subjects. MRP has also implemented several adaptations to respond to significant changing events that affected the project implementation. Among these significant events are the 2019 transition to the Bangsamoro Autonomous Region of Muslim Mindanao (BARMM), the 2022 local and national elections, and the Covid19 pandemic in 2020. Local violent events such as “rido” or clan conflicts have also occurred during the project life. The evaluation team will include in its study how MRP adapted implementation of the program in these changing contexts. EVALUATION METHODOLOGY This section discusses in detail the evaluation design. Annex 3 summarizes the design in a matrix form (MRP Evaluation Design Matrix). SCOPE OF THE EVALUATION The evaluation will gather information from the different aspects of the project that are relevant to answering the key evaluation questions. The following are the project elements that the evaluation team will review to better understand the internal and external factors affecting the project performance. General Environment. The general environment includes various external factors that influence MRP’s implementation and results. These environmental factors include significant political, economic, socio-demographic, technological, legal, and physical/environmental changes. The evaluation team has initially identified these occurrences, such as the 2019 BARMM transition, 2019 & 2022 local/national elections, and the Covid-19 pandemic. The evaluation also will consider other contextual issues such as the arrival of government and international donor development assistance in the areas and updated data from the Philippine Statistics Authority (PSA)’s poverty-related statistics from 2018 to the present. The evaluation team will assess how these factors have contributed to MRP’s performance. (MRP External Evaluation Annexes) Page 45 | 286 Specific Environment. The specific environment refers to the beneficiaries, families, communities, organizations, leaders, business sector groups, local government units, government organizations, NGOs, and other important actors, directly and indirectly, involved in implementing MRP. Overall, Purpose. The general and specific purposes of the activity include the theory of change/development hypotheses, results framework, indicators, and all other information related to institutional mandates that guide the activity implementation. Actual Outputs/Outcomes. Outcomes refer to information about MRP’s intended and unintended results. This evaluation will thus consider and integrate the three outcome indicators that the MRP baseline study highlighted, namely: level of self-reliance, level of polarization between host and displaced communities, and level of participation (i.e., the voice or level of representation of IDPs in their new locality). The level of analysis will be both at the individual (i.e., IDPs and host community members) and group (e.g., MSME and community levels) Inputs and Operations. The information gathered from these two elements deals with internal project management and relates to efficiency. Although the evaluation will not focus on this, the team will consider information related to this, especially on how MRP managed the project implementation, especially in changing and complex situations. The succeeding sections explain this further in the project cycle approach. Although efficiency is not a focus criterion, the evaluation will note efficiency aspects related to operational processes when relevant to answering the evaluation questions. EVALUATION APPROACHES The evaluation team will adopt a comprehensive approach in undertaking the evaluation. It will cover the implementation and outputs of the project up to March 2022. They will employ the project cycle approach and conduct the research in a participatory, transparent, inclusive, and consultative manner together with concerned stakeholders. Necessarily, the evaluation will consider multiple levels of analysis, including the community (Barangay), municipal, city, provincial, and national levels, as appropriate for answering the evaluation questions. PROJECT CYCLE PERSPECTIVE This evaluation will use the project cycle method. This method is all-inclusive because it looks at the project's processes, products, performance, and design. In some cases, project design affects the implementation performance. When the team uses the project cycle method, they will look at the following: 1. Project Design. The project design review will focus on the relevance of the design considering the prevailing situation at the time of formulation; its alignment with local, national, and international policies; and the appropriateness of the interventions. 2. Project Operation and Implementation. Various factors influence the effectiveness, results, and outcomes of projects. Invariably, operational bottlenecks come up and cause delays. Therefore, the evaluation team will review the following aspects of the project: a. Management systems and practices at project office and field levels. Weaknesses in project management capacities in implementing agencies pose risks in project quality, cost, and schedule. Lengthy review and approval procedures, unclear and highly discretionary assessment and approval criteria, delays in the review and approvals of project activities, highly centralized decision-making, and weak monitoring and control pose quality, timeliness, and (MRP External Evaluation Annexes) Page 46 | 286 cost problems, among others. The evaluation team is aware that the project has instituted various systems to improve focus, efficiency, and effectiveness in project operations. b. Issues related to the procurement of goods and services for the project. c. Operational planning and project governance d. Project resource allocation, disposition, availability, and timeliness e. Monitoring and reporting issues that may delay appropriate and immediate response by management on operational issues. The evaluation team will pay special attention to tracking the progress of crucial output performance indicators that directly influence the achievement of results and outcomes. 3. Sustainability and Exit Plan – The evaluation team will review the project's sustainability and exit strategy plan (if available) to determine its appropriateness and adequacy in sustaining the gains and institutionalization of products, systems, and practices. There are pre-conditions to a successful exit and sustainability strategy. a. First, the project must provide the resources required (for example, staff, budget, equipment, and technology) to enable continued operations. b. Second, there should be sufficient capacity in terms of staff competencies and sustained commitment to continue implementing the systems and practices. c. Third, the project should institutionalize the preconditions for sustainability, such as integrating the project’s policies and processes into formal policies and regulatory regimes. An effective project strategy anchors this approach into its work to ensure a smooth transition from assisted implementation to self-managed and institutionalization. Apart from undertaking the activities cited in the TOR, the evaluation team will assist in exploring, identifying, and recommending measures to enhance the sustainability of gains. 4. Knowledge Management – The evaluation team will review the systems and procedures and their implementation to determine their capacity to generate information and new knowledge. The review will cover how MRP collects, organizes, processes, shares, and disseminates information with stakeholders. MRP has numerous partners and has produced valuable lessons and products which may be found useful in its partners’ operations. The study will review its knowledge management processes. KEY CONSIDERATIONS Given the complex situation in the project area, the following principles and considerations will guide the evaluation team in conducting the evaluation. 1. Performance-Focus. The evaluation team will assess MRP’S effectiveness in accomplishing the goals, objectives, and outcome indicators (level of self-reliance, polarization between IDPs and HCMS, and participation. 2. Transitional Nature of MRP. MRP assistance to beneficiaries is transitional and aligned with the next state of beneficiaries rather than aiming for their end state. The assistance focuses on finding long-term or durable solutions and preparing them for their desired permanent settlements. It is important to note that Bangon Marawi Task Force is undertaking massive rehabilitation, recovery, and reconstruction efforts in Marawi. (MRP External Evaluation Annexes) Page 47 | 286 3. Complexity-Awareness. The project area environment is complex, and the Covid-19 pandemic and other events such as the recent national elections and conflicts among tribal groups or clans exacerbate this complexity. The evaluation team will use USAID’s Complexity-Aware Monitoring and Evaluation Approaches (CAME), particularly the Most Significant Change (MSC) and Network Analysis tools. We have embedded this awareness in all the evaluation tools. 4. Gender Equality and Social Inclusivity. The evaluation will consider the gender equality and social inclusion dimensions of MRP. Further, the evaluation will cover social equity such as the rural/urban, IDPs/HCM, youth, PWD, and elderly dimensions. A gender specialist will advise the evaluation team on integrating these concerns into the evaluation tools and processes. 5. Conflict Sensitivity and Do No Harm (DNH) Approach. The evaluation team will be conflict￾sensitive given the historical conflict situation in the project area. Some violent incidents occasionally occur, particularly among warring clans/tribes. The evaluation team, including the enumerators, will observe DNH and use it as a lens to identify conflict-triggering effects of the evaluation processes. 6. Transparency. The evaluation team will ensure transparency and create safe spaces to reduce tension and encourage open dialogue and sensitivity in sharing information. 7. Safety. Because the project area is prone to conflict, the evaluation team will prioritize security. In choosing sample barangays and municipalities, safety will be a major consideration. The team will continue to receive USAID’s security and public health updates in Mindanao, regularly distributed to all implementing partners, and adheres to the safety instructions in ADS Chapter 303 (Operational Security – General Information and Additional Help for Chapter 303). The evaluation team will observe the above principles and considerations in the evaluation design, data collection and analysis, and when communicating the evaluation results. GEOGRAPHICAL SCOPE AND LOCATION MRP’s interventions have reached the IDPs, and host communities located in twenty-two (22) municipalities and two (2) cities in the Lanao del Sur and Lanao del Norte provinces. Given the constraints of time and resources, the external evaluation will focus on carefully selected areas to collect data that is representative of the entire MRP intervention areas. The criteria for selecting the evaluation sample areas are the following: 1. Presence of both IDPs and host communities receiving assistance from MRP 2. The extent of project assistance provided by MRP (in terms of resources, efforts, etc.) 3. Presence of mixed types of beneficiaries such as trained individuals, entrepreneurs, employed persons, etc. 4. Existence of organized community solidarity groups. 5. Presence of baseline survey respondents The evaluation team will also consider security risks, road accessibility, and communication connectivity in selecting sample areas. UNITS OF ANALYSIS (MRP External Evaluation Annexes) Page 48 | 286 The units of analysis of the evaluation will be individuals, organizations or groups, and geographical locations. The evaluation team will disaggregate the data, as shown in Table 6. TABLE 6: UNITS OF ANALYSIS UNITS DISAGGREGATION Individuals • Gender (Male/Female) • Beneficiary type (IDP/HCM) • Age (Youth/Adult) • Geography (rural, urban) • Project design (Intervention/Control) • For entrepreneurs – micro, small, and medium levels Group • Community solidarity groups • Sectoral Organizations (youth, women, etc.) • Economic-related organizations (livelihood clusters, cooperatives, etc.) SAMPLING METHODS The evaluation team considered appropriate sampling methodologies in identifying the various respondents for this evaluation, given the limitations and constraints of time, resources, accessibility, and other risk factors in the locality. Table 7 summarizes the number of participants according to the types of data collection methods. (MRP External Evaluation Annexes) Page 49 | 286 TABLE 7. SUMMARY OF DATA COLLECTION METHODS TYPE METHODS RESPONDENTS FREQUENC Y OF CONDUCT NUMBER OF PARTICIPANTS QUANTITATIVE SURVEY ENDLINE • Intervention 1 358 • Comparison 1 44 MSME • Intervention 1 369 QUALITATIVE FGD** GOs - - NGOs - - POs 17 136 MSME Traders 2 16 Livelihood Clusters (IDP/CHM - CSGs) 5 40 Social Groups (farmers, women, and youth) 15 120 KII GOs 28 50 NGOs 3 3 Academe 1 1 MSME 10 10 DOCUMENT REVIEW n/a n/a n/a (MRP External Evaluation Annexes) Page 50 | 286 Endline Survey Sampling Method. The endline survey samples will seek to replicate the 2018 baseline survey sample, which consisted of the intervention group (n=358) and comparison group (n=44) using quota and snowball sampling. Once the team gathers more information regarding the design and actual implementation of the baseline survey, the team will specifically discuss in the evaluation report MRP’s reasoning for the baseline sample size and the manner in selecting and keeping track of the respondents that belong to the comparison group. The evaluation will determine the validity of the comparison group and how it will use the results from the group to answer the evaluation questions, particularly in measuring the effects of the project. If a comparative analysis is not appropriate, the evaluation will employ alternative methods, such as longitudinal analysis, which will use baseline and endline data results from the intervention group to answer the evaluation questions. Depending on the baseline data behavior, the evaluation team will also explore the most appropriate methodologies, such as the “panel-like sampling” and the use of substitution for determining the respondents. This applies when the baseline respondents are no longer available for the endline survey. The table below shows the geographic distribution of the baseline respondents. The evaluation team will follow up with these people for the end-line survey. MRP's baseline survey design information will guide the evaluation of how this distribution will work. TABLE 8. GEOGRAPHICAL SCOPE OF ENDLINE SURVEY Endline Survey Groups Municipality /City Barangay Number of Participants Total Project Group Marawi City Mipaga 38 358 East Poblacion 40 Pagalamatan 39 Lumbaca Toros 39 Cadayonan 39 Iligan City Sto. Rosario 40 Del Carmen 24 (MRP External Evaluation Annexes) Page 51 | 286 Tubod 50 Buruun 15 Masiu Matao Raza 34 Control Group Marantao Inudaran Lumbac 20 44 Iligan City Hinaplanon 24 MSME survey sampling method. The evaluation team will survey a sample of MRP's enterprise, livelihood, and employment beneficiaries. This MSME survey will especially cover business owners and traders who lost their establishments during the Marawi Siege. To determine the survey sample size, the evaluation will consider the total number of people who received MRP’s business recovery microgrants, enterprise management training, workforce development programs, and livelihood microgrants using a 95% confidence level and 5% margin of error. Since many of the businesses of some micro-grant beneficiaries are no longer operating or are in non-MAA areas in Marawi City, the purposive sampling method will also guide the evaluation in selecting samples from the existing/operating enterprises within the MAA (most affected areas). Random sampling is impractical; instead, the evaluation will use quotas to create a reliable sample for the MSME survey. Below is the computation of the indicative sample size for the MSME survey. Once the evaluation team confirms the total number of USG-assisted workforce development completers, they will recalculate the sample. TABLE 9. DISTRIBUTION OF MSME SURVEY RESPONDENTS Types of Project Recipients Accomplished as of March 2022 Sample Size % Distribution Total 8,130 369 100% Displaced business owners who have benefitted from business recovery micro-grants 2,797 115 31% Displaced business owners trained in enterprise management 845 35 9% IDPs and HCMs have benefitted from livelihood micro-grants 4,488 184 50% (MRP External Evaluation Annexes) Page 52 | 286 Number of individuals having completed USG-assisted workforce development programs 35 10% FGD and KII Sampling Methods. The evaluation team will select participants for FGDs and KIIs purposively. The evaluation team will carefully choose the people who will participate in the FGDs and KIIs. In general, the evaluation team will talk to people who are familiar with MRP activities and can give relevant information and their thoughts on those outcomes on the beneficiaries and their communities. There will be 22 FGDs with no more than eight people in each group and 26 KIIs. For the FGD, the team will adopt the snowballing technique in identifying the participants. They will ask KII and survey respondents to assist in identifying the participants based on the criteria below. The evaluation team will cross-reference the recommended participants with Plan International’s beneficiary list. The evaluation will use the following considerations when assembling the focus groups: 1. Participants shall be beneficiaries of MRP’s IR1 and IR2 activities. 2. Each FGD session shall be composed of more women than men. 3. For FGDs with youth groups, participants should be at least 18 years old. 4. For FGDs with MSME representatives, participants should have existing businesses operating for at least six months. 5. Participants should be in minimal to no-risk areas. 6. Participants expressed consent to participate through a signed informed consent form. The evaluation will use the following considerations when choosing interviewees: 1. For KIIs about livelihood, interviewees should be beneficiaries of MRP’s activities on IR1 and IR2. 2. For LGUs, interviewees should be incumbent officers or their duly appointed representatives. 3. For MSME representatives, interviewees should have existing businesses operating for at least six months. 4. There should be more women than men. 5. Interviewees should be in minimal to no-risk areas. 6. Interviewees expressed their consent to participate through a signed informed consent form. Table 10 describes the distribution of the FGDs and KIIs according to the types of respondents. The evaluation team will finalize the FGD groupings based on Plan International’s master list that includes beneficiary profiles. (MRP External Evaluation Annexes) Page 53 | 286 TABLE 10. GEOGRAPHICAL SCOPE OF FGD AND KII Municipality FGD KII Frequency Number of Participants Frequency Number of Participants Iligan - Livelihood/Community Solidarity Groups 2 16 2 2 Marawi – Livelihood/Community Solidarity Groups 3 24 3 3 Marawi - MSME 2 16 10 10 LGU Officers 9 18 Youth Groups (Survey municipalities) 5 40 Women’s Groups (survey municipalities) 5 40 Farmers’ Groups (survey municipalities) 5 40 Academe (MSU) 1 2 Consensus Group 1 3 DATA COLLECTION METHODS The evaluation will employ a convergent mixed-methods design. This design will facilitate the generation of both qualitative and quantitative information simultaneously. FGDs and KIIs will generate the primary qualitative data. Data from document reviews will supplement the KIIs and FGDs. The end-line and MSME surveys will generate the primary quantitative data. The evaluation team will converge and triangulate data from these multiple sources to develop insights into how the MRP performed based on the evaluation criteria. The evaluation team will observe principles related to complexity-aware evaluation, gender, and cultural sensitivity and do no harm in all facets of the evaluation. (MRP External Evaluation Annexes) Page 54 | 286 The mixed-method nature of the evaluation design promotes the utilization of a battery of data gathering approaches. The table below depicts the data collection methods and each method’s objective in the evaluation. TABLE 11. SUMMARY OF DATA COLLECTION METHODS AND OBJECTIVES METHODS OBJECTIVES Quantitative Survey Endline To gather data on the socio-demographics of IDPs and HCMs To measure the level of self-perceived self-reliance, self-perceived level of polarization, and self-perceived level of public representation of IDPs and HCMs To determine changes in the level of self-perceived self-reliance, self￾perceived level of polarization, and self-perceived level of public representation of IDPs and HCMs based on baseline data MSME To determine the types, locations, and amounts of micro-grants given to the affected traders in Marawi City. To establish the reasons for the businesses that failed to re-start and continue despite MRP’s In-Kind Grants. To assess the contributions made by the businesses that re-started and continued to succeed and reap profits. Qualitative FGD To document impressions regarding the effectiveness, relevance, and sustainability of MRP’s outcomes, results, and effect from representatives of GOs, NGOs, POs, MSME traders, livelihood clusters (IDPs/HCMs), and social groups (Women/Youth/Farmers) KII To document impressions regarding the effectiveness, relevance, and sustainability of MRP’s outcomes, results, and effect from key stakeholders, including GOs, NGOs, POs, MSME Traders, and the Academe. Document Review To gather information pertinent to MRP’s project design, operation and implementation, and knowledge management. To gather documentary evidence that can triangulate data from other methods to better evaluate MRP’s effectiveness, relevance, and sustainability. DATA COLLECTION TOOLS (MRP External Evaluation Annexes) Page 55 | 286 End-line Survey Instruments. The end-line survey will use the baseline survey tool that MRP used at the start of the project. The tool measures IDPs and HCMs’ socio-demographics, level of self-perceived self-reliance (Manageable Impact; Indicator 1), self-perceived level of polarization (Intermediate Result 2; Indicator 8), and self-perceived level of public representation (Intermediate Result 2; Indicator 9). The instrument underwent a content validation process and reliability testing during the baseline. See Annex 5 (MRP Endline Survey Questionnaire). MSME Survey. The MSME survey will utilize a tool developed by the evaluation team’s MSME Specialist. The tool will measure the effects of MRP’s micro-grants on the recovery and improvement of the livelihood and enterprise of the beneficiaries. The survey data will be relevant in measuring MRP’s intermediate results on improving economic conditions (Intermediate Result 1) and social cohesion (Intermediate Result 2) of IDPs and HCMs. The evaluation will validate the MSME survey tool during a survey pretest. The endline and MSME survey tools will be pretested on a small sample of respondents to examine their clarity, answerability, and response latency. Determining response latency which indicates the time needed by participants to complete answering items, helps estimate survey completion time. This would, in turn, inform the team in refining the strategies for the fieldwork. For the MSME survey, the evaluation team will use information from the pretest to enhance the instrument. For the endline survey, the evaluation will pilot the survey with fifty respondents to determine its reliability characteristics. The evaluation team will note the information from the endline survey pretest and pilot in the report but will not change the tool based on the pretest and pilot to ensure consistency with the baseline survey. The design illustrated in Annex 6 will guide the pretest processes. Eighteen field enumerators will collect the data for both the endline and MSME surveys using the survey instruments installed on the tablets. FGD and KII Guides. The evaluation team will conduct FGDs and KIIs using its guides. The guides will help conduct the FGDs and KIIs to gather information such as experiences and opinions about MRP’s performance from the group of entrepreneurs, livelihood owners, and community leaders from different sectors such as women, youth, farmers, and project implementors. Annex 7 includes the FGD Guide Questions on MSME, Annex 8 includes the FGD Guide for farmers, women, and youth groups, and Annex 9 includes a separate KII Guide for the financial institutions, chambers, and financiers. DATA COLLECTION PROCEDURES CLAimDev recruited field coordinators and enumerators for the endline and MSME survey. All the selected coordinators and enumerators are local residents who are familiar with the evaluation's geographical scope and context. This survey personnel will undergo training from May 31 to June 3, 2022. During the training, they will pilot test the instruments as a hands-on exercise in their training. Annex 10 includes the Training Design. The evaluation team will field the coordinators and enumerators in the specified cities and municipalities. Consultation, coordination, and integration among team members will include the utilization of online meeting platforms. The enumerators will upload survey data to a dedicated cloud space every time there is Internet access each day. The evaluation intends to conduct the end-line and MSME surveys from June 6 to June 24, 2022. (MRP External Evaluation Annexes) Page 56 | 286 The evaluation team will conduct the FGDs and KIIs. CLAimDev, Plan International, and the evaluation team will work together to coordinate the sites and logistics for the visits, given security concerns in many areas where MRP works. If there are safety concerns in an area, the evaluation team will substitute the sample location and group with a similar group in a more secure location. There will be no more than eight participants in each FGD, and the FGDs will have a duration of between 60 and 90 minutes. Each KII will be with one or two participants and have a duration of approximately 60 minutes. The team will do KIIs to learn more about MRP's results, outcomes, and manageable impacts. In the evaluation report, narratives from KII will tell stories that are not told through other types of evidence and as examples of findings derived from the other data collection methods. The evaluation team will conduct the KIIs, and each KII will have a duration of approximately one hour. Before conducting the survey, enumerators will obtain all required consents from survey respondents, including informed consent. DNH, gender/cultural sensitivity, and complexity-aware evaluation will guide these activities. To protect respondents’ identities, survey instruments will not include their names. Field coordinators and enumerators will sign a non-disclosure agreement and will not store participant data on personal devices. DATA ANALYSIS Quantitative. The team will analyze data from the end-line and MSME surveys using descriptive and inferential statistical approaches. The team will generate means, standard deviations, frequency distribution, and percentage distribution of the survey variables. The team will consider panel data analysis, tests of difference, and effect-size analysis in comparing baseline-endline data. The team will develop graphical displays using pertinent descriptive results. They will also perform inferential tests that would determine significant changes in the variables in the MRP results framework and matrix. The analysis also will include network analysis to delve into the associations between the participants ’responses in the self-reliance, polarization, and public participation items. Moreover, the analyses will disaggregate data based on identified social groupings. The evaluation team will use the JASP statistical package to perform these analyses. Qualitative Analysis. The evaluation team will use qualitative analytic approaches for the data from FGDs, KIIs, and document reviews. They will transcribe the audio or video recordings of FGDs and KIIs, tag and code the data using NVIVO software and use content analysis to understand the data. Among the main codes that the analysis will use are those related to the (a) evaluation criteria, namely, effectiveness, relevance, and sustainability, and (b) project impact/outcome, namely, socio￾economic and social cohesion. Based on the transcripts, the evaluation will develop other relevant codes. (MRP External Evaluation Annexes) Page 57 | 286 Results Convergence. The team will weave together results from the mixed-methods data gathering approaches to formulate insights relevant to the evaluation questions related to relevance, effectiveness, and sustainability. The convergence of results will be guided by the framework espoused by Creswell (2014), as shown below: In particular, the STTA team will compare the results using the matrix below to triangulate the evaluation findings from the qualitative and quantitative methods. The matrix suggests a process of converging quantitative and qualitative evaluation data collection results. These results will be organized in the matrix to help identify consistencies and contradictions in the results and generate more insights. Consistencies in the quantitative and qualitative results will substantiate the findings. The analysis will subject contradictions to further analysis to support understanding why the quantitative and qualitative results are inconsistent. This process of converging quantitative and qualitative results will help facilitate the generation of more robust evaluation findings and insights. TABLE 12. QUALITATIVE AND QUANTITATIVE FINDINGS CONVERGENCE MATRIX Project Outcomes Method Evaluation Criteria Insights Relevance Effectiveness Sustainability Qualitative Self￾Reliance Quantitative Economic Conditions Social Cohesion Insights ETHICAL CONSIDERATIONS The evaluation team will ensure that ethical conventions in program evaluations are adhered to. The team will secure informed consent from all participants in FGD and KII and all survey respondents as part of the data gathering process. The evaluation team will prepare the Participant Informed Consent Forms customized for various types of respondents. Moreover, the team will observe cultural sensitivity and DNH principles in the entire spectrum of the evaluation process. See Annex 11 (Informed Consent Form) Figure 2: Results Convergence Framework (MRP External Evaluation Annexes) Page 58 | 286 The participants will receive information during the informed consent process about the: (1) purpose of the evaluation, (2) the extent of their involvement, (3) potential risks and discomforts that they might encounter, (4) their right to refuse or discontinue participation, (4) potential benefits of the evaluation, (5) confidentiality of their responses, (6) cost, compensation, and reimbursement in case they participate. Data gathering will ensue only when the participant expresses voluntary participation. Following the survey administration process that the baseline survey used, the evaluation team also will seek consent from the respective barangay council officials. LIMITATIONS AND CONSTRAINTS The evaluation team notes some limitations. The intervention group's sample size is larger than the comparison groups. It may be difficult to determine the comparison group’s exposure to MRP or similar activities. Some IDPs/HCMs may have undocumented relocations by the end-line survey, making it difficult to follow up with all baseline assessment participants. These two sampling-related factors could affect the analytical plan of comparing baseline and end-line survey data, such as self￾reliance, polarization, and public representation. The team also observed that the baseline survey tool underwent content validation and reliability testing. However, the tool’s factor structure and construct validity are not well documented. The importance of and strategies for factor structure analysis and construct validation involving tools that measure identified constructs, such as those measured by the endline survey tool (i.e., perceived self-reliance, perceived level of polarization, perceived level of public participation), has been highlighted in the literature.64,65 Considering that performing construct validation at this juncture of MRP’s project cycle is not feasible, the team will note the possible implications of this methodological aspect in the findings of the present evaluation. These are the initial observations that the team will consider when putting this methodology into practice. For example, the possible effects of the first and second observations on actual data characteristics (like group variances) could be considered by changing analytical methods to create parameters for unequal group variances. The third observation could be dealt with by looking at the baseline assessment data to learn how the survey designers constructed the survey tool. LEARNING AND DISSEMINATION PLAN The evaluation team will work closely with CLAimDev Learning and Dissemination team to organize and conduct two learning events for key stakeholders. The first event will be for the local stakeholders in the MRP implementation areas, and the other will target national stakeholders. The learning events will disseminate evaluation findings, highlight good practices and lessons learned 64 Tsang S, Royse C, Terkawi AS. 2017. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5463570/pdf/SJA-11-80.pdf. 65 Yusoff MSB, Arifin WN, Hadie SNH. 2021. ABC of questionnaire development and validation for survey research. https://eduimed.usm.my/EIMJ20211301/EIMJ20211301_10.pdf. (MRP External Evaluation Annexes) Page 59 | 286 related to key MRP thematic foci, and present ways to enhance the sustainability of MRP's results. CLAimDev will use the evaluation report to develop learning materials that may include a web-based report, infographics, and a video explainer of key evaluation concepts or results. EVALUATION WORKPLAN The evaluation duration will be six (6) months (see Annex 12), followed by the learning events and dissemination. The workplan consists of the following milestones: Preparatory Phase (April 9 to May 23). This key output is this Inception Report incorporating the Evaluation Design, Work Plan, and Inception Presentation from the inception report briefing. Data Collection (May to June). The key output is a Progress Report indicating the completed data gathering activities for the evaluation and a series of update meetings with USAID and CLAimDev. The planned dates for this are a five-to-six-week period beginning at the end of May through the end of June. Data Processing (July to mid-August 2022). The evaluation team will complete the data analysis and validate their initial findings during this period. The key output is the presentation of the initial evaluation results during the out-brief presentation with USAID. Draft Report (mid-August to September). The key output is the submission of the draft evaluation report. Final Report and Dissemination during Learning. (September 2022). The evaluation team will submit the final report in September. Once the USAID approves the final report, CLAimDev will produce learning materials and organize the learning events. (MRP External Evaluation Annexes) Page 60 | 286 MRP INCEPTION REPORT ATTACHMENTS (Annex 2) Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 61 | 286 INCEPTION REPORT ATTACHMENT 1: CONSOLIDATED MEETING HIGHLIGHTS MARAWI RESPONSE PROJECT EXTERNAL EVALUATION HIGHLIGHTS OF THE EVALUATION TEAM CHECK-IN MEETINGS WITH USAID & PLAN INTL. APRIL 20, 21 & 25, 2022 | GOOGLE MEET OVERVIEW The series of check-in meetings were initiated to introduce the CLAimDev External Evaluation Team to USAID and Plan International and discuss concerns that are important for the inception report. The attendees during the first meeting were: Albert Aquino (USAID-PRM), Susan Minushkin/Noriel Sicad/Christie Lumaad (CLAimDev), and Johnson Mercader/John Curada/Roberto Palo/Heizel Rapista (CLAimDev External Evaluation Team). At the second meeting, the previous attended the meeting with the addition of Ria Orca, USAID MRP AOR. The MRP key staff led by Marlon Libot, MRP Chief of Party joined the 3rd meeting together with those who attended the previous meetings. HIGHLIGHTS The table below shows the important matters discussed during these meetings: HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Evaluation Questions The suggested tweaks of the STTA aimed to sharpen the evaluation questions without losing their focus and essence. (Refer to attached) CLAimDev to check the implications of the changes vis-à-vis the approved SOW/TO and document these for future reference. Performance Period covered MRP project life was September 2018–September 2021 (first extended to March 2022). The project is extended further to March 2023. The evaluation will cover MRP’s accomplishments as of March/April 2022. It will also be referring to the initial phase (level 1 humanitarian response). It was clarified that MRP’s project was not part of the initial phase of humanitarian assistance but was designed for the early recovery intervention. CLAimDev to document this change on the covered period for future reference. CLAimDev will request the most recent QPR when available. (MRP External Evaluation Annexes) Page 62 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Durable Solutions in the Results Framework The evaluation is a performance evaluation that will show the basic indications of MRP’s differential outcomes. “Durable solutions” is MRP’s impact as presented in the results framework. Assessing this may require an impact assessment, which is not the scope of the endline evaluation. Probing self-reliance becomes more attainable in research scope than measuring durable solutions. Self-reliance is introduced as an intermediate goal in the MRP results framework. The level of self-reliance of assisted citizens was included in the baseline survey (not durable solutions). CLAimDev STTA to look further at how to approach or not approach measuring “durable results.” Further discussions and probing are needed on the emerging outcomes like “self-reliance” Baseline Survey Survey Design: The baseline survey methodology was formulated with a group of local consultants from USEP-Davao who developed the tool and US-based Novametrics which studied the evaluation questions. It found the survey questions diverse in covering the high-level indicators of MRP (questions are not skewed towards some high￾level indicators over others and are well structured. The baseline survey format, the survey design, and methodology will be used by CLAimDev in the endline survey. CLAimDev STTA will develop a common analytical framework and method following how the baseline survey was designed, conducted, and analyzed. Survey Respondents: Considering the baseline survey respondents for the endline survey is highly encouraged. This is possible since more than half (60%) of the IDPs remain in the host communities. Challenges are expected in locating the substantial proportion of respondent IDPs who transferred to other areas. AOR to ask Plan International to provide a copy of the baseline research design to CLAimDev. STTA to follow the grand design of the baseline for the endline in terms of tools, sampling design, measurements, etc. STTA needs to have access to the baseline survey datasets to be used for comparative analysis in the endline survey. A case study may be done to get the story or overriding reasons why the IDPs moved out of Mindanao. (MRP External Evaluation Annexes) Page 63 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED USAID reminded not to overexert on locating the baseline survey respondents considering the challenges. Locating the baseline respondents is possible since Plan International has a database and knows the movements of the IDPs. Plan International also has excellent access to the local governments and the communities with the support of two local organizations, particularly Ecoweb and Maradeca. Baseline Survey (continuation) Survey Control Group: CLAimDev STTA will consider the host communities and displaced citizens in areas where MRP did not operate (control group) in the endline survey. MRP has less ability to locate the respondents from the control group since they are not included in its beneficiaries’ databases. Challenges are also seen in locating the respondents due to changes in residence and mobile numbers. The use substitution will be considered by selecting respondents with similar characteristics in terms of gender, identity groups, and others. A panel data may not be possible; it would be presented as a possible scenario. Survey Questionnaire: o Baseline results manifest that in terms of three high-level indicators, there was there no nuances in some aspects, but not all. We found some nuances Depending on the baseline data behavior, the CLAimDev STTA will explore the most appropriate methodologies such as the “panel-like sampling and the use of substitution for determining the respondents. Seek assistance from Plan International in tracing the respondents to expedite the data collection. Preliminary questions that will lay the context for the moving out should be included in the endline survey questions. CLAimDev to be provided with access to the baseline data or data matrix with the number of respondents (both control and treatment) and a spreadsheet of responses. This is to check the possible nuances of the baseline (MRP External Evaluation Annexes) Page 64 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED between MRP and non-MRP sites, these were noted in the baseline report. Both (the treatment and control groups) showed a high level of cohesion, representation, and level of self-reliance. o The seeming possible ceiling effect may be explained by the way the high-level indicators were measured or scaled. o The “level of trust” is a construct considered by MRP to describe the extent of polarization towards other identity groups - IDPs and HCMs. MRP probed trust as an element of polarization between IDPs and host communities. o On the way to measure polarization, MRP had to find constructs that could represent the context, which led to the addition of ‘trust’ as one of the variables. o In the early stages of the violent conflict and emergency post-conflict, the massive displacement led to the doubling of population in some HCMs, or the displaced communities are even larger than the HCM population. The HCMs are facing significant development challenges - health, economy, etc. o The complex conflict dynamics in Lanao, are triggered by resource/land disputes, clan disputes, GBV, shadow economy activities, and political competition. o The displaced individuals/communities will be staying in the HCMs for a long time, especially since the reconstruction of Marawi is not yet done, and IDPs won’t be able to go back to their homes. o USAID and Plan Intl anticipated that IDPs will stay with host communities over an extended period given the delays in Marawi MAA reconstruction/rehabilitation. o MRP focused on minimizing polarization between IDPs and host citizens that could add to the layer of violent conflict triggers in Lanao survey results from the control group. CLAimDev STTA will follow the scaling method in the survey questionnaire and see how the endline results may show little significant difference. CLAimDev to gather more information from Plan International (through KII) that will help in further understanding the key constructs in the survey tool. Consider adding information on the emerging trends that affect the outcomes of MRP. CLAimDev to request the translated version of the tools from Plan International. Plan International to provide CLAimDev access to the Kobo Toolbox baseline survey questionnaire. CLAimDev to configure how the existing electronic questionnaire will be used for the endline survey tool. MSME CLAimDev STTA plans to conduct a separate data collection for MSMEs through FGDs and KIIs. The data gathering will also look into the possible assistance they received from other agencies. Plan International to assist in locating the beneficiaries (individuals, groups, etc.) (MRP External Evaluation Annexes) Page 65 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED who have received assistance from MRP. Direct NGO partners of MRP regardless of funding Ecoweb and Maradeca are the two biggest partners of Plan that received the biggest chunks of the sub-award budget. Both are also part of the business plan model of Plan International. They have a key role in the implementation of the micro-grant assistance component of MRP. There are two types of partners: 1) sub-awardees and 2) organizations that have no sub-awards with Plan International but support MRP activities and objective Plan International will provide the list of partners (main, sub-awards, and sub-grantees) Set up a separate meeting with Ecoweb and Maradeca to know their roles in the micro-grant process. Other KII and FGD Respondents CLAimDev list of the types of respondents using the various data collection methodologies was presented, The list of other POs not directly assisted by MRP (e.g., women/youth/farmers / religious, etc.) refers to the control group. For partner private sector organizations, it is important to note that MRP works with many private sector partners, some even contribute cost-share. There are still other respondents who will still be identified based on additional information needed for this evaluation. USAID suggests providing more details/guidance on this group of respondents. CLAimDev to finalize the list with more detailed information on the tools to be used and the information that will be gathered from these respondents. (MRP External Evaluation Annexes) Page 66 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Gender Questions Gender question on evaluating MRP’s effectiveness implies that the project is expected to demonstrate “reduced gaps between females and males and other social groups”. Gender parity is not the primary goal/intent of the MRP. Gender is considered a guiding principle, even if it is not articulated in the results framework. The project has an intention of empowering, but not necessarily gap reduction. All interventions of MRP have predominantly female participants. The evaluation covers the changes in the economic and social cohesion conditions of target communities affected by the siege. The evaluation begins by covering the ‘what’ instead of the ‘how.’ The how is already pre-defined in the MRP workplan based on budget and interventions. Need for clear guidance on how deeply gender can be analyzed from an empirical standpoint. Disaggregation can be done in the analysis but probing further should consider that gender parity is not the main intent of MRP. The gender questions of the evaluation may be answered by just simply describing whether the actual outcomes manifest gaps between gender and other social groups changed even if this was not the intention of the project. On Handling Sensitive Information Official reports show negative findings on Bangon Marawi. Sensitive information may come out from the data collected (e.g., graft and corruption, violent extremism, etc.). Sensitive information can be redacted and should be carefully mentioned in the evaluation findings. USAID reminds the evaluation team that the unit of analysis of the evaluation is the IDPs and HCMs and by looking closely at how USAID provided development assistance and the results to its target groups. Sensitive information will be considered as just a piece of information about the developments in the local context and does not have to be elaborated on in the evaluation. Information from other reports, that will be mentioned in the evaluation, will need probing and triangulation. (MRP External Evaluation Annexes) Page 67 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Inclusion of other OECD criteria (efficiency and cohesion) The question of efficiency relates to the cost of doing business in implementing MRP in a highly complex environment. The following are some of the project management processes/approaches that were discussed during the three meetings that relate to project efficiency: - MRP has a system/scorecard using specific criteria for the selection of the project target sites (e.g., presence of USAID projects, the concentration of IDPs, etc.). - MRP solicitation/awarding process is more purposive than competitive due to access issues of the IDPs. (Cost-of-doing business) - MRP conducts counterterrorism vetting processes in selecting its project beneficiaries. - MRP does context monitoring and reflects this in the workplan and annual reports. Every year, MRP also has short scenario plans within different themes as reflected in the project work plans. - As part of adaptation interventions. Plan piloted some economic recovery engaging with the civil society to minimize the impact of the pandemic on the recovery of the communities. - There are emerging challenges that happened during the implementation of MRP. This includes – Covid, BARMM transition, Elections, etc. These were intervening situations that have affected project interventions. - The COVID-19 response received 2.2M USD funding, but it is part of the 25M USD. It is to ensure that the results framework remains intact. - As part of MRP adaptation, Gender-Based Violence became an important intervention of MRP that was intensified further in 2021 during the pandemic. ▭ While MRP is not a violent extremism intervention project, MRP supports communities and partners on GBV. It is because of the links between GBV and violent extremism. The evaluation should focus on the three questions: relevance, effectiveness, and sustainability. Efficiency will not be included as one of the evaluation criteria. Efficiency may be covered indirectly only when there is a significant correlation between the 3 evaluation questions and the project’s efficiency. CLAimDev STTA to ask for MRP documents: • Copy of the grant agreement, particularly Section C: Technical Description. • Annual work plans • Scenario Plans (MRP External Evaluation Annexes) Page 68 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED ▭ COVID-19 created additional; vulnerabilities because of the restrictions that limited the ability to report and address GBV. ▭ A UN Women study also highlights the importance of GBV. Case studies in conflict areas in the Philippines show that communities tolerant of hostile misogyny and GBV against women and children are three times more likely to accept violent extremism. (MRP External Evaluation Annexes) Page 69 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Covid Emergency Response The COVID-19 response covered four components: 1) capacity training for frontline service delivery for health workers and LGUs, 2) Commodity grants for PPEs and WASH facilities, 3) Risk communication support for COVID-19 response; and 4) Economic impact through the provision of livelihood assistance to affected communities. For example, commodity grants, training, and market linkages. CLAimDev STTA to consider this as part of the MRP interventions. Sustainability There are distinct interventions in rural and urban settings. Tie together how inclusion relates to sustainability, which can be asked as an additional evaluation question. Even though some interventions such as MRP are not inclusive, the interventions are sustainable to some extent. The idea of sustainability with inclusivity in MRP is not transitive. CLAimDev to discuss further how to frame inclusivity from the evaluation standpoint. Poverty statistics in Lanao provinces after the Marawi Siege The latest 2021 PSA household poverty incidence shows that Lanao provinces no longer belong to the top 20 poorest municipalities in the Philippines. Other poverty metrics also a significant improvement in the provinces and the entire BARMM. If MRP could have contributed to the condition due to the significant investments to help poor communities in Lanao after the siege, the economic recovery support intervention can be linked to this progress. Meso-level analysis may be considered to reveal the connection of MRP to the poverty data. However, the attribution of PSA poverty data and MRP may be done mistakenly considering other factors such as, among others, the increasing investments from various agencies in the province after the Marawi siege. The evaluation team should exercise due care regarding poverty data since the evaluation methodology does not allow such attribution. The discussion of poverty in the evaluation can be weaved as part of describing the local context but not necessarily to describe the attribution. (MRP External Evaluation Annexes) Page 70 | 286 HIGHLIGHTS OF THE MEETINGS TOPICS POINTS RAISED AGREEMENTS/ACTIONS NEEDED Human trafficking cases This is beyond the scope of the evaluation since MRP is not addressing this issue. The evaluation will not probe whether trafficking is a consequence of conflict. But in probing and yielding information on human trafficking or any form of gender violence, it will be reported to the proper authorities (local and USAID) and be referred to services, protection, and assistance. In cases when the project has contributed to human trafficking cases, this should be properly reported to the Office of the Inspector General as required in the cooperative agreement between the implementing partner and USAID. Although human trafficking is not included in the design, it may be related to MRP’s gender-based violence intervention, which mainly falls under domestic violence as a form of GBV. This will not be intentionally considered in the evaluation. Document this information, if present, for reporting and referral to proper authorities. Other agencies like UNDP, KOICA, JICA, EU, etc. MRP works closely with Task Force Bangon Marawi. TFBM can provide information about complementation among national government and funding channels. There is also a separate path for funding from the LGUs. The donor investment changed before the conflict and after the siege. Donor assistance is fluid. The investment scenario or the influx of funding assistance started to decrease in 2019. Schedule an interview with Plan International as one of the respondents. Group the funding path to see the complementation with MRP. CLAimDev evaluation team should understand this situation as part of describing the project environment. KOICA may be interviewed and invited to attend some of the evaluation activities. (This is mentioned in the Evaluation SOW.) Emerging Trends and Challenges There are other emerging trends and challenges that affect the outcomes of MRP. Document emerging trends and challenges that affect the implementation and outcomes of MRP. (MRP External Evaluation Annexes) Page 71 | 286 Inception Report Attachment 2: MRP EVALUATION DESIGN MATRIX Evaluation Questions Data Sources Data Collection Methods Data Analysis Methods RELEVANCE. Were MRP’s development interventions relevant to the needs of the IDPs and Host Communities and adaptive to the complex and changing external environment in Mindanao, especially in the project area where violent conflicts exist? MRP documents, Project Reports, Social groups (Women, Youth, Farmers, Traders), NGOs (Maradeca, Ecoweb), GOs (LGU, DTI, DOLE) Document Review Qualitative: Content Analysis ● In what ways have MRP’s objectives and design [Theory of Change (ToC), Results Framework (RF)] responded to the needs of the IDPs and host communities and in finding longer-term durable solutions to their displacement? Project Reports, IDPs and HCMs, Social groups (Women, Youth, Farmers, Traders) Document Review, FGD Qualitative: Content Analysis ● In what ways have MRP’s development interventions been relevant and adaptive to the distinct needs of the vulnerable groups according to female/male and other social groups [e.g. Internally Displaced Persons (IDPs)/Host Community Members (HCMs), urban/rural, etc.]? IDPs and HCMs, Social groups (Women, Youth, Farmers, Traders) FGD Qualitative: Content Analysis (MRP External Evaluation Annexes) Page 72 | 286 Evaluation Questions Data Sources Data Collection Methods Data Analysis Methods ● In what ways have interventions been sensitive and adaptive to the complex and changing economic, environmental, social, political economy, and capacity conditions in the environment in which the project is operating? Project Reports, NGOs (Maradeca, Ecoweb), Social groups (Women, Youth, Farmers, Traders), GOs (LGUs, DTI, DOLE, Dept. of Finance*, Bangon Marawi*) Document Review, FGD, KII Qualitative: Content Analysis ● How aligned and coherent is MRP’s intervention with existing government priorities in responding to the Marawi Crisis as well as with the USAID’s policy guidance in assisting Internally Displaced Persons, especially in transitioning them to safer communities and restoring livelihood and income opportunities? Project Reports, GOs (LGU, DTI, DOLE, Dept. of Finance*, Bangon Marawi*) Document Review, KII Qualitative: Content Analysis EFFECTIVENESS. Did MRP accomplish the project’s expected outcomes of improving self-reliance and strengthening social cohesion among IDPs and HCMs? IDPs and HCMs Endline Survey MSME Survey Quantitative: Descriptive and inferential analysis ● In what ways have MRP achieved its intended outputs and outcomes in improving the economic self-reliance and social cohesion of IDPs and host communities? Project Reports, Social Groups (Women, Youth, Farmers, Traders) Document Review, FGD KII Qualitative: Content Analysis (MRP External Evaluation Annexes) Page 73 | 286 Evaluation Questions Data Sources Data Collection Methods Data Analysis Methods ● In what ways do MRP’s actual outcomes demonstrate reduced gaps between gender groups and other social groups as identified in the baseline study? Did all the target IDPs and HCMs including the most disadvantaged and vulnerable –women, children, and youth, benefit equally from the interventions? Project Reports, Social Groups (Women, Youth, Farmers, Traders) Document Review, FGD Qualitative: Content Analysis ● Has MRP caused a significant change in the lives of the intended beneficiaries and their communities such as improving economic self￾reliance and strengthening social cohesion? Has MRP’s intervention transformed and created more enduring positive changes in the IDP’s perceptions and community norms/processes (whether intended or not) in responding to their situation? IDPs and HCMs, Social groups (Women, Youth, Farmers, Traders) Endline Survey MSME Survey FGD, KII Quantitative: Descriptive and inferential analysis Qualitative: Content Analysis ● In what ways has MRP caused higher-level effects particularly in finding long￾term and durable solutions in integrating the IDPs and host communities? Social groups (Women, Youth, Farmers, Traders) FGD Qualitative: Content Analysis (MRP External Evaluation Annexes) Page 74 | 286 Evaluation Questions Data Sources Data Collection Methods Data Analysis Methods SUSTAINABILITY. Did MRP establish mechanisms to support the sustainability of the improved economic conditions and strengthen social cohesion between IDPs and HCMs? Project Reports, NGOs (Maradeca, Ecoweb), GOs (LGUs, DTI, DOLE), Social groups (Women, Youth, Farmers, Traders) Document Review, FGD, KII Qualitative: Content Analysis ● In what ways did MRP establish sustainability mechanisms that promote equitable economic improvement and equal social participation among females and males and various social groups? Project Reports, Social groups (Women, Youth, Farmers, Traders) Document Review, FGD Qualitative: Content Analysis ● In what ways has MRP established approaches to continue, scale-up, and replicate project outcomes on the IDPs/HCMs? Project Reports, NGOs (Maradeca, Ecoweb), GOs (LGUs, DTI, DOLE), Social groups (Women, Youth, Farmers, Traders) Document Review, FGD, KII Qualitative: Content Analysis ● What mechanisms have evolved out of MRP interventions in creating the enabling environment to sustain and further improve economic conditions for beneficiaries and social cohesion between IDPs and HCMs? Project Reports, NGOs (Maradeca, Ecoweb), GOs (LGUs, DTI, DOLE), Social groups (Women, Youth, Farmers, Traders) Document Review, FGD, KII Qualitative: Content Analysis ● To what extent do the various opportunities and risk factors enhance or threaten the sustainability of project benefits over time? Project Reports, NGOs (Maradeca, Ecoweb), GOs (LGUs, DTI, DOLE), Social groups (Women, Youth, Farmers, Traders) Document Review, FGD, KII Qualitative: Content Analysis (MRP External Evaluation Annexes) Page 75 | 286 Inception Report Attachment 3: SUMMARY OF MRP EVALUATION METHODOLOGY Indicator Data Source Frequency Unit of Measure Unit of analysis Method/Tool Analytical Approach High-Level Impact – Durable solution: local integration of IDPs Manageable Impact – Self-reliance of IDPs and host community members who were affected by the Marawi siege. 1 Level of self-perceived self-reliance of assisted displaced and host communities. Survey Baseline and end line Percent of IDPs and host community members per categories/levels of self-reliance Individual-level (IDPs and HCMs) Endline Survey Questionnaire Description of survey ratings ● Mean, SD ● Graphs; Joint displays ● Network analysis Comparison of baseline￾endline data ● Panel data analysis ● Tests of difference ● Effect size analysis Intermediate Result 1: Improved economic conditions of IDPs and host communities. (MRP External Evaluation Annexes) Page 76 | 286 Indicator Data Source Frequency Unit of Measure Unit of analysis Method/Tool Analytical Approach 2 Number of displaced business owners with new or re-started businesses Project records Annual Cumulative number of business owners Individual-level (Business owners) Document review; FGD; KII MSME Survey Questionnaire ● Content/thematic analysis ● Graphs; Joint displays 3 Percent of trained displaced/host community members gainfully employed Project records Annual Cumulative number of IDPs and host community members Individual-level (Business owners) Document review; FGD; KII ● Content/thematic analysis ● Graphs; Joint displays Output 1.1 Business recovery opportunities expanded 4 Number of displaced business owners who benefitted from business recovery micro-grants Project records Quarterly Number of businesses owners Individual-level (Business owners) Document review; FGD; KII MSME Survey Questionnaire ● Content/thematic analysis ● Graphs; Joint displays (MRP External Evaluation Annexes) Page 77 | 286 Indicator Data Source Frequency Unit of Measure Unit of analysis Method/Tool Analytical Approach 5 Number of displaced business owners trained in enterprise management Project records Quarterly Number of displaced business owners Individual-level (Business owners) Document review; FGD; KII MSME Survey Questionnaire ● Content/thematic analysis ● Graphs; Joint displays Output 1.2 Employment opportunities expanded 6 Number of displaced/host community members trained in workforce readiness and life skills Project records Quarterly Number of displaced/host community members Individual-level (IDPs and HCMs) Document review; FGD; KII ● Content/thematic analysis ● Graphs; Joint displays Output 1.3 Livelihood opportunities expanded 7 Number of displaced/host community members who benefitted from livelihood micro￾grants Project records Quarterly Number of displaced/host community Members Individual-level (IDPs and HCMs) Document review; FGD; KII ● Content/thematic analysis ● Graphs; Joint displays (MRP External Evaluation Annexes) Page 78 | 286 Indicator Data Source Frequency Unit of Measure Unit of analysis Method/Tool Analytical Approach Intermediate Result 2: Strengthened community cohesion in communities hosting large numbers of IDPs. 8 Level of polarization towards the other identity group Survey Baseline and end line* Level of polarization Individual-level (IDPs and HCMs) Endline Survey Questionnaire Description of survey ratings ● Mean, SD ● Graphs; Joint displays ● Network analysis Comparison of baseline￾endline data ● Panel data analysis* ● Tests of difference ● Effect size analysis 9 Level of self￾perception of public representation Survey Baseline and end line* Level of self￾perception of public representation Individual-level (IDPs and HCMs) Endline Survey Questionnaire Description of survey ratings ● Mean, SD ● Graphs; Joint displays ● Network analysis Comparison of baseline￾endline data ● Panel data analysis* ● Tests of difference; Effect size analysis (MRP External Evaluation Annexes) Page 79 | 286 Indicator Data Source Frequency Unit of Measure Unit of analysis Method/Tool Analytical Approach Output 2.1: Social cohesion grants to host and displaced communities 10 Number of displaced/host community members who benefitted from social cohesion grants Project records Quarterly Number of displaced/host community members Individual-level (IDPs and HCMs) Document review; FGD; KII ● Content/thematic analysis ● Graphs; Joint displays 11 Number of displaced/host community members trained on civic engagement Project records Quarterly Number of displaced/host community members Individual-level (IDPs and HCMs) Document review; FGD; KII ● Content/thematic analysis (MRP External Evaluation Annexes) Page 80 | 286 Inception Report Attachment 4 : MRP ENDLINE SURVEY QUESTIONNAIRE MARAWI RESPONSE PROJECT ENDLINE EVALUATION SURVEY QUESTIONNAIRE # VARIABLE CODE RESPONSE 1. Location 1.1 Region 0 Region 10 1 BARMM 1.2 Province 0 Lanao Del Norte 1 Lanao Del Sur 1.3 Municipality 1 Iligan City 2 Baloi 3 Saguiaran 4 Poona Bayabao 5 Masiu 1.4 Barangay 1 Sto. Rosario 2 Del Carmen 3 Tubod (MRP External Evaluation Annexes) Page 81 | 286 4 Buruun 5 East Poblacion 6 Pagalamatan 7 Mipaga 8 Lumbaca Toros 9 Cadayonan 10 Matao Raza # VARIABLE CODE RESPONSE 2. Respondent’s Profile 2.1 Name of CSG 2.2 Sex 0 1 2.3 Age 2.4 Marital Status 1 Married 2 Separated 3 Widow/Widower 4 Single/Unmarried (MRP External Evaluation Annexes) Page 82 | 286 # VARIABLE CODE RESPONSE 88 Others. Please specify 2.5 Religion 0 None 1 Christian (including Catholic, Protestant, Evangelical Christian, Seventh Day Adventist, and other Christian Denominations 2 Islam 3 Jehovah’s Witnesses 4 Iglesia ni Kristo 5 Buddhist 99 Others, please specify: 2.6 Highest educational attainment 0 Did not attend school 1 Elementary level 2 Elementary graduate 3 High school level 4 High school graduate 5 College level 6 College graduate 7 Vocational graduate (MRP External Evaluation Annexes) Page 83 | 286 # VARIABLE CODE RESPONSE 8 Master’s degree/Doctor’s degree level 9 Master’s degree/Doctor’s degree graduate 99 Others, please specify: 2.7 Are you currently employed? 1 Yes 0 No (Proceed to 2.8) 2.7.1 Type of employment 1 Permanent 2 Probationary/Temporary 3 Part-time 4 Self-employed 2.7.2 What industry are you employed in? (You may answer more than one.) 1 Farming crops 2 Farming Livestock/Fishing 3 Sari-sari Store/Food Retailing 4 Non-food Retailing (e.g., RTW, textile) 5 Private Company 6 Government Unit/Agency 7 Non-Profit organization (MRP External Evaluation Annexes) Page 84 | 286 # VARIABLE CODE RESPONSE 99 Others, please specify: 2.7.3 How long have you been employed or self￾employed? Specify: __________ years __________ months 2.8 If no, why are you not currently employed? 1 There are no jobs in my area. 2 I am not qualified for the jobs in my area. 3 I have been applying for jobs but have not been offered any job 4 I am unable to work due to a health issue or disability. 99 Others, please specify: 2.9 At present, are you a member of any group/association/organizati on? 1 Yes 0 No 66 Refused to answer 2.9.1 If yes, what groups/associations/organiz ations are you a member of? 1 Community Solidarity Group 2 Farmers’ Association 3 Women’s Association (MRP External Evaluation Annexes) Page 85 | 286 # VARIABLE CODE RESPONSE 4 Trade Union 5 Mosque/Church/Religious Group 6 Youth Group 99 Others, please specify: 3. Household Profile 3.1 Who leads in the family decision-making? 1 Father 2 Mother 3 Both father and mother 4 Uncle 5 Aunt 6 Grandparents 99 Others _________________ (please specify) 3.2 How many members of this household are 0-29 years old? Specify the number: _____________________ 3.3 How many in this household are 30 and above? Specify the number: _____________________ (MRP External Evaluation Annexes) Page 86 | 286 # VARIABLE CODE RESPONSE 3.4 How long have you been living in this current residence? Specify the number: _____________________ 3.5 What is the ownership status of the house you are currently living in? 1 We own the house. 2 We rent the house/room (Proceed to 4.1). 3 We are the caretaker of the house. 4 We occupy the house for free. 5 We co-own the house. 6 We are living with the host family. 7 The house is provided free (NGO, government) 99 Others, please specify: 4. Household Profile and Debt 4.1 If renting, how much is your monthly rent? Specify the amount PhP____________________ ______ (Put N/A or Not Applicable if not renting) 4.2 During the past month, what are the top 3 expenditures your HH has spent most on? Food Non-food items (includes clothing, hygiene kits, etc.) (MRP External Evaluation Annexes) Page 87 | 286 # VARIABLE CODE RESPONSE (Please write the rank (1, 2, 3) on the space provided). If food is top 1, then write 1 on the space. Medicine and/or hospital bill Loan payment Child/Children’s education Monthly bills (e.g., electricity, water) Shelter repair/building of a house House rent 99 Others, please specify: 4.3 How much is your household monthly expenditure for ___ (say the answer Ranked 1 in 4.2)? Specify amount PhP __________________ 4.4 How much is your household’s monthly expenditure for ___ (say the answer Ranked 2 in 4.2 Specify amount PhP __________________ 4.5 How much is your household’s monthly expenditure for ___ (say the answer Ranked 3 in 4.2 Specify amount PhP __________________ 4.6 Do you or anyone in your household currently have any debt/loan? 1 Yes 0 No 55 Don’t know 66 Refused to answer (MRP External Evaluation Annexes) Page 88 | 286 # VARIABLE CODE RESPONSE 4.7 If Yes, what was the source of the debt/loan? (You may answer more than one) 1 Friend/Neighbor 2 Relative 3 Moneylender 4 Bank/formal lending institution 5 Traders 55 Don’t know 66 Refused to answer 99 Others, please specify: 4.8 If yes, what was the main purpose for taking out the loan? (You may answer more than one.) 1 Buy food 2 Buy medicine and/or pay the hospital bill 3 Pay school expenses 4 Invest in business 5 Pay previous debt/loan 6 Pay monthly bills (water, electric, etc.) 7 Pay house rent 55 Don’t know (MRP External Evaluation Annexes) Page 89 | 286 # VARIABLE CODE RESPONSE 66 Refused to answer 99 Others, please specify: 5. Household Income 5.1 What are your household’s sources or sources of income at present? (You may answer more than one.) 1 Farming crops 2 Farming livestock/fishing 3 Sari-sari store/food retailing 4 Non-food retailing (e.g., RTW, textile) 5 Paid work (including wages and pay for services/labor) 6 Donor assistance (e.g., from an aid agency, a religious 7 Institution) 8 4Ps 9 Remittance 10 Others, please specify: 99 5.2 Do you have savings? 1 Yes 0 No (MRP External Evaluation Annexes) Page 90 | 286 # VARIABLE CODE RESPONSE If “No,” proceed to _________ 5.2.1 How do you save money? 1 Alkansiya/Piggy Bank 2 Bank 3 Cooperative 4 Community Savings Group (like Paluwagan) 99 Others, please specify: 5.3 Did anyone in your household receive skills training in the past 6 months? 1 Yes 0 No 55 Don’t know 66 Refused to answer 5.4 Did anyone in your household receive business development training (including financial literacy) in the past 6 months? 1 Yes 0 No 55 Don’t know 66 Refused to answer 6. Perceived Self-reliance 6.1 1 Strongly disagree (MRP External Evaluation Annexes) Page 91 | 286 # VARIABLE CODE RESPONSE I have the right skills to find a job. 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.2 There are sufficient job opportunities in my community. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.3 If I were to lose my job or livelihood tomorrow, I feel confident that I would find another job within a month. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer (MRP External Evaluation Annexes) Page 92 | 286 # VARIABLE CODE RESPONSE 6.4 If I were to lose my job or livelihood tomorrow, I feel confident that I would start a new livelihood within a month. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.5 The income I earn from my job/livelihood is sufficient to cover the basic needs of my family 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.6 In times when our income is not enough for the basic needs of my family, I have enough assets that I can sell to cover the basic needs of my family. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know (MRP External Evaluation Annexes) Page 93 | 286 # VARIABLE CODE RESPONSE 66 Refused to answer 6.7 In times when our income is not enough for the basic needs of my family, I have enough savings to cover the basic needs of my family. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.8 In times when our income is not enough for the basic needs of my family, I can easily borrow money from relatives and/or friends to cover the basic needs of my family. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 6.9 In times when our income is not enough for the basic needs of my family, the remittances we receive are enough to cover the basic needs of my family. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree (MRP External Evaluation Annexes) Page 94 | 286 # VARIABLE CODE RESPONSE 55 Don’t know 66 Refused to answer 6.10 I can easily access skills training opportunities in may barangay. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7. Perceived Level of Polarization 7.1 In the barangay where you are currently living in, you are a: ____ 1 Internally Displaced Person (IDP)? 2 Host Community Member (HCM)? 7.2 Most people in the neighborhood are willing to help if you ask 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know (MRP External Evaluation Annexes) Page 95 | 286 # VARIABLE CODE RESPONSE 66 Refused to answer 7.3 Most people in the neighborhood can be trusted. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.4 Most people in the neighborhood are honest. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.5 It is naive to trust people. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree (MRP External Evaluation Annexes) Page 96 | 286 # VARIABLE CODE RESPONSE 55 Don’t know 66 Refused to answer 7.6 I feel that I belong to my immediate community/neighborhood. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.7 Differences often exist between people living in the same neighborhood. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.8 Different ethnic tribes tend to divide the people in our neighborhood. 1 Strongly disagree 2 Disagree 3 Agree (MRP External Evaluation Annexes) Page 97 | 286 # VARIABLE CODE RESPONSE 4 Strongly agree 55 Don’t know 66 Refused to answer 7.9 Religious differences tend to divide people in my neighborhood. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer I would like to ask you about the trust you have in different groups of people. Please tell me if you agree or disagree with the following statements: 7.10 I trust the CSG officials. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.11 1 Strongly disagree (MRP External Evaluation Annexes) Page 98 | 286 # VARIABLE CODE RESPONSE I trust my fellow CSG members. 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.12 I trust the local elected officials in the barangay I am currently living in. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.13 I trust the traditional/religious leaders in the barangay I am currently living in. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer (MRP External Evaluation Annexes) Page 99 | 286 # VARIABLE CODE RESPONSE 7.14 I trust the non￾governmental organizations. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.15 I trust the police. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 7.16 I trust the military. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know (MRP External Evaluation Annexes) Page 100 | 286 # VARIABLE CODE RESPONSE 66 Refused to answer. 7.17 I am willing to work with the IDP/HCMs to implement projects. 1 Strongly disagree 2 Disagree 3 Agree 4 Strongly agree 55 Don’t know 66 Refused to answer 8. Perceived Public Representation 8.1 I am consulted and involved in the implementation of projects in my community (e.g., for the water system, and security) 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.2 Ordinary people or ordinary members participate when important decisions are 1 Never 2 Sometimes (MRP External Evaluation Annexes) Page 101 | 286 # VARIABLE CODE RESPONSE made in the community or CSG. 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.3 Youth participate when important decisions are made in the community or CSG. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.4 Women participate when important decisions are made in the community or CSG. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.5 1 Never (MRP External Evaluation Annexes) Page 102 | 286 # VARIABLE CODE RESPONSE People from my own ethnic group participate when important decisions are made in the community or CSG. 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.6 People who share my religious views participate when important decisions are made in the community or CSG. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.7 I attended a community or CSG meeting. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer (MRP External Evaluation Annexes) Page 103 | 286 # VARIABLE CODE RESPONSE 8.8 I talked personally to the CSG leader/BLGU Official about a problem concerning our community. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.9 I talked called the CSG leader/BLGU Official about a problem concerning our community. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.10 I wrote a letter to the CSG leader/BLGU Official about a problem concerning our community. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know (MRP External Evaluation Annexes) Page 104 | 286 # VARIABLE CODE RESPONSE 66 Refused to answer 8.11 I voted during the election of CSG officers. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.12 I voted during the barangay election. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.13 I participated in the implementation of community or CSG projects or activities. 1 Never 2 Sometimes 3 Often 4 Very Often (MRP External Evaluation Annexes) Page 105 | 286 # VARIABLE CODE RESPONSE 55 Don’t know 66 Refused to answer 8.14 I am involved in approaching government and non-governmental organizations to make use of resources for projects to improve our community (e.g., for a water system, and security). 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.15 I am an active member of civic/community/religious organizations (s) like CSG, Farmers’ Association, Women’s Association, Trade Union, Mosque/Church/religious group/youth group, and others. 1 Never 2 Sometimes 3 Often 4 Very Often 55 Don’t know 66 Refused to answer 8.16 I participate in training to develop ways to solve community problems, find work or a better job, run a business, or write an 1 Never 2 Sometimes 3 Often (MRP External Evaluation Annexes) Page 106 | 286 # VARIABLE CODE RESPONSE application for a loan or business grant. 4 Very Often 55 Don’t know 66 Refused to answer (MRP External Evaluation Annexes) Page 107 | 286 Inception Report Attachment 5: DESIGN IN PRETESTING THE ENDLINE AND MSME SURVEY QUESTIONNAIRES Rationale Pretesting of questionnaires is an essential process in research, surveys, and other data gathering endeavors. It allows researchers and evaluators to spot potential problems related to the tools. These could include among other problems related to the clarity of direction, items, and response options. The process can also help ensure that prospective respondents will have a homogenous understanding of the terms, phrases, or statements in the tool. If performed properly, it will result in tools with robust validity and reliability properties. Ultimately, the process helps achieve better data quality. In this activity, both the endline and MSME questionnaires will be pretested. Both questionnaires will be subjected to a respondent-driven pretest process. Objectives The pretesting activity is designed to achieve the following two-fold objectives: 1. Administer the initial versions of the endline and MSME questionnaires to a small group of respondents to gather data on: a. Clarity of directions, items, and response options b. Response latency c. Behavioral responses to items 2. Utilize pretest data to enhance the endline and MSME survey questionnaires. Methodology (MRP External Evaluation Annexes) Page 108 | 286 Sampling Twelve purposively selected respondents will be involved in the pretest. Six will be for the endline and the other six for the MSME survey. For both endline and MSME respondents, there has to be proportional representation in terms of respondent type (IDPs and HCMs) and gender (Male and Female). Procedures 1. The STTA Team will identify a pretest proctor and a documenter. The main role of the proctor is to administer the survey to the respondents. The role of the documenter is to record the administration process using the Pretest Observation Form. It is suggested that the two field coordinators will take the role of being proctor and a documenter. 2. The STTA Team will orient the proctor and documenter of the pretest process and of their roles. 3. Purposively identify 12 pretest respondents. Six respondents will be involved in the endline survey pretest. The other six respondents will be in the MSME survey pretest. 4. The proctor and the documenter will locate each respondent. 5. The proctor and documenter will execute the pretest process following the steps below: a. The proctor will explain the objective of the activity. b. The proctor will read the direction of the questionnaire. After the reading, the proctor will ask about the respondent about the clarity of the direction. The documenter will record observations using the Pretest Observation Form. c. The proctor will read the survey items and let the respondent give his/her response. The documenter will record the response latency and any behavioral response for each item. d. After completing an item, the proctor will ask the respondent about the clarity of the item and the response options. The documenter will record the observations. This will be done for each item in the questionnaire. e. The proctor will thank each respondent after completing the pretest. 6. The STTA Team will consolidate the data and identify areas for enhancement. 7. The STTA Team will improve the questionnaires based on the pretest data. Data Gathering Tools 1. Pretest Probe Questions. The following questions will be asked by the proctor to each respondent: Questions Response Remarks 1. How will you describe the clarity of the direction? (MRP External Evaluation Annexes) Page 109 | 286 2. What word, phrase, or sentence do you think needs to be clarified in the direction? 3. How will you describe the clarity of the item? 4. What word, phrase, or sentence do you think needs to be clarified in the item you just responded? 5. How will you describe the clarity of the options given to you? 6. What part of the options is not clear to you? Note to the proctor: Repeat asking items 3, 4, 5, and 6 for all the items in the survey. 2. Pretest Observation Form. The following form will be utilized by the documenter to record the observations from respondents: Respondent Profile Type [ ] IDP [ ] HCM Date: Gender [ ] Male [ ] Female Place: Observations Questionnaire Part Clarity Response latency (sec) Behavioral response observed Respondent suggestion; Other remarks Directions [ ] Clear [ ] Vague --- Item 1 – Item [ ] Clear [ ] Vague Item 1 – Options [ ] Clear [ ] Vague (MRP External Evaluation Annexes) Page 110 | 286 Item 2 – Item [ ] Clear [ ] Vague Item 2 – Options [ ] Clear [ ] Vague Item n Schedule Date/Time Activity In-charge Day 1 (May 19, 2022) Morning Team Arrival Afternoon Orientation of proctor and documenter STTA Team Pretest of sample respondents STTA Team Day 2 (May 20, 2022) Morning Pretest: Endline respondent 1; Area 1; M; IDP Proctor; Documenter Pretest: Endline respondent 2; Area 1; F; IDP Pretest: MSME respondent 1; Area 1; M; IDP Pretest: MSME respondent 2; Area 2; F; IDP Afternoon Pretest: Endline respondent 3; Area 2; M; HCM Pretest: Endline respondent 4; Area 2; F; HCM (MRP External Evaluation Annexes) Page 111 | 286 Date/Time Activity In-charge Pretest: MSME respondent 3; Area 2; M; HCM Pretest: MSME respondent 4; Area 2; F; HCM Day 2 (May 21, 2022) Morning Pretest: Endline respondent 5; Area 3; M; IDP Proctor; Documenter Pretest: Endline respondent 6; Area 3; F; HCM Pretest: MSME respondent 5; Area 3; M; IDP Pretest: MSME respondent 6; Area 3; F; HCM Afternoon Reflection; Planning; Homeward bound STTA Team; Proctor; Documenter Limitations One salient limitation of the pretest process is that it will only generate respondent-driven data. Expert-driven pretest data will not be collected. Another limitation is the number of pretest respondents. These limitations are in consideration of the constraints in time and resources. (MRP External Evaluation Annexes) Page 112 | 286 Inception Report Attachment 6: MICRO AND SMALL ENTREPRENEURS (MARAWI MSMEs) FGD Guide Questions (Note: The facilitator must ensure that all participants have signed the Consent Form before the start of the FGD session.) FGD Group Location Time Start Time End Date Venue Facilitator Documenter Introduction Good day! Thank you for giving your time to participate in our activity. My name is… Warm-up Questions: 1. As a start, may I know what is your type of business and its present status? (MRP External Evaluation Annexes) Page 113 | 286 (Responses will show the different types of businesses and their status) 2. What assistance did you receive for your business? (Responses will show the recipients of various grants) 3. In the assistance that you received, what stage was the most convenient for you? (Name the stages: selection, interview, validation, distribution) Why? (Responses from respondents will identify the stages and the ease or difficulty they encountered) 4. In the identification of recipients, who in the household attended the process – you, spouse, other family members? Why? (Responses will show the household characterization and identify their own gender perception) A. Business Recovery (Criteria: Effectiveness) 1. How has the MRP micro- grant responded to your business's need to restart? ✓ Timely and enough to re-start and recover? Too little, too late? ✓ Is In-Kind Grant better than Cash Transfer of the other way around? (Responses will be used to trace the assistance, its timeliness, and response to need) (MRP External Evaluation Annexes) Page 114 | 286 2. What other support assistance did you receive from MRP? ✓ Have you attended any training relating to the micro- grant you received? ✓ Have you been referred by MRP to other organizations giving similar assistance? 3. What other business assistance did you get from other organizations – GO, NGO, relatives, private banks, other financiers – to re-start and recover? ✓ Grants – IKG and/or Cash Transfer? Loans with low or no interest? ✓ Were they enough for your business to re-start and recover? 4. Have your business finally recovered? (“Recovered” means the business is back in at the state before the Marawi Incident) ✓ How many of you are already reaping profits? How many are break -even? ✓ What is the biggest factor for your business's fast recovery? What is the biggest factor for your business's slow recovery? (Responses will be used to validate the perceived outcome of assistance) 5. Looking back, what better assistance could have been provided to make your business recover faster? ✓ A Bigger amount of grant? Cash versus in-kind? ✓ More business training? On-site mentoring? ✓ Linking with financial institutions? (MRP External Evaluation Annexes) Page 115 | 286 B. Plan International and other Business Assistance (Criteria: Relevance) 6. If there was no Plan International micro- grant, would you still have re-started and recovered with your business? ✓ Which organization (other than Plan International) was able to assist you in your business recovery – government (which one?), private bank, others? ✓ Similarly, who was able to assist you in your business recovery – relatives, friends, financiers? (Responses will show their level of maturity through their ability to cope using finance and planning) 7. With Plan International and other interventions for the business/private sector already concluded, do you feel you can already continue with your business (going concerns) without any further intervention? (Criteria: Sustainability) ✓ Has everything in your business community already ‘normalized’? ✓ How many businesses have not reopened yet until now? What are the reasons they have not been re-started or reopened? 8. Are the assistance – grants and/or loans – given fair enough to all business owners? ✓ Were there any complaints of unfair distribution of financial assistance? ✓ How are they remedied? How can they be remedied? C. Business and Community (Criteria: Effectiveness) 9. How do your suppliers and customers responded to your having re-started your business? (MRP External Evaluation Annexes) Page 116 | 286 ✓ Were you able to get back to your suppliers? New ones? ✓ Do you have the same customers? New ones? 10. As you re-started your business, have you and other business owners created more jobs (employment) now than before the Marawi Incident? ✓ Has the business reopening also revitalized the population’s spending/buying capacity? ✓ Have the business activities benefitted the general local business owners and the local population? (Or are these benefits going outside of Marawi and similarly-affected populations?) D. Cross-cutting Themes 11. How are your business activities received by the immediately affected community? (Do-No-Harm) ✓ What is the biggest factor for your business’ positive reception from your buying community? What is the biggest factor for your business’ negative reception? How did you overcome these adverse reactions? ✓ What business situations and activities that created more tension/division? Who are the ‘dividers’ (‘spoilers’)? ✓ What are business situations and activities that promote more understanding/cohesion? Who are the ‘connectors’ (‘transformers’) ✓ What was done to improve the situation? What can be done further? 12. Have the business stimulated more employment and livelihood across the gender – women, and men – population? (Gender Sensitivity) ✓ Given the aftermath of the Marawi Incident’s urgency of economic needs, have there been any changes in gender preferences/biases in the employment? ✓ Since the aftermath of the siege, have there been any changes in the participation between males and females in community meetings and decision makings? (MRP External Evaluation Annexes) Page 117 | 286 D. Government regulations and Business Associations (Criteria: Sustainability) 13. What are the advantages and disadvantages for a business to be registered with the government regulatory bodies and accredited with business associations? ✓ What are the alternatives for not registering? ✓ Are there parallel (shadow) business organizations your business would rather be with than the mainstream or publicly acknowledged associations? 14. Do you normally seek assistance from mainstream financial institutions (banks, lending investors)? ✓ What are your hesitations? ✓ Where do you go instead? E. The Marawi Incident (Criteria: Sustainability) 15. Do you think/believe that the Marawi Incident can happen again? ✓ How can you as a business owner help prevent it? ✓ How can the business/private sector help prevent it? (MRP External Evaluation Annexes) Page 118 | 286 Inception Report Attachment 7: FARMERS, WOMEN AND YOUTH GROUPS FGD Guide Questions (Note: The facilitator must ensure that all participants have signed the Consent Form before the start of the FGD session.) FGD Group Location Time Start Time End Date Venue Facilitator Documenter Introduction Good day! Thank you for giving your time to participate in our activity. My name is… A. Marawi Siege 1. In what ways did the Marawi siege affect your life? 2. How are you coping with the effects? 3. What are your plans for the future? 4. What were your immediate needs after the siege? (MRP External Evaluation Annexes) Page 119 | 286 5. What are your current needs? B. Knowledge about USAID/Plan International Project 1. What do you know about Plan International's (or USAID) involvement in the Marawi Incident? 2. Is their intervention in Marawi and other similarly affected areas timely and responsive to your needs? 1. What difference did they make in with the lives of IDPs/HCMs? 2. Would Plan International have done any better? 3. What other interventions could they have introduced? 4. Was the assistance delivered properly? 5. To your knowledge, were the assistance delivered and equal opportunities provided among men and women, youth, and elderlies? IDPs and HCMs? C. Assistance Received/Not Received 1. What specific assistance did you receive? 2. How did that assistance help you to address your needs? 3. Do you feel confident that you can now survive without further assistance? Why? D. Participation in Community activities 1. In what community activities did you participate together with IDPs/HCM? 2. Did you feel safe in participating? Why? 3. While being an IDP/HCM, was there tension in such a mixed community? How and why? (MRP External Evaluation Annexes) Page 120 | 286 Inception Report Attachment 8: TRAINING DESIGN FOR ENDLINE AND MSME SURVEYS Rationale The coordinators and field enumerators will be trained to properly administer the endline and MSME surveys before the actual data gathering. The three-day training intends to let the field coordinators and field enumerators be familiar with the survey instrument, methodology, and technology. The activity also intends to orient the participants with pertinent evaluation policies and administrative guidelines that are being adhered to by USAID/MRP/Plan/Panagora. Objectives The training will be conducted for field coordinators and field enumerators to: 1. Adhere to MRPs policies on DNH, safeguarding children and young people, and data security. 2. Be familiar with the endline survey methodology. 3. Perform ICT-based conduct of the endline survey. 4. Simulate the actual survey administration. 5. Understand pertinent administrative guidelines. 6. Identify lessons that need to be done/avoided based on the training experience. Methodology The following methodologies will be employed during the training: 1. Lecture with slide presentations 2. Role-playing and critiquing 3. Field simulation/pilot testing 1. Reflection Schedule (MRP External Evaluation Annexes) Page 121 | 286 Date/Time Topics In-charge Day 1 (May 31, 2022) Morning Rationale of the evaluation STTA Team Policies on DNH; safeguarding of children, young people STTA Team Policies on data security TBD Other relevant policies TBD Survey tablet finalization (ex. Geo-tagging)* Afternoon Overview of the survey methodology STTA Team Understanding the survey questionnaires: Endline and MSME Survey Questionnaires STTA Team Roles of coordinators STTA Team Roles of field enumerators STTA Team Day 2 (June 01, 2022) Morning Administrative guidelines CLAimDev Survey gameplan (schedule, deliverables, etc.) STTA Team Survey using the tablet STTA Team Role-playing and critiquing STTA Team Afternoon Role-playing and critiquing STTA Team (MRP External Evaluation Annexes) Page 122 | 286 Processing of experience and reflection STTA Team Day 3 (June 02, 2022) Morning Field simulation/Pilot testing Coordinators; Enumerators Afternoon Processing of experience and reflection STTA Team Team planning STTA Team Closing notes TBD * Could be done earlier (MRP External Evaluation Annexes) Page 123 | 286 Inception Report Attachment 9: KEY INFORMANT INTERVIEW (KII) Key Informant Interview [ ] GO [ ] NGO [ ] Fin’l Inst [ ] Financier [ ] Chamber [ ] Donor. Name Location Time Start Time End Date Venue Facilitator Documenter Introduction Good day! Thank you for giving your time to participate in our activity. My name is… Notes: the colored texts are intended to distinguish the other respondents – financial institutions, chamber/association, and financiers. Generally, the plain black fonts are for the rest of the respondents. A. Institutional/Organizational Mandate (Criteria: Effectiveness) (MRP External Evaluation Annexes) Page 124 | 286 1. What is your institutional/organizational mandate, programs, and projects? ✓ When was your institutional/organizational founded/organized? ✓ What is your coverage of operations? Who are your clients/beneficiaries? ✓ Concerning the Marawi Siege, what interventions have you done, and are still doing? For Financial Institutions – rural banks, development banks, lending investors/ pawnshops, - catering to the needs of the affected business and general population of Marawi siege. ✓ Concerning the Marawi Siege, what financial services have you extended, and are still doing? For Financier/Lender – informal/parallel/shadow person/family, usually among Maranao – helping the traders in the larger Marawi City’s most affected area. 1. When have you started this financing business? ✓ Who are your clients? Traders? Manufacturers? ✓ Since the Marawi siege, have you been able to collect the payments? ✓ Have you increased or decreased the number of clients? Have you increased or decreased the amount per client? For Chamber or associations of traders, etc. – formal, registered, and duly recognized association of micro and small enterprises in Marawi City and Iligan City. ✓ Concerning the Marawi siege, what responses have you made to your members (on non￾members)? 2. What makes your interventions different from the other institutions/organizations? ✓ What is the focus of your interventions – women and children, youths, farmers, MSMEs, etc? (MRP External Evaluation Annexes) Page 125 | 286 ✓ Are your interventions complementary to, or duplication of, or in competition with, other institutions/ organizations? 2. What makes your financing/lending different from the other financiers? ✓ Concerning the Marawi Siege, what financial services have you extended, and are still doing? ✓ Have you increased or decreased your clients? 2. Did you try to differ in your financing strategy from the other financiers? ✓ Who are your priority clients? ✓ What have you done to your delinquent clients brought about by the siege or pandemic? Financing policy changes? B. Knowledge about Plan International (or USAID) (Criteria: Relevance) 3. What do you know about Plan International's (or USAID) involvement in the Marawi Siege? ✓ Is their intervention in Marawi and other similarly affected areas timely or rather late? ✓ What difference did they make with the other institutions/organizations’ interventions? ✓ Do you think/believe their interventions to the affected population will last without any further presence? 4. Would Plan International have done any better? ✓ What other interventions could they have introduced? ✓ What other groups/sectors of the affected population could have benefitted? C. Business and Community (MRP External Evaluation Annexes) Page 126 | 286 5. What is the current general situation in Marawi and similarly affected areas? ✓ How is the population’s movement affected by? ✓ Security? ✓ Pandemic? ✓ Elections? ✓ How normal (or abnormal) is the “new normal”? (Or the “next to normal”?) 6. How do you compare the business activities before the Marawi Siege, after (how many months), and the present one (since when)? ✓ Has the number of local micro and small businesses, at present time increased in: numbers? product/services lines? capitalization? ✓ Has the business re-starting/reopening revitalized the population’s spending capacity? D. Cross-cutting Themes 7. How are the business activities received by the immediately affected community? ✓ What business situations or activities that created more tension/division? Who are the ‘dividers’ (‘spoilers’)? ✓ What business situations and activities that promoted more understanding/cohesion? Who are the ‘connectors’ (‘transformers’) ✓ What was done to improve the situation? What can be done further? 8. Have the business stimulated more employment across the gender – women, and men – population? ✓ Given the aftermath of the Marawi Siege’s urgency of economic needs, have there been any changes in gender preferences/biases in the employment? ✓ Similarly, has there been any changes in the participation between male and females in community meetings and decision makings? (MRP External Evaluation Annexes) Page 127 | 286 9. Have the business encouraged more livelihood activities across gender – women, and men – population? ✓ Between males and females, who usually takes the lead in livelihood activities? ✓ Has MRP encouraged more women’s participation in livelihood activities? How? D. Preventing the Marawi Siege 10. Do you think/believe that the Marawi Siege can happen again? ✓ How can the business/private sector help prevent it? ✓ How can your institution/organization help prevent it? (MRP External Evaluation Annexes) Page 128 | 286 Inception Report Attachment 10: INFORMED CONSENT (SURVEY) 1. Rationale and Purpose of the Evaluation The Marawi Response Project (MRP) is part of USAID’s development assistance that supports the reintegration of IDPs by addressing the early recovery needs of displaced individuals, and, more importantly, by addressing the long-term rehabilitation need of affected communities and the local governments in and around Marawi. Currently, we are evaluating the project. The purpose of the evaluation is to evaluate MRP’s performance in accomplishing its target outcomes. We will also examine its relevance, effectiveness, and sustainability. 2. Participant Involvement You are selected because you are considered a valuable project stakeholder. For this reason, the team hopes that you could spend time to share to your perceptions, thoughts, and views regarding the MRP. In particular, you will be involved by sharing your answers to the survey questions. It will take you about 10 to 15 minutes to complete. 3. Potential Risks, Discomforts, Discontinuance There are no risks associated with your participation. However, if you feel uncomfortable answering some questions, you could decide not to answer them or decide not to continue with your participation. You can decide to do this without any consequence or penalty. 4. Potential Benefits (MRP External Evaluation Annexes) Page 129 | 286 If you will participate, the information that you will share will help the team come up with valuable insights regarding the project. These insights could help USAID improve the way it facilitates projects to help communities such as those that are in your area. 5. Confidentiality All the information that you will share will be treated as confidential. This means that your responses in the survey will not be associated with you in a personal way after you share them. This will be done by coding your responses. No name or other identifying information will be used when discussing or reporting data. 7. Cost, Compensation, Reimbursement There is no cost for you to participate in this endeavor. Moreover, you will not be compensated for your participation. In case, you will spend an amount for your fare to go to a certain venue, you will be refunded the amount that you spent. 8. Voluntary Participation and Authorization Your decision to participate in this study is completely voluntary. If you decide to not participate in this study, it will not affect the services, or benefits of the project to which you are entitled. Consent Considering all the information given above, I voluntarily agree to participate in this evaluation: □ Yes □ No (MRP External Evaluation Annexes) Page 130 | 286 By affixing my signature below, I further authorize the use of my responses in this evaluation for purposes of learning, and/or presentation. Name of Participant (print): _______________________________________ Signature of Participant: _______________________________________ Date: ______________________ (MRP External Evaluation Annexes) Page 131 | 286 (MRP External Evaluation Annexes) Page 132 | 286 INFORMED CONSENT (FGD/KII) 1. Rationale and Purpose of the Evaluation The Marawi Response Project (MRP) is part of USAID’s development assistance that supports the reintegration of IDPs by addressing the early recovery needs of displaced individuals, and, more importantly, by addressing the long-term rehabilitation need of affected communities and the local governments in and around Marawi. Currently, we are evaluating the project. The purpose of the evaluation is to evaluate MRP’s performance in accomplishing its target outcomes. We will also examine its relevance, effectiveness, and sustainability. 2. Participant Involvement You are selected because you are considered a valuable project stakeholder. For this reason, the team hopes that you could spend time to share to your perceptions, thoughts, and views regarding the MRP. In particular, you will be involved by sharing your answers to the focus group discussion (FGD)/key informant interview (KII) questions. You will participate in the FGD/KII for about 1 hour to 1 ½ hours. 3. Potential Risks, Discomforts, Discontinuance There are no risks associated with your participation. However, if you feel uncomfortable answering some questions, you could decide not to answer them or decide not to continue with your participation. You can decide to do this without any consequence or penalty. 4. Potential Benefits (MRP External Evaluation Annexes) Page 133 | 286 If you will participate, the information that you will share will help the team come up with valuable insights regarding the project. These insights could help USAID improve the way it facilitates projects to help communities such as those that are in your area. 5. Confidentiality All efforts will be made to ensure that your identity will be anonymous during the evaluation. The information that you will share during the FGD/KII will be treated as confidential. This will be done by coding your responses. No name or other identifying information will be used when discussing or reporting data. 8. Cost, Compensation, Reimbursement There is no cost for you to participate in this endeavor. Moreover, you will not be compensated for your participation. In case, you will spend an amount for your fare to go to a certain venue, you will be refunded the amount that you spent. (MRP External Evaluation Annexes) Page 134 | 286 9. Voluntary Participation and Authorization Your decision to participate in this study is completely voluntary. If you decide to not participate in this study, it will not affect the services, or benefits of the project to which you are entitled. Consent Considering all the information given above, I voluntarily agree to participate in this evaluation: □ Yes □ No By affixing my signature below, I further authorize the use of my responses, and the video and audio recordings of the FGD/KII for purposes of learning, and/or presentation. Name of Participant (print): _______________________________________ Signature of Participant: _______________________________________ Date: ______________________ (MRP External Evaluation Annexes) Page 135 | 286 Inception Report Attachment 11: MRP EVALUATION WORKPLAN (Based on the Task Order and Milestone) MRP EVALUATION WORKPLAN (BASED ON THE TASK ORDER AND MILESTONE Tasks Schedule Inception Report with Evaluation Design, Work Plan, and Inception Presentation / Initial briefing and an inception report briefing In-Person Team Planning Session 09-Apr-22 Submission of workshop outputs to COR and AOR by Panagora 11-Apr-22 Virtual In Brief Meeting with USAID 11-Apr-22 STTA data gathering and Desk Review of Documents April 11-15, 2022 Formulate key questions for the initial consultation with the LGUs April 11-12, 2022 Analysis of gathered data and information April 18-20, 2022 Drafting the Inception Report, with development of evaluation plan and tools (desk work and virtual meetings) April 21-May 9, 2022 Submission of draft inception report to Panagora May 10, 2022 Submission of inception report to USAID May 13, 2022 In-Brief Meeting tbd Revision of Inception Report, with development of evaluation plan and tools (desk work and virtual meetings) May 10-12, 2022 Submission of report by Panagora to USAID May 23, 2022 (MRP External Evaluation Annexes) Page 136 | 286 MRP EVALUATION WORKPLAN (BASED ON THE TASK ORDER AND MILESTONE Tasks Schedule Mid-term briefing and interim meetings Installation of the electronic survey tool 13-May-22 Pilot-Test and finalize tools (virtual and onsite) May 16-20, 2022 Field coordination and logistic preparation (on-site) May 16-20, 2022 Analysis of pilot test data and revision of data gathering tools 23-May-22 Develop training design for enumerators and facilitators 24-May-22 Train the end line field enumerators on survey administration and FGD facilitation (on-site) May 25-27, 2022 Actual Data Collection (survey, FGD, KII, etc.) (virtual and on-site) May 30-June 17, 2022 Progress report preparation June 20-22, 2022 Submission of progress report 23-Jun-22 Review by Panagora and USAID June 24-28,2022 Virtual Midterm briefing with USAID/PH PRM after completing the fieldwork 29-Jun-22 Out-briefing and USAID presentations Data Processing and Analysis (deskwork and virtual meetings) June 30-July 20, 2022 Packaging of Initial Summary Findings, Conclusions, and Recommendations (deskwork) July 21-27, 2022 Series of Virtual Presentation Workshops on the Findings, Conclusions, and Recommendations to the IP and various stakeholders July 28-August 5, 2022 (MRP External Evaluation Annexes) Page 137 | 286 MRP EVALUATION WORKPLAN (BASED ON THE TASK ORDER AND MILESTONE Tasks Schedule Refinements on the Summary of Findings, Conclusions, and Recommendations August 8-12, 2022 (Out-Brief) Presentation of Refined Findings, Conclusion, and Recommendations to USAID (PRM and OEDG offices) August 15, 2022 Draft Evaluation Report Drafting the 1st Version of the Evaluation Report August 16-31, 2022 Submission of 1st draft of the evaluation report to Panagora August 31, 2022 Review by Panagora September 1-8, 2022 Revision of 1st draft evaluation report September 9-12, 2022 Submission of revised draft Evaluation report incorporating recommendations from CLAimDev and USAID. 13-Sep-22 Learning Events Preparation of learning materials for the Learning Events Sept 14-15, 2022 STTA Presentation of the Evaluation Results during the Learning events TBD Final Evaluation Report and Learning Event Presentations Final report revisions incorporating recommendations from USAID and CLAimDev September 15-30, 2022 Submission of the Final Report October 5, 2022 (MRP EXTERNAL EVALUATION ANNEXES) PAGE 138 | 344 Annex 3 SAMPLE SELECTION CRITERIA AND PROCESS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 139 | 286 Annex 3 SAMPLE SELECTION CRITERIA AND PROCESS membership ( of individuals livelihood grants ( of grants social cohesion grants ( of grants arious training ( of individuals den fy the pro ect sites with the most numbers of bene ciaries per type of interven on. Provide a score of for each site per type of interven on ompute the total score per pro ect site ssign total scores ( very high, high, somewhat high, somewhat low, low ther pro ect sites with scores are considered P (MRP External Evaluation Annexes) Page 140 | 286 MRP External Evaluation Annexes) Page 141 | 286 (MRP External Evaluation Annexes) Page 142 | 286 SAMPLING PROCESS FOR THE EXPANDED ENDLINE SURVEY Intervention group The locations of the intervention group cross-sectional samples were identified using the following procedures: 1. Municipalities were described based on the extent of MRP intervention. Here the municipalities were scored using a 5-point scale (5-very high, 4-high, 3-somewhat high, 2- somewhat low, 1-low) in relation to number of CSG members, number of CMG IR 1 grants, number of CMG IR 2 grants, number of BRG grants, number of training recipients. 2. Municipalities were described based on safety and accessibility. The municipalities were indexed using four elements namely, communication, road condition, distance, and peace and order. The enumerators having familiarity with the areas were asked to use a 4- point scheme (4=very favorable, 3=favorable, 2=somewhat favorable, 1=unfavorable) to describe each municipality in relation to the four elements. 3. Municipalities were classified as “Go Location” or “Go Location with Caution” by combining the two criteria above. The following table shows the summary of this procedure: MRP External Evaluation Annexes) Page 143 | 286 Comparison group The locations for the comparison group cross-sectional samples were identified using the following criteria: 1. Risk and Accessibility of non-MRP municipalities (or non-MRP barangays in covered municipalities) 2. Similar geographical and demographic features with MRP’s intervention survey sites (e.g., rural/urban, class, population, culture, etc.) 3. Presence of IDPs and HCMs who are not located in MRP project sites (municipalities or barangays) 4. Presence of IDPs and HCMs who have NOT received grants or participated in any activities/trainings under MRP through Plan, ECOWEB or MARADECA. 5. If possible, presence of IDPs and HCMs who have NOT received similar grants or participated in any activities/training of other international donor agencies. Baloi Iligan City Municipality Barangays Urban/Rural Population # of MRP beneficiaries % Type of respondents # of respondents Baloi Cadayonan Rural 2,170 31 1% intervention 15 Baloi Matampay Rural 2,781 154 6% intervention 13 Baloi Poblacion East Rural 6,595 274 4% intervention 10 Baloi Poblacion West Urban 6,078 346 6% intervention 17 Baloi Sandor Rural 1,963 107 5% intervention 12 Baloi Abaga Rural 3,907 none comparison 18 Baloi Nangka Rural 4,752 none comparison 17 (MRP External Evaluation Annexes) Page 144 | 286 Masiu Saguiaran Note: Barangays Cadayon and Pawak of Saguiaran were replaced by Barangay Olowa Ambolong, Marawi City as the alternative comparison group area based on the assessment and recommendation of the field coordinator. Municipality Barangays Urban/Rural Population # of MRP beneficiaries % Type of respondents # of respondents Iligan Buru-un Urban 16,835 28 0% Intervention 15 Iligan Del Carmen Urban 9,662 30 0% Intervention 16 Iligan Mahayhay Urban 7,965 197 2% Intervention 21 Iligan Santo Rosario Urban 1,839 69 4% Intervention 17 Iligan Tambacan Urban 19,261 204 1% Intervention 14 Iligan Tubod Urban 33,243 159 0% Intervention 17 Iligan Hinaplanon Urban 15,424 0% Comparison 36 Municipality Barangays Urban/Rural Population # of MRP Beneficiaries % Type of Respondents # of respondents Masui Talub Langi Rural 1,704 126 7% Intervention 21 Masui Macalupang Lumbac Caramian Rural 733 109 15% Intervention 32 Masui Manalocon Talub Rural 957 141 15% Intervention 35 Masui Matao Araza Rural 715 112 16% Intervention 14 Masui Pantao Rural 2,336 0% Comparison 18 Masui Laila Lumbac Bacon Rural 1,163 0% Comparison 17 Municipality Barangays Urban/Rural Population # of MRP Beneficiaries % Type of Respondents # of respondents Saguiaran Lumbac Toros Rural 920 129 14% Intervention 18 Saguiaran Mipaga Rural 661 73 11% Intervention 16 Saguiaran Pagalamatan Rural 680 145 21% Intervention 22 Saguiaran Pantao Raya Rural 613 88 14% Intervention 11 Saguiaran Sungcod Rural 640 93 15% Intervention 13 Saguiaran Bubong Rural 1,508 199 13% Intervention 16 Saguiaran Cadayon Rural 1,098 0% Comparison 15 Saguiaran Pawak Urban 2,430 0% Comparison 15 MRP External Evaluation Annexes) Page 145 | 286 (MRP External Evaluation Annexes) Page 146 | 286 Annex 4 EVALUATION PARTICIPANTS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 147 | 286 Annex 4 EVALUATION PARTICIPANTS A. Selected Survey Samples Municipality / Barangays Number of Participants Baloi 147 Abaga 29 Nangka 59 Sandor 16 Cadayonan 1 Poblacion East 25 Matampay 10 Poblacion West 7 Iligan City 118 Buruun 13 Del Carmen 4 Hinaplanon 26 Mahayahay 11 Sto. Rosario 23 Tambacan 14 Tubod 27 Marantao 1 Nataron 1 Marawi City 62 Olowa Ambolong 62 Masiu 213 (MRP External Evaluation Annexes) Page 148 | 286 Municipality / Barangays Number of Participants Laila Lumbac Bangon 37 Matao Araza 43 Pantao 25 Macalupang Lumbac Caramian 51 Talub Langi 21 Manalocon Talub 36 Poona Bayabao 2 Cadayonan 2 Saguiaran 136 Bubong 12 Lumbac Toros 31 Mipaga 41 Pagalamatan 32 Pantao Raya 7 Sungcod 13 Grand Total 679 B. Key Informant Interview Participants Municipality / Role or Organization Number of Participants Balindong, Lanao del Sur 6 HCM Leader 1 IDP/Women Empowerment 2 LCE/MPDC 1 MRP External Evaluation Annexes) Page 149 | 286 Municipality / Role or Organization Number of Participants Pilumbay Marantao Youth 1 Saguiaran United Organization 1 Baloi, Lanao del Norte 2 Achiever 2 Buadiposo Buntong, Lanao del Sur 5 HCM, Leader 1 IDP, Leader 1 LCE/MPDC 1 MLGU 2 Bubong, Lanao del Sur 3 Administrator, MLGU 1 Farmer Leader 1 Former Administrator, MLGU 1 Cagayan de Oro City, Misamis Oriental 1 Department of Trade and Industry 1 Ditsaan Ramain, Lanao del Sur 4 HCM, Leader 1 IDP, Leader 1 MPDC 1 New Townians Women 1 Iligan City, Lanao del Norte 9 BRG Beneficiary 2 Dissolved CSG 2 MSU-IIT IPDM 1 (MRP External Evaluation Annexes) Page 150 | 286 Municipality / Role or Organization Number of Participants Plan International 4 Marantao, Lanao del Sur 1 Panimbang ko Ingud a Kormat ko Bangsa 1 Marawi City, Lanao del Sur 3 Bangon Marawi Chamber of Commerce 1 BRG Beneficiary 1 Dayawan Handicraft Loom (BRG Marawi) 1 Grand Total 34 C. Focus Group Discussion Participants Municipality / Role or Organization Number of Participants Balindong, LDS 4 Kangodaan A Rawaten 4 Buadiposo Buntong, LDS 10 Sowara o mga Bae Compt. Services and Dressmaking 10 Bubong, LDS 5 United Suggod Organization 5 Butig, LDS 15 Ragayan Didaitun-Farmers Group 8 Reposition Groceries Women 7 Ditsaan Ramain, LDS 6 Kalilintad Women Organization 6 MRP External Evaluation Annexes) Page 151 | 286 Iligan City, LDN 22 ECOWEB 12 Muslim-Christian Group 10 Marantao, LDS 25 Kanggiginawae ko Masa (Women) 10 MARADECA 15 Grand Total 87 (MRP External Evaluation Annexes) Page 152 | 286 Annex 5 ENUMERATORS’ TRAINING DESIGN Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 153 | 286 Annex 5 ENUMERATORS’ TRAINING DESIGN Rationale The coordinators and field enumerators will be trained to properly administer the endline survey prior to the actual data gathering. The three-day training intends to let the field coordinators and field enumerators be familiar with the survey instrument, methodology and technology. The activity also intends to orient the participants with pertinent evaluation policies and administrative guidelines that are being adhered to by USAID/MRP/Plan/Panagora. Objectives The training will be conducted for field coordinators and field enumerators to: 1. Adhere to MRP’s policies on DNH, safeguarding children and young people and data security. 2. Be familiar with the endline survey methodology. 3. Perform ICT-based conduct of the endline survey. 4. Simulate the actual survey administration. 5. Understand pertinent administrative guidelines. 6. Identify lessons that need to be done/avoided based on the training experience. Methodology The following methodologies will be employed during the training: 1. Lecture with slide presentations 2. Role playing and critiquing. 3. Field simulation/pilot testing 4. Reflection Schedule (MRP External Evaluation Annexes) Page 154 | 286 Date/Time Topics In-charge Day 1 (May 31, 2022) Morning Rationale of the evaluation STTA Team Policies on DNH; safeguarding of children, young people Larry Agpalo (Plan International) Ethical considerations STTA Team Afternoon Overview on survey methodology STTA Team Understanding the endline survey questionnaire STTA Team Roles of coordinators STTA Team Roles of field enumerators STTA Team Day 2 (June 1, 2022) Morning Survey gameplan (schedule, deliverables etc.) STTA Team Survey using the tablet STTA Team Role playing and critiquing STTA Team Afternoon Role playing and critiquing STTA Team Processing of experience and reflection STTA Team Day 3 (June, 2 2022) Morning Field simulation Coordinators; Enumerators Afternoon Processing of experience and reflection STTA Team Team planning STTA Team Closing notes TBD MRP External Evaluation Annexes) Page 155 | 286 Annex 6 PILOT TEST DESIGN AND ADMINISTRATION GUIDE Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 156 | 286 Annex 6 PILOT TEST DESIGN AND ADMINISTRATION GUIDE 1. PILOT TEST DESIGN Rationale Pilot testing of questionnaires is an essential process for research, surveys, and other data gathering endeavors. It allows researchers and evaluators to spot potential problems related to the tools. These could include among other problems related to clarity of direction, items and response options. The process can also help ensure that prospective respondents will have a homogenous understanding of the terms, phrases or statements in the tool. If performed properly, it will result in tools with robust validity and reliability properties. Ultimately, the process helps achieve better data quality. In this activity, the endline questionnaire will be pretested. The questionnaire will be subjected to a respondent-driven pilot testing process. Objectives The activity is designed to achieve the following two-fold objectives: 7. Administer the initial version of the endline questionnaire to a small group of respondents to gather data on: a. Clarity of directions, items and response options b. Response latency c. Behavioral responses to items 8. Utilize pilot test data to enhance the endline survey questionnaire. Methodology Sampling MRP External Evaluation Annexes) Page 157 | 286 Twelve purposively selected respondents will be involved in the pilot test. The sample has to have proportional representation in terms of respondent type (IDPs and HCMs) and gender (Male and Female). Procedures 1. The STTA Team will identify a pretest proctor and a documenter. The main role of the proctor is to administer the survey to the respondents. The role of the documenter is to record the administration process using the Pilot-test Observation Form. It is suggested that the two field coordinators will take the role of being proctor and a documenter. 2. The STTA Team will orient the proctor and documenter regarding the process and of their roles. 3. Purposively identify 12 pretest respondents. Six respondents will be involved in the endline survey pretest. The other six respondents will be in the MSME survey pretest. 4. The proctor and the documenter will locate each respondent. 5. The proctor and documenter will execute the pretest process following the steps below: a. The proctor will explain the objective of the activity. b. The proctor will read the direction of the questionnaire. c. After the reading, the proctor will ask the respondent about the clarity of the direction. The documenter will record observations using the Pilot Test Observation Form. d. The proctor will read the survey items and let the respondent give his/her response. The documenter will record the response latency and any behavioral response for each item. e. After completing an item, the proctor will ask the respondent about clarity of the item and the response options. The documenter will record the observations. This will be done for each item in the questionnaire. f. The proctor will thank each respondent after completing the pretest. 6. The STTA Team will consolidate the data and identify areas for enhancement. 7. The STTA Team will improve the questionnaires based on the pilot test data. Data Gathering Tools 1. Pretest Probe Questions. The following questions will be asked by the proctor to each respondent: Questions Response Remarks 1. How will you describe the clarity of the direction? 2. What word, phrase or sentence do you think needs to be clarified in the direction? 3. How will you describe the clarity of the item? (MRP External Evaluation Annexes) Page 158 | 286 4. What word, phrase or sentence do you think needs to be clarified in the item you just responded? 5. How will you describe the clarity of the options given to you? 6. What part of the options is not clear to you? Note to the proctor: Repeat asking item 3, 4, 5, and 6 for all the items in the survey. 2. Pretest Observation Form. The following form will be utilized by the documenter to record the observations from respondents: Respondent Profile Type [ ] IDP [ ] HCM Date: Gender [ ] Male [ ] Female Place: Observations Questionnaire Part Clarity Response latency (sec) Behavioral response observed Respondent suggestion; Other remarks Directions [ ] Clear [ ] Vague --- Item 1 – Item [ ] Clear [ ] Vague Item 1 – Options [ ] Clear [ ] Vague Item 2 – Item [ ] Clear [ ] Vague Item 2 – Options [ ] Clear [ ] Vague Item n Schedule MRP External Evaluation Annexes) Page 159 | 286 Date/Time In-charge Day 1 Morning Team Arrival Afternoon Orientation of proctor and documenter STTA Team Sampling of pretest respondents STTA Team Day 2 Morning Endline respondent 1; Area 1; M; IDP Proctor; Documenter Endline respondent 2; Area 1; F; IDP Afternoon Endline respondent 3; Area 2; M; HCM Endline respondent 4; Area 2; F; HCM Day 3 Morning Endline respondent 5; Area 3; M; IDP Proctor; Documenter Endline respondent 6; Area 3; F; HCM Afternoon Reflection; Planning; Homeward bound STTA Team; Proctor; Documenter 2. PILOT-TEST ADMINISTRATION GUIDE Objectives The activity is designed to achieve the following two-fold objectives: 1. Administer the initial version of the endline and MSME survey tools to a small group of respondents to gather data on: a. Clarity of directions, items and response options b. Response latency c. Behavioral responses to items 2. Utilize pilot-test data to enhance the endline survey questionnaires. (MRP External Evaluation Annexes) Page 160 | 286 Procedures 1. The STTA Team will identify proctors (JSC & MCL) and documenters (TBD). In particular the following will be considered: Team 1: JSC – Proctor; TBD (Documenter); JPM – Supervisor Team 2: MCL – Proctor; TBD (Documenter); JPM – Supervisor The main role of the proctor is to administer the survey to the respondents. The proctor will also ask the Probe Questions. The role of the documenter is to record the administration process using the Observation Form. 2. The Project Assistant will establish the location and schedule of each participant. 3. The proctor and the documenter will locate each respondent based on the schedule. 4. The proctor and documenter will execute the pretest process following the steps below: a. The proctor will explain the objective of the activity. The proctor will say: “Good day. We are here because you were referred by ECOWEB/MARADECA. You were referred because you are a beneficiary of MRP. Our purpose is to test a survey questionnaire that will be used for MRP evaluation. Your role is to share your answers to the questions and give us feedback about the clarity of the questions and the response options that you will choose from. This will take about 30-45 minutes to complete.” b. The proctor will read the direction of the questionnaire. After reading, the proctor will ask the respondent about the clarity of the direction. After dealing with the directions, the proctor will proceed to the items in the questionnaire. The proctor will say the complete item number for the documenter to record. The proctor will ask each item and present the response options. After completing an item, the proctor will ask the respondent about clarity of the item and the response options. The proctor will use the following as guide: Questions Response Remarks 1. How will you describe the clarity of the direction? 2. What word, phrase or sentence do you think needs MRP External Evaluation Annexes) Page 161 | 286 to be clarified in the direction? 3. How will you describe the clarity of the item? 4. What word, phrase or sentence do you think needs to be clarified in the item you just responded? 5. How will you describe the clarity of the options given to you? 6. What part of the options is not clear to you? Note to the proctor: Repeat asking item 3, 4, 5, and 6 for all the items in the survey. c. The documenter will record observations in Step 4.b using the Observation Form. The documenter has to be keen in recording the process for each part or item being tested. Respondent Profile Type [ ] IDP [ ] HCM Date: Gender [ ] Male [ ] Female Place: Observations Questionnaire Part Clarity Response latency (sec) Behavioral response observed Respondent suggestion; Other remarks Directions [ ] Clear [ ] Vague --- Item 1 – Item [ ] Clear [ ] Vague Item 1 – Options [ ] Clear [ ] Vague Item 2 – Item [ ] Clear [ ] Vague Item 2 – Options [ ] Clear [ ] Vague Item n d. The proctor will thank each respondent after completing the pretest. The proctor could say: “Maraming salamat po sa inyong participation. Salamat sa time na binigay niyo sa pretest na ito.” 5. The STTA Team will consolidate the data and identify areas for enhancement. 6. The STTA Team will improve the questionnaires based on the pilot-test data. (MRP External Evaluation Annexes) Page 162 | 286 List of Respondents during the Pilot Test A. Endline Name Location Sex Identity Group Respondent #1 Tubod, Iligan City Male IDP Respondent #2 Tubod, Iligan City Female HCM Respondent #3 Tubod, Iligan City Female IDP Respondent #4 Tubod, Iligan City Male IDP Respondent #5 Tubod, Iligan City Female HCM Respondent #6 Sto. Rosario, Iligan City Male IDP Respondent #7 Sto. Rosario, Iligan City Male IDP Respondent #8 Tubod, Iligan City Female IDP Respondent #9 Tubod, Iligan City Female IDP Respondent #10 Tubod, Iligan City Female IDP Respondent #11 Buru-un, Iligan City Female HCM B. Socio-economic Name Location Sex Identity Group Respondent #1 Nangka Balo Female IDP Respondent #2 West Poblacion, Baloi Female IDP Respondent #3 East Poblacion, Baloi Male IDP Respondent #4 Poblacion, Iligan City Female IDP MRP External Evaluation Annexes) Page 163 | 286 Respondent #5 Poblacion, Iligan City Male IDP Respondent #6 Iligan City Female IDP Respondent #7 Mahayahay, Iligan City Male IDP (MRP External Evaluation Annexes) Page 164 | 286 Annex 7 ENDLINE SURVEY TOOL Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 165 | 286 Annex 7 ENDLINE SURVEY TOOL (ENGLISH VERSION) Participant Code: ____________________ Time Start: _______________________ Date of Interview: ____________________ Time End: _______________________ 1. LOCATION 1.1 Province 0 1 Lanao del Norte Lanao del Sur 1.2 Municipality 1 2 3 4 5 Ilihan City Baloi Saguiaran Poona Bayabao Masiu 1.3 Barangay 1 2 3 4 5 6 7 8 9 10 Sto. Rosario Del Carmen Tubod Buruun East Poblacion Pagalamatan Mipaga Lumbaca Toros Cadayonan Matao Raza 2. RESPONDENT’S PROFILE 2.1 Location category 0 1 Comparison group area Intervention group area (MRP External Evaluation Annexes) Page 166 | 286 2.2 Baseline identity 0 1 IDP HCM 2.3 Name of CSG ___________________________________________ 2.4 Sex 0 1 Female Male 2.5 Age Specify age in years _____ 2.6 Marital Status 1 2 3 4 88 Married Separated Widow/Widower Single/Unmarried Others, please specify 2.7 Religion 0 1 2 3 4 5 99 None Christian (including Catholic, Protestant, Evangelical Christian, Seventh Day Adventist, and other Christian Denominations) Islam Jehovah’s Witnesses Iglesia ni Kristo Buddhist Others _________________ (please specify) 2.8 Highest educational attainment 0 1 2 3 4 5 Did not attend school. Elementary level Elementary graduate High school level High school graduate College level MRP External Evaluation Annexes) Page 167 | 286 6 7 8 9 99 College graduate Vocational graduate Master’s degree/Doctor’s degree level Master’s degree/Doctor’s degree graduate Others _________________ (please specify) 2.9 Are you currently employed? 1 0 Yes No (Proceed to 2.10) 2.9.1 Type of employment 1 2 3 4 Permanent Probation/Temporary Part-time Self-employed 2.9.2 What industry are you employed in? (You may answer more than one.) 1 2 3 4 5 6 7 99 Farming crops Farming Livestock/Fishing Sari-sari Store/Food Retailing Non-food Retailing (e.g., RTW, textile) Private Company Government Unit/Agency Non-Profit organization Others_________________________________ 2.9.3 How long have you been employed or self-employed? Specify: ____ years ____ months 2.10 If no, why are you not currently employed? 1 2 3 4 There are no jobs in my area. I am not qualified for the jobs in my area. I have been applying for jobs but have not been offered any job. I am unable to work due to a health issue or disability. Others __________________ (MRP External Evaluation Annexes) Page 168 | 286 99 2.11 At present, are you a member of any group/association/ organization? 1 0 66 Yes (If yes, proceed to 2.11.1) No (If no, proceed to 3.1) Refused to answer 2.11.1 If yes, what groups/ association/organization are you a member of? 1 2 3 4 5 6 99 Community Solidarity Group Farmers’ Association Women’s Association Trade Union Mosque/Church/Religious Group Youth Group Others ____________________________________ 3. HOUSEHOLD PROFILE 3.1 Who leads in the family decision-making? 1 2 3 4 5 6 99 Father Mother Both father and mother Uncle Aunt Grandparents Others _________________ (please specify) 3.2 How many members of this household are 0-29 years old? Specify the number: _______ 3.3 How many in this household are 30 and above? Specify the number: _______ 3.4 How long have you been living in this current residence? Specify the number of years: _______ MRP External Evaluation Annexes) Page 169 | 286 3.5 What is the ownership status of the house you are currently living in? 1 2 3 4 5 6 7 99 We own the house. We rent the house/room (Proceed to 4.1). We are the caretaker of the house. We occupy the house for free. We co-own the house. We are living with the host family. The house is provided free (NGO, government, …) Others, please specify: ________________ 4. HOUSEHOLD EXPENDITURES AND DEBT 4.1 If renting, how much is your monthly rent? Specify the amount. PhP__________________________ (Put N/A or Not Applicable if not renting) 4.2 During the past month, what are the top 3 expenditures your HH has spent most on? (Please write the number 1, 2, 3) on the space provided). If food is top 1, then write 1 on the space. ___ ___ ___ ___ ___ ___ ___ ___ 99 Food Non-food items (includes clothing, hygiene kits, etc.) Medicine and/or hospital bill Loan payment Child/Children’s education Monthly bills (e.g., electricity, water) Shelter repair/building of house House rent Others, please specify: __________________ 4.2.1 How much is your household monthly expenditure for ___ (say the answer Ranked 1 in 4.2)? Specify the amount. PhP__________________________ 4.2.2 How much is your household monthly expenditure for ___ (say the answer Ranked 2 in 4.2)? Specify the amount. PhP__________________________ (MRP External Evaluation Annexes) Page 170 | 286 4.2.3 How much is your household monthly expenditure for ___ (say the answer Ranked 3 in 4.2)? Specify the amount. PhP__________________________ 4.3 Do you or anyone in your household currently have any debt/loan? 1 0 55 66 Yes No Don’t know. Refused to answer 4.3.1 If yes, what was the source of the debt/loan? (You may answer more than one.) 1 2 3 4 5 55 66 99 Friend/Neighbor Relative Moneylender Bank/formal lending institution Traders Don’t know. Refused to answer. Others______________________________________ 4.3.2 If yes, what is the main purpose for taking out the loan? (You may answer more than one.) 1 2 3 4 5 6 7 55 66 99 Buy food. Buy medicine and/or pay hospital bill. Pay school expenses. Invest in business. Pay previous debt/loan. Pay monthly bills (water, electric, etc.) Pay house rent. Don’t know. Refused to answer. Others_____________________________________ MRP External Evaluation Annexes) Page 171 | 286 5. HOUSEHOLD INCOME 5.1 What are your household’s source or sources of income at present? (You may answer more than one.) 1 2 3 4 5 6 7 8 99 Farming crops Farming livestock/fishing Sari-sari store/food retailing Non-food retailing (e.g., RTW, textile) Paid work (including wages and pay for services/labor) Donor assistance (e.g., from an aid agency, a religious institution) 4Ps Remittance Others _________________ 5.2 Do you have savings? 1 0 Yes No If “No,” proceed to 5.3 5.2.1 How do you save money? 1 2 3 4 99 Alkansiya/Piggy Bank Bank Cooperative Community Savings Group (like Paluwagan) Others __________________________________ 5.3 Did anyone in your household receive skills training in the past 6 months? 1 0 55 66 Yes No Don’t know. Refused to answer 5.4 Did anyone in your household receive business development training (including financial literacy) in the past 6 months? 1 0 Yes No (MRP External Evaluation Annexes) Page 172 | 286 55 66 Don’t know. Refused to answer 6. PERCEIVED SELF-RELIANCE 6.1 I have the right skills to find a job. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.2 There are sufficient job opportunities in my community. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.3 If I were to lose my job or livelihood tomorrow, I feel confident that I would find another job within a month. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.4 If I were to lose my job or livelihood tomorrow, I feel confident that I would start a new livelihood within a month. 1 2 3 4 55 Strongly disagree. Disagree Agree Strongly agree. Don’t know. MRP External Evaluation Annexes) Page 173 | 286 66 Refused to answer 6.5 The income I earn from my job/livelihood is sufficient to cover the basic needs of my family. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.6 In times when our income is not enough for the basic needs of my family, I have enough assets that I can sell to cover the basic needs of my family. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.7 In times when our income is not enough for the basic needs of my family, I have enough savings to cover the basic needs of my family. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.8 In times when our income is not enough for the basic needs of my family, I can easily borrow money from relatives and/or friends to cover the basic needs of my family. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 6.9 In times when our income is not enough for the basic needs of my family, the remittances 1 Strongly disagree. (MRP External Evaluation Annexes) Page 174 | 286 we receive are enough to cover the basic needs of my family. 2 3 4 55 66 Disagree Agree Strongly agree. Don’t know. Refused to answer 6.10 I can easily access skills training opportunities in my barangay. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 7. PERCEIVED LEVEL OF POLARIZATION 7.1 In the barangay where you are currently living in, are you are considered a: 1 2 Internally Displaced Person (IDP)? Host Community Member (HCM)? 7.2 Most people in the neighborhood are willing to help if you ask. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 7.3 Most people in the neighborhood can be trusted. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer MRP External Evaluation Annexes) Page 175 | 286 7.4 Most people in the neighborhood are honest. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.5 It is naïve to trust people. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.6 I feel that I belong to my immediate community/neighborhood. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 7.7 Differences often exist between people living in the same neighborhood. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.8 Different ethnic tribes tend to divide the people in our neighborhood. 1 2 Strongly disagree. Disagree (MRP External Evaluation Annexes) Page 176 | 286 3 4 55 66 Agree Strongly agree. Don’t know. Refused to answer 7.9 Religious differences tend to divide people in my neighborhood. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. I would like to ask you about the trust you have in different groups of people. Please tell me if you agree or disagree with the following statements: 7.10 I trust the CSG officials. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 7.11 I trust my fellow CSG members. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.12 I trust the local elected officials in the barangay I am currently living in. 1 2 3 Strongly disagree. Disagree Agree MRP External Evaluation Annexes) Page 177 | 286 4 55 66 Strongly agree. Don’t know. Refused to answer. 7.13 I trust the traditional/religious leaders in the barangay I am currently living in. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.14 I trust the non-government organizations. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 7.15 I trust the police. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer. 7.16 I trust the military. 1 2 3 4 55 Strongly disagree. Disagree Agree Strongly agree. Don’t know. (MRP External Evaluation Annexes) Page 178 | 286 66 Refused to answer 7.17 I am willing to work with IDP/HCMs to implement projects. 1 2 3 4 55 66 Strongly disagree. Disagree Agree Strongly agree. Don’t know. Refused to answer 8. PERCEIVED PUBLIC REPRESENTATION 8.1 I am consulted and involved in the implementation of projects in my community (e.g., for water system, security) 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer. 8.2 Ordinary people or ordinary members participate when important decisions are made in the community or CSG. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.3 Youth participate when important decisions are made in the community or CSG. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer MRP External Evaluation Annexes) Page 179 | 286 8.4 Women participate when important decisions are made in the community or CSG. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.5 People from my own ethnic group participate when important decisions are made in the community or CSG. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.6 People who share my religious views participate when important decisions are made in the community or CSG. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.7 I attended a community or CSG meeting. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.8 I talked personally to the CSG leader/BLGU Official about a problem concerning our community. 1 2 Never Sometimes (MRP External Evaluation Annexes) Page 180 | 286 3 4 55 66 Often Very Often Don’t know. Refused to answer 8.9 I talked to/called the CSG leader/BLGU Official about a problem concerning our community. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.10 I wrote a letter to the CSG leader/BLGU Official about a problem concerning our community. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t knowz. Refused to answer 8.11 I voted during the election of CSG officers. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.12 I talked personally to the CSG leader/BLGU Official about a problem concerning our community. 1 2 3 4 Never Sometimes Often Very Often MRP External Evaluation Annexes) Page 181 | 286 55 66 Don’t know. Refused to answer 8.13 I participated in the implementation of community or CSG projects or activities. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.14 I am involved in approaching government and non￾government organizations to make use of resources for projects to improve our community (e.g., for water system, security). 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.15 I am an active member of civic/community/religious organization(s) like CSG, Farmers’ Association, Trade Union, Mosque/Church/religious group, youth group and others. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer 8.16 I participate in training to develop ways to solve community problems, find work or better job, run a business, or write an application for a loan or business grant. 1 2 3 4 55 66 Never Sometimes Often Very Often Don’t know. Refused to answer (MRP External Evaluation Annexes) Page 182 | 286 ENDLINE SURVEY TOOL (TAGALOG VERSION) Date of Interview: _______ Auto set Time starts : _______ Auto set. Name of Coordinator/Enumerator: Drop down list of names of coordinators/enumerators. Type of Interview: Drop down: 1 – Phone 2 – In-person 1. LOCATION 1.1 Province Drop down 1.2 Municipality Drop down filter by province 1.3 Barangay Drop down filter by municipality 1.4 Participant’s Name Drop down list. Others, please specify ___________ 1.5 Type of Respondent 1 – IA (Intervention A) 2 – IB (Intervention B) 3 – CA (Comparison A) 4 – CB (Comparison B) INFORMED CONSENT MRP External Evaluation Annexes) Page 183 | 286 Ang survey na ito ay bahagi ng MRP Performance Assessment. Ang layunin ng survey ay upang maunawaan ang epekto ng proyekto mula noong nagsimula ito noong 2017. Ang iyong mga sagot ay makakatulong sa pagpapabuti ng mga proyekto para sa mga IDP at HCM sa hinaharap. Kung ayaw mong lumahok, puwede niyo pong ipalam upang hindi namin itutuloy ang survey. Kapag nakapagsimula ka na at ayaw mong nang magpatuloy, maaari kang ring huminto anumang oras. At kung ikaw ay hihinto, hindi po maapektuhan ang iyong paglahok sa proyekto ng MRP. Kung may tanong na hindi malinaw, ipaalam niyo lang po at ipapaliwanag ko sa iyo hanggang ito ay maging malinaw para sayo. Hindi po namin ipapalam sa mga taong walang kinalaman sa MRP ang iyong mga sagot. Handa ka bang lumahok sa survey na ito?  Oo  Hindi (If Hindi, proceed #10 – Reason for not consenting.) 2. RESPONDENT’S PROFILE 2.1 Sex 1 2 Female Male 2.2 Ilang taon kana ngayon? 1 Specify age in years _____ 2.3 Ano ang iyong estado ngayon? Ikaw ba’y may asawa, hiwalay, biyudo/biyuda o single? 1 2 3 4 99 Married Separated Widow/Widower Single/Unmarried Others _________________ (please specify) 2.4 Ano ang iyong relihiyon? 0 1 None Christian (including Catholic, Protestant, Evangelical Christian, Seventh Day Adventist, Jehovah’s (MRP External Evaluation Annexes) Page 184 | 286 2 3 99 Witnesses, Iglesia ni Kristo and other Christian Denominations) Islam Buddhist Others _________________ (please specify) 2.5 Anong pinaka mataas mong naabot sa pag-aaral? 0 1 2 3 4 5 6 7 8 9 99 Hindi nakapag-aral Elementary level Elementary graduate High school level High school graduate College level College graduate Vocational graduate Master’s degree/Doctor’s degree level Master’s degree/Doctor’s degree graduate Others _________________ (please specify) 2.7 Nagtatrabaho ka ba ngayon para kumita? 1 0 Yes (If Yes, proceed to 2.7.1-2.7.3; but skip 2.8) No (If No, proceed to 2.8) 2.7.1 Ano ang uri ng iyong trabaho sa kasalukuyan? Permanente, hindi permanente, pansamantala? 1 2 3 4 Permanente Hindi permanente Part-time Self-employed 2.7.2 Anong klase ang trabaho mo ngayon? 1 2 3 4 Pagsasaka (gulay, prutas, etc) Pagsasaka (hayop, pangingisda, etc) Sari-sari store/Pagtitinda ng pagkain, ulam etc. Pagtitinda ng mga gamit (e.g., RTW, textile) MRP External Evaluation Annexes) Page 185 | 286 5 6 7 99 Pribadong kompanya Gobyerno NGO, POs Others _________________ (please specify) 2.7.3 Gaano ka na katagal sa trabaho o kabuhayan mo ngayon? Specify: ____ years ____ months _____ days 2.8 Bakit wala kang trabaho ngayon? 1 2 3 4 99 Walang mahanap na trabaho sa aming lugar Hindi ako kuwalipikado Hindi natatanggap kahit nag-aaplay Meron akong kapansanan o karamdaman Others _________________ (please specify) 2.9 Miyembro k aba ng isang grupo, asosasyon, o organisasyon sa kasalukuyan? 1 0 66 Yes (If yes, proceed to 2.9.1) No (If no, proceed to 3.1) Refused to answer 2.9.1 Anu-ano ang mga grupo na ito? (Puwede higit 1 ang sagot) 1 2 3 4 5 6 99 Community Solidarity Group Farmers’ Association Women’s Association Trade Union Mosque/Church/Religious Group Youth Group Others _________________ (please specify) 3. HOUSEHOLD PROFILE 3.1 Sino ba ang nangunguna sa mga desisyon na ginagawa ng inyong pamilya? 1 2 Tatay Nanay (MRP External Evaluation Annexes) Page 186 | 286 3 4 5 6 99 Pareho – Tatay at Nanay Uncle Aunt Lolo at Lola Others _________________ (please specify) 3.2 Ilang miyembro sa inyong tahanan ang 0-29 taong gulang? Specify the number: _______ 3.3 Ilan naman ang 30 taong gulang o mas matanda pa? Specify the number: _______ 3.4 Ilang taon na kayo dito sa bahay niyo ngayon? Specify the number of years: _______ 3.5 Sino ang nagmamay-ari ng bahay na tinitirhan niyo ngayon? 1 2 3 4 5 6 7 99 Kami ang may-ari. (Proceed to 4.2.1) Inuupahan namin ang bahay/kuwarto. (Proceed to 4.1). Kami ang tagapag-alaga. Tinitirahan naming ng libre. Isa kami sa may-ari. Nakitira kami kasama ng may-ari. Ibinigay ng libre ng NGO, government, at iba pa Others, please specify: ________________ 4. HOUSEHOLD EXPENDITURES AND DEBT 4.1 Magkano ang upa/renta niyo sa bahay bawat buwan? Specify the amount. PhP__________________________ 4.2.1 Noong nakalipas an buwan, ano ang pangunahing ginastusan ng iyong pamilya? 1 2 Drop down options: Pagkain Non-food items (clothing, hygiene kits, etc) MRP External Evaluation Annexes) Page 187 | 286 3 4 5 6 7 8 99 Gamot o bayad sa hospital Bayad sa loan Pag-aaral ng mga anak Monthly bills (ex. Electricity, water) Pagpaa-ayos ng bahay Upa/renta ng bahay Others 4.2.2 Noong nakalipas an buwan, ano ang pangalawang ginastusan ng iyong pamilya? 1 2 3 4 5 6 7 8 99 Drop down options: Pagkain Non-food items (clothing, hygiene kits, etc) Gamot o bayad sa hospital Bayad sa loan Pag-aaral ng mga anak Monthly bills (ex. Electricity, water) Pagpapa-ayos ng bahay Upa/renta ng bahay Others 4.2.3 Noong nakalipas an buwan, ano ang pangatlong ginastusan ng iyong pamilya? 1 2 3 4 5 6 7 8 Drop down options: Pagkain Non-food items (clothing, hygiene kits, etc) Gamot o bayad sa hospital Bayad sa loan Pag-aaral ng mga anak Monthly bills (ex. Electricity, water) Pagpapa-ayos ng bahay Upa/renta ng bahay (MRP External Evaluation Annexes) Page 188 | 286 99 Others 4.3 Magkano naman ang nagastos niyo sa __________? (Pangunahing ginastusan) Write actual amount: _____________________ 4.4 Magkano naman ang nagastos niyo sa __________? (Pangalawang ginastusan) Write actual amount: _____________________ 4.5 Magkano naman ang nagastos niyo sa __________? (Pangatlong ginastusan) Write actual amount: _____________________ 4.6 Meron ka ba o kahit sinong miyembro ng inyong tahanan ang may utang o loan sa ngayon? 1 0 55 66 Yes No (If no, proceed to 5.1) Hindi alam Hindi sumagot 4.7 Kung Meron, saan galling ang utang o loan? (Puwede kang sumagot ng higit sa 1.) 1 2 3 4 5 55 66 99 Kaibigan o kapit-bahay Kamag-anak Tagapagpahiram ng pera Banko/lending (mga pormal na institusyon) Traders Hindi alam Hindi sumagot Others _________________ (please specify) 4.8 Ano naman ang rason kung bakit kayo umutang o nag-loan? (Puwedeng higit 1 ang sagot) 1 2 3 4 5 6 7 Pambili ng pagkain Pambili ng gamot o pambayad sa ospital Pambayad ng gastos sa pag-aaral Puhunan sa Negosyo Pambayad ng loan/utang Pambayad ng monthly bills (water, electric, etc.) Pambayad ng upa/renta ng bahay MRP External Evaluation Annexes) Page 189 | 286 55 66 99 Di alam Hindi sumagot Others _________________ (please specify) 5. HOUSEHOLD INCOME 5.1 Ano ang mga pinagkakakitaan ng mga miyembro ng inyong pamilya sa kasalukuyan? (Puwedeng higit 1 ang sagot) 1 2 3 4 5 6 7 8 99 Farming crops Farming livestock/fishing Sari-sari store/food retailing Non-food retailing (e.g., RTW, textile) Paid work (including wages and pay for services/labor) Donor assistance (e.g., from an aid agency, a religious institution) 4Ps Remittance Others _________________ (please specify) 5.2 May ipon ka na ba ngayon? 1 0 Yes No (If “No,” proceed to 5.3) 5.2.1 Paano kayo nag-iipon 1 2 3 4 99 Alkansiya Banko Kooperatiba Community Savings Group (ex. Paluwagan) Others _________________ (please specify) 5.3 Meron bang miyembro ng inyong tahanan ang sumali ng skills training na ginawa ng ECOWEB/MARADECA/MRP? 1 0 55 66 Yes No Hindi alam Hindi sumagot (MRP External Evaluation Annexes) Page 190 | 286 5.4 Meron bang miyembro ng inyong tahanan ang sumali ng business development o financial literacy training na ginawa ng ECOWEB/MARADECA/MRP? 1 0 55 66 Yes No Hindi alam Hindi sumagot 6. PERCEIVED SELF-RELIANCE 6.1 Meron akong sapat na Kakayahan o kasanayan upang makahanap ng trabaho. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.2 May sapat na opportonidad para maka pagtrabaho dito sa aming lugar. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.3 Kung mawalan ako ng trabaho bukas, ako’y naniniwala na makakahanap ako ng trabaho sa loob ng isangbuwan. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.4 Kung mawalan ako ng kabuhayan bukas, ako’y 1 2 Lubos na hindi sumasang-ayon Hindi sumasang-ayon MRP External Evaluation Annexes) Page 191 | 286 naniniwala namakakapagsimula ulit sa loob ng isang buwan. 3 4 55 66 Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.5 Ang aking kita mula sa trabaho o pangkabuhayan ay sapat para matugonan ang pangangailangan ng aking pamilya. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.6 Meron akong mga ari-arian na puwedeng maibenta para sa pangangailangan ng pamilya kung kulang ang aking kita. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.7 Kung kukulangin ang aking kitameron akong sapat na ipon para sa pangangailangan ng pamilya. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.8 Madali akong makakahiram Ng pera sa aking mga kamag￾Anak at kaibigan para sa aming pangangailangan kung kulang 1 2 3 4 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon (MRP External Evaluation Annexes) Page 192 | 286 ang aking kita. 55 66 Hindi alam Hindi sumagot 6.9 Kung kulang ang aking kita, sapat naman ang pera galing sa mga kamag- anak sa labas ng bansa para sa aming pangangailangan. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 6.10 Madali lang akong nakaka hanap ng mga skills training (ex. TESDA) dito sa aming barangay. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7. PERCEIVED LEVEL OF POLARIZATION 7.1 Karamihan sa mga tao dito sa aming komunidad ay handang tumulong kung iyong papaki￾usapan. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.2 Karamihan sa mga tao dito sa amin ay mapagkakatiwalaan. 1 2 3 4 55 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam MRP External Evaluation Annexes) Page 193 | 286 66 Hindi sumagot 7.3 Karamihan sa mga tao dito sa amin ay honest o tapat. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.4 Hindi dapat basta-basta magtitiwala sa ibang tao. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.5 Pakiramdam ko ako’y bahagi o kabilang dito sa aming komunidad o kapitbahayan. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.6 Madalas umiiral ang pagkaka￾ibang mga taong naninirahan saisang kapitbahayan. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.7 Ang pagkakaiba ng tribu 1 Lubos na hindi sumasang-ayon (MRP External Evaluation Annexes) Page 194 | 286 ng mga tao sa aming lugar ay nagiging rason ng pagkakahiwalay. 2 3 4 55 66 Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.8 Ang pagkakaiba ng relihiyon ng mga tao sa aming lugar ay nagiging rason ng pagkakahiwalay. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot PERCEIVED LEVEL OF TRUST 7.9 Nagtitiwala ako sa mga opisyal ng mga grupo at organisasyon (katulad ng CSG) dito sa aming lugar. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.10 Nagtitiwala ako sa mga kasamahan ko sa grupo at organisasyon (katulad ng CSG) dito sa aming lugar. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.11 Nagtitiwala ako sa mga opisyal ng barangay kung saan 1 2 Lubos na hindi sumasang-ayon Hindi sumasang-ayon MRP External Evaluation Annexes) Page 195 | 286 ako nakatira ngayon. 3 4 55 66 Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.12 Nagtitiwala ako sa mga lider ng relihiyon sa barangay kung saan ako nakatira ngayon. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.13 Nagtitiwala ako sa mga non-governmental organizations o NGOs. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.14 Nagtitiwala ako sa pulis. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 7.15 Nagtitiwala ako sa military/ sundalo. 1 2 3 4 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon (MRP External Evaluation Annexes) Page 196 | 286 55 66 Hindi alam Hindi sumagot 7.16 Handa akong tumulong sa mga IDPs at HCMs para maisagawa ang mga proyekto. 1 2 3 4 55 66 Lubos na hindi sumasang-ayon Hindi sumasang-ayon Sumasang-ayon Lubos na sumasang-ayon Hindi alam Hindi sumagot 8. PERCEIVED PUBLIC REPRESENTATION 8.1 Ako ay kinonsulta at sinama para maisagawa ang mga proyekto para sa komunidad (ex.water system, security) 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.2 Ang mga karaniwang tao at miyembro ng mga organisasyon sa aming lugar ay nakikilahok sa mga desisyon ngkomunidad o ng CSG. 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.3 Ang mga kabataan sa aming lugar ay nakikilahok sa mga desisyong ginagawa sa komunidad o CSG. 1 2 3 4 55 Hindi kailanman Minsan Madalas Napakadalas Hindi alam MRP External Evaluation Annexes) Page 197 | 286 66 Hindi sumagot 8.4 Ang mga kababaihan sa aming lugar ay nakikilahok sa mga desisyong ginagawa sa komunidad o ng grupo kung saan miyembro (ex. CSG) 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.5 Ang aking mga katribu ay nakikilaahok sa mga desisyong ginagawa sa komunidad o ng grupo kung saan miyembro (ex. CSG). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.6 Ang mga kapareha ko ng relihiyon ay nakikilaahok sa mga desisyong ginagawa sa komunidad o ng grupo kung saan miyembro sila (ex. CSG). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.7 Sumasali ako sa mga meetings sa komunidad o ng grupo kung saan ako miyembro (ex. CSG). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.8 Kina-usap ko ng personal 1 Hindi kailanman (MRP External Evaluation Annexes) Page 198 | 286 ang CSG lider namin o ang BLGUtungkol sa isang problema sa aming komunidad. 2 3 4 55 66 Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.9 Tumatawag ako sa lider ng aming grupo, asosasyon o BLGU tungkol sa problema sa aming komunidad. 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.10 Sumusulat ako sa lider ng aming grupo, asosasyon o BLGU tungkol sa problema sa aming komunidad. 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.11 Bumuto ako noong eleksyon ng opisyales sa aming grupo o asosasyon (ex. CSG). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.12 Bumuto ako noong nag-eleksyon sa barangay dito sa aming lugarngayon. 1 2 3 Hindi kailanman Minsan Madalas MRP External Evaluation Annexes) Page 199 | 286 4 55 66 Napakadalas Hindi alam Hindi sumagot 8.13 Nakikilahok ako sa pagsasagawa ng mga proyekto at Gawain ng grupo o asosasyon kung saan ako miyembro (ex. CSG). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.14 Nakikilahok ako para himukin ang gobyerno at ibang organisasyon na maglaan ng pondo para sa mga proyekto sa aming komunidad (ex. water system, security). 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.15 Ako ay aktibong miyembro ng mga asosasyon dito sa aming lugar. (ex. CSG, Women’s, Youth) 1 2 3 4 55 66 Hindi kailanman Minsan Madalas Napakadalas Hindi alam Hindi sumagot 8.16 Sumasali ako sa mga trainings na ginagawa dito sa amin. (ex. Skills, grants/loans, mgt.) 1 2 3 4 55 Hindi kailanman Minsan Madalas Napakadalas Hindi alam (MRP External Evaluation Annexes) Page 200 | 286 66 Hindi sumagot 9. DURABLE SOLUTIONS 9.1 Ikaw ba naging IDP dahil sa Marawi seige? 1 0 Yes No (Proceed to Time End) 9.2 Ano ang iyong hinahangad o ng iyong pamilya sa mga darating na buwan o taon? 1 2 Ako/kami ay makabalik na sa Marawi. (If 1, proceed to 9.3 to 9.5) Ako/kami ay mananatili na dito sa lugar kung saan kami ngayon.(If 2, proceed to 9.6 to 9.7) 9.3 Kung balak mong bumalik sa Marawi, ano ang iyong balak gawin doon? 1 2 3 Doon na manirahan, maghanapbuhay o magtrabaho Doon lang magtrabaho o maghanapbuhay pero hindi doon manirahan Doon lang maninirahan pero hindi doon magtrabaho o maghanapbuhay 9.4 Ano ang dahilan ng iyong pagdesisyon? 1 2 3 4 5 Pangmatagalang kaligtasan at seguridad Mayroong kabuhayan at trabaho Nandoon ang aming ari-arian tulad ng bahay, lupa at iba pa Ito ang tahanan ng aming angkan/lahi na aming kinagisnan. Pagsasama-sama ng pamilya at mga malalapit na kaibigan. Mas malayang makikilahok sa mga gawain sa komunidad Others _________________ (specify) 9.5 Gaano ka katiyak na ang iyong plano/opsyon ay tama? 1 2 Napaka sigurado Medyo sigurado MRP External Evaluation Annexes) Page 201 | 286 3 4 66 Hindi sigurado Hinding-hindi sigurado Hindi ko alam 9.6 Ano ang iyong plano kung hindi ka babalik sa Marawi? 1 2 3 4 5 Wala pa akong malinaw na pagpipilian. Mananatili ako sa host community para sa aming permanenteng paninirahan at para sa trabaho (trabaho/negosyo/kabuhayan) Pupunta ako sa mga kalapit na munisipyo/lungsod dito sa Mindanao. Pupunta ako sa ibang lugar na nasaVisayas o Luzon. Pupunta ako sa ibang bansa. 9.7 Gaano ka katiyak na ang iyong plano/opsyon ay tama? 1 2 3 4 66 Napaka sigurado Medyo sigurado Hindi sigurado Hinding-hindi sigurado Hindi ko alam 9.8 Bakit hindi ka pa nakapagdesisyon? 1 2 3 Wala pa akong sapat na impormasyon tungkol sa mangyayari sa akin at sa aking pamilya sa hinaharap. Mayroon akong impormasyon ngunit kailangan ko pa ring malaman ang tungkol dito sa iba. Hindi ko alam 9.9 Reason for not consenting 1 2 3 Hindi naka tanggap ng MRP support Nakatanggap pero hindi kuntento Walang panahon para sa interview (MRP External Evaluation Annexes) Page 202 | 286 4v Others (Please specify) _____________ MRP External Evaluation Annexes) Page 203 | 286 Annex 8 FGD AND KII TOOLS Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 204 | 286 Annex 8 FGD and KII Tools Tool A: HCM, IDP, Youth, Farmers, and Women Leaders KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive from MRP? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. When you compare your economic condition before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your economic condition that MRP has contributed to? (Note: contextualize based on intervention received) i. Probe: How did the livelihood grants strengthen the bonds between HCMs and IDPs in your community? 2. When you look at the relationship between HCMs and IDPs in your community before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in the social cohesion in your community that MRP has contributed to? i. What types of conflicts arose and how did you resolve them? ii. How did the livelihood and social cohesion grants strengthen the bonds between HCMs and IDPs in your community? 3. When you compare your participation in community affairs and activities before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your participation in community affairs and activities that MRP has contributed to? MRP External Evaluation Annexes) Page 205 | 286 i. Note to facilitator: Expand to cover both civic and business activities. 4. When you compare your ability to meet your personal and family needs in a sustainable manner before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your ability to meet the needs of your family in a sustainable manner that MRP has contributed to? 5. What changes, positive/negative or intended/unintended, happened as a result of the grant or training that you received from MRP? 6. What do you think are the factors that contributed to the achievement or non-achievement of the objectives you set at the start? a. What are your views about the manner the assistance was given? i. Note to facilitator: Related to DNH QUESTIONS ON RELEVANCE 1. How appropriate is the MRP intervention that you received in addressing your needs: a. Right after the Marawi siege happened? b. In the last 3 years? c. At present? 2. What is your plan in the near future: a. Will you go back to Marawi? b. Will stay in the community where you are now? c. Will you go to a different place? 3. Has MRP contributed to your decision about the future? a. If yes, can you elaborate how MRP has influenced your decision? b. If no, can you elaborate why? QUESTIONS ON SUSTAINABILITY 1. With the exit of MRP, how can your organization protect and sustain the gains you achieved over the years? a. Probe: What factors do you think will help sustain the intervention that you received? LESSONS AND RECOMMENDATIONS (MRP External Evaluation Annexes) Page 206 | 286 1. What were the key challenges that you experienced in MRP project implementation? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the: a. Economic condition of IDPs/HCMs? b. Relationship between IDPs/HCMs? 4. If the MRP were implemented again, what would be done to improve it? MRP External Evaluation Annexes) Page 207 | 286 TOOL B. CSG Federation KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive from MRP? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. When you compare the economic condition of the members of your sector before they became beneficiaries of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in their economic condition that MRP has contributed to? i. Probe: How did the livelihood grants strengthen the bonds between HCMs and IDPs in your sector? c. Note: Contextualize based on intervention received 2. When you look at the relationship between HCMs and IDPs in your sector before they became beneficiaries of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in the social cohesion in your sector that MRP has contributed? i. Probe: What types of conflicts arose and how did you resolve them? ii. Probe: How did the livelihood and social cohesion grants strengthen the bonds between HCMs and IDPs in your sector? 3. When you compare the participation in community affairs and activities of the members of your sector before they became beneficiaries of MRP and at present: (MRP External Evaluation Annexes) Page 208 | 286 a. What has improved? Why? b. What do you think is the most significant change in their participation in community affairs and activities that MRP has contributed to? (Note to facilitator: Expand to cover both civic and business activities) c. As a federation what is your priority advocacy agenda? d. In what special bodies have you been representing? e. What have been the results in your public representations so far? 4. What are your observations regarding the ability of IDPs/HCMs in your sector to meet their personal and family needs in a sustainable manner? QUESTIONS ON RELEVANCE 1. Based on your personal knowledge, what do you think are the plans of the members in your sector: a. Will they go back to Marawi? b. Will stay in the community where they are now? c. Will they go a different place? 2. Has MRP contributed to their decision about the future? a. If yes, can you elaborate how MRP has influenced their decision? b. If no, can you elaborate why? QUESTIONS ON SUSTAINABILITY 1. With the exit of MRP, how can your sector protect and sustain the gains you achieved over the years? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the: a. Economic condition of IDPs/HCMs? b. Relationship between IDPs/HCMs? 4. If the MRP were implemented again, what would be done to improve it? MRP External Evaluation Annexes) Page 209 | 286 TOOL C: Dissolved Community Solidarity Groups KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive from MRP? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. How has the dissolution of your CSG affected the economic condition among the HCMs and IDPs in your community? 2. How has the dissolution of your CSG affected the relationship between the HCMs and IDPs in your community? 3. How has the dissolution of your CSG affected the participation of the HCMs and IDPs in the affairs and activities in your community? 4. How has the dissolution of your CSG affected the ability of the HCMs and IDPs to meet their personal and family needs in a sustainable manner? QUESTIONS ON RELEVANCE 1. What is your plan in the near future: a. Will you go back to Marawi? b. Will stay in the community where you are now? c. Will you go a different place? (MRP External Evaluation Annexes) Page 210 | 286 2. Has MRP contributed to your decision about the future? a. If yes, can you elaborate how MRP has influenced your decision? b. If not, can you elaborate why? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? a. What are the key factors that resulted in your dissolution? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the: a. Economic condition of IDPs/HCMs? 4. Relationship between IDPs/HCMs? 5. If the MRP were implemented again, what would be done to improve it? MRP External Evaluation Annexes) Page 211 | 286 TOOL D - Business Recovery Grantees KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive from MRP? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. When you compare your economic condition before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your economic condition that MRP has contributed to? (Note: contextualize based on intervention received) i. Probe: How did the livelihood grants strengthen the bonds between HCMs and IDPs in your community? 2. When you compare your ability to meet your personal and family needs in a sustainable manner before you became a beneficiary of MRP and at present: c. Can you say that it has improved? Why? d. What do you think is the most significant change in your ability to meet the needs of your family is a sustainable manner that MRP has contributed? (MRP External Evaluation Annexes) Page 212 | 286 QUESTIONS ON RELEVANCE 1. What is your plan in the near future: a. Will you go back to Marawi? b. Will stay in the community where you are now? c. Will you go to a different place? 2. Has MRP contributed to your decision about the future? a. If yes, can you elaborate how MRP has influenced your decision? b. If no, can you elaborate why? QUESTIONS ON SUSTAINABILITY 1. With the exit of MRP, how can your sector protect and sustain the gains you achieved over the years? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the economic condition of SMEs? 4. If the MRP were implemented again, what would be done to improve it? MRP External Evaluation Annexes) Page 213 | 286 TOOL F : Workforce Development Completers KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive from MRP? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. When you compare your economic condition before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your economic condition that MRP has contributed to? i. Probe: Did your training help to improve your employment or earning opportunities? 2. When you look at the relationship between HCMs and IDPs in your community before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in the social cohesion in your community that MRP has contributed to? 3. When you compare your participation in community affairs and activities before you became a beneficiary of MRP and at present: a. What has improved? Why? (MRP External Evaluation Annexes) Page 214 | 286 b. What do you think is the most significant change in your participation in community affairs and activities that MRP has contributed to? 4. When you compare your ability to meet your personal and family needs in a sustainable manner before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your ability to meet the needs of your family in a sustainable manner that MRP has contributed to? QUESTIONS ON SUSTAINABILITY 1. With the exit of MRP, how can your sector protect and sustain the gains you achieved over the years? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the economic condition of SMEs? 4. If the MRP were implemented again, what would be done to improve it? MRP External Evaluation Annexes) Page 215 | 286 TOOL G : LGU Chief Executives and BLGU Officials KII with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. In what ways have you been involved in MRP? 2. What was your role in the project? 3. What has been the contribution of the municipality or barangay in the implementation of the project? QUESTIONS ON EFFECTIVENESS 1. What are your observations regarding the economic condition among the IDPs/HCMs because of the MRP? 2. What are your observations regarding the relationships between the IDPs/HCMs because of the MRP? 3. What are your observations regarding the participation of the IDPs/HCMs in community affairs and activities because of the MRP? 4. What are your observations regarding the ability of IDPs/HCMs to meet their personal and family needs in a sustainable manner? QUESTIONS ON RELEVANCE (MRP External Evaluation Annexes) Page 216 | 286 1. What can you say about the relevance of the MRP to existing government priorities in responding to the Marawi crisis? QUESTIONS ON SUSTAINABILITY 1. What has been done by MRP so far to ensure that the interventions received by the IDPs and HCMs in your area would be sustained? 2. With the exit of MRP, how can the LGU help protect and sustain the gains that the IDPs/HCMs achieved over the years? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation in your area? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the: a. Economic condition of IDPs/HCMs? b. Relationship between IDPs/HCMs? MRP External Evaluation Annexes) Page 217 | 286 TOOL H : Community Solidarity Groups FGD with Location Time Start Time End Date Venue Facilitator Asst. Facilitator WARM-UP QUESTIONS 1. How did you become a beneficiary of MRP? 2. What intervention(s) did you receive? 3. What is its status at present? QUESTIONS ON EFFECTIVENESS 1. When you compare your economic condition before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your economic condition that MRP has contributed to? (Note: contextualize based on intervention received) i. Probe: How did the livelihood grants strengthen the bonds between HCMs and IDPs in your community? 2. When you look at the relationship between HCMs and IDPs in your community before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in the social cohesion in your community that MRP has contributed to? i. What types of conflicts arose and how did you resolve them? ii. How did the livelihood and social cohesion grants strengthen the bonds between HCMs and IDPs in your community? (MRP External Evaluation Annexes) Page 218 | 286 3. When you compare your participation in community affairs and activities before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your participation in community affairs and activities that MRP has contributed to? i. Note to facilitator: Expand to cover both civic and business activities. 4. When you compare your ability to meet your personal and family needs in a sustainable manner before you became a beneficiary of MRP and at present: a. What has improved? Why? b. What do you think is the most significant change in your ability to meet the needs of your family in a sustainable manner that MRP has contributed to? 5. What changes, positive/negative or intended/unintended, happened because of the grant or training that you received from MRP? 6. What do you think are the factors that contributed to the achievement or non￾achievement of the objectives you set at the start? a. What are your views about the manner the assistance was given? i. Note to facilitator: Related to DNH QUESTIONS ON RELEVANCE 1. What were/are your most essential needs that MRP addressed: a. Right after the Marawi siege happened? b. In the last 3 years? c. At present? 2. How appropriate is the MRP intervention that you received in addressing your needs: d. Right after the Marawi siege happened? e. In the last 3 years? f. At present? 3. What is your plan in the near future: a. Will you go back to Marawi? b. Will stay in the community where you are now? c. Will you go to a different place? 4. Has MRP contributed to your decision about the future? a. If yes, can you elaborate how MRP has influenced your decision? b. If no, can you elaborate why? MRP External Evaluation Annexes) Page 219 | 286 QUESTIONS ON SUSTAINABILITY 1. What has been done by MRP so far to ensure that the interventions that you receive would be sustained? a. Note to facilitator: Consider value chain approach, federation of CSGs, representation in special bodies, generation of LGU commitments. 2. What have you done to sustain the MRP interventions you received? 3. With the exit of MRP, how can your organization protect and sustain the gains you achieved over the years? a. Probe: What factors do you think will help sustain the intervention that you received? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? a. How did you resolve the challenges? 2. What key lessons have you learned as part of the MRP project? 3. What other suggestions could you share to improve the: a. Economic condition of IDPs/HCMs? 4. Relationship between IDPs/HCMs? 5. If the MRP were implemented again, what would be done to improve it? (MRP External Evaluation Annexes) Page 220 | 286 TOOL I: PLAN International WARM-UP QUESTIONS 1. How do you feel about being part of the MRP Project? 2. When you look at the overall project results, what are the key results that you are most proud of? QUESTIONS ON EFFECTIVENESS 1. Based on your observations, what are the manifestations of change in the lives of MRP beneficiaries in the following areas that you can attribute to the project? a. Economic condition b. Relationship between IDPs and HCMs c. Public participation d. Self-reliance * Note to facilitator: Relate responses to the indicator definitions. 2. What have been your strategies to ensure that the interventions to improve the economic conditions (IR1) and interventions to improve the social cohesion (IR2) of beneficiaries are closely linked and are mutually reinforcing as suggested in MRP’s TOC? a. How intentional were you in ensuring that the interventions under IR1 and IR2 reinforce each other? b. To what extent have you implemented your planned strategies to ensure adherence to the results of the co-creation process? c. What have been the deviations and why? 3. Looking at your project strategy, what have been the essential features or elements that made significant contributions in achieving the project outcomes? a. What do you think are the facilitating or hindering factors that contributed to the achievement or non-achievement of the outcomes you set at the start? b. How has the funding dynamics or the colors of money influenced the achievement of project outcomes? c. There are notable peaks and downturns in the delivery BRGs and CMGs across the project life based on the databases you shared. What do you think are significant factors that influence this phenomenon? MRP External Evaluation Annexes) Page 221 | 286 d. There has been noted staff turnover over the life of the project. How has this influenced the achievement of the project outcomes? 4. What have been the feedbacks of the implementing partners in relation to how satisfied they are in the delivery of the project? a. How did you address the feedback? Note: Relate discussion to the issue of delay. b. What systems that you implemented helped? What systems did not? c. Probe: Documentation of monthly meetings with IPs 5. What unintended outcomes has the project generated? a. What do you think are the reasons that triggered these outcomes? QUESTIONS ON RELEVANCE 1. Based on your involvement in the project, how was it ensured that the interventions were relevant to the evolving needs and contexts of the beneficiaries? a. There are noted grants given to a group of CSGs. What were the bases for the management decisions in providing grants to a single CSG or a group of CSGs? b. Have you observed intervention-needs mismatch? How was it addressed? 2. How do you construe adaptive management? How was this exercised in the implementation of the project? a. During the height of the COVID-19 pandemic, what strategies were employed to ensure that MRP addressed the essential needs of the beneficiaries? 3. In what ways do you think has MRP influenced the durable solutions decisions of the IDPs? a. The Durable Solution Study that Plan conducted suggests that the settle preference of MRP beneficiaries is different from those in the control group. Local integration is higher among the control group (77%) than the MRP beneficiaries (70%). Return to Marawi is higher for the MRP beneficiaries (30%) than the control group (22%). With these results, in what ways has MRP influenced the durable solution preference of the IDPs who are MRP beneficiaries? QUESTIONS ON SUSTAINABILITY (MRP External Evaluation Annexes) Page 222 | 286 1. What has been done by MRP so far to ensure that the interventions the beneficiaries received would be sustained? 2. With the exit of MRP in the project areas, how do you think the gains of the project can be sustained? a. What factors do you consider are essential to sustain the interventions and gains of the project? 3. There are notable sustainability measures that are being implemented. What were the considerations in implementing the sustainability interventions? LESSONS AND RECOMMENDATIONS 1. What were the key challenges that you experienced in MRP project implementation? a. How did you resolve the challenges? 2. What key lessons have you learned as part of the MRP project? 3. If the MRP project were implemented again, what would be done differently to improve it? MRP External Evaluation Annexes) Page 223 | 286 TOOL J : USAID 1. How has the co-design process contributed to the relevance, effectiveness, and sustainability of the project? 2. Were MRP’s development interventions relevant and adaptive to Mindanao ‘s complex and changing external environment, especially in the project areas? 3. Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic activities among IDPs and HCMs? 4. Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs? (MRP External Evaluation Annexes) Page 224 | 286 Annex 9 INFORMED CONSENT FORM Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 225 | 286 Annex 9 INFORMED CONSENT FORM 1. Rationale and Purpose of the Evaluation The Marawi Response Project (MRP) is part of USAID’s development assistance that supports the reintegration of IDPs by addressing the early recovery needs of displaced individuals, and, more importantly, by addressing the long-term rehabilitation need of affected communities and the local governments in and around Marawi. Currently, we are evaluating the project. The purpose of the evaluation is to evaluate MRP’s performance in accomplishing its target outcomes. We will also examine its relevance, effectiveness and sustainability. 2. Participant Involvement You are selected because you are considered a valuable stakeholder of the project. For this reason, the team hopes that you could spend time to share to your perceptions, thoughts and views regarding the MRP. In particular, you will be involved by sharing your answers to the focus group discussion (FGD)/key informant interview (KII) questions. You will participate in the FGD/KII for about 1 hour to 1½ hours. 3. Potential Risks, Discomforts, Discontinuance There are no risks associated to your participation. However, if you feel uncomfortable in answering some questions, you could decide not to answer them or decide not to continue with your participation. You can decide to do this without any consequence or penalty. 4. Potential Benefits If you will participate, the information that you will share will help the team come up with valuable insights regarding the project. These insights could help the USAID improve the way it facilitates projects to help communities such as those that are in your area. 5. Confidentiality All efforts will be made to ensure that your identity will be anonymous during the evaluation. The information that you will share during the FGD/KII will be treated confidential. This will be done by (MRP External Evaluation Annexes) Page 226 | 286 coding your responses. No name or other identifying information will be used when discussing or reporting data. 6. Cost, Compensation, Reimbursement There is no cost for you to participate in this endeavor. Moreover, you will not be compensated for your participation. In case, you will spend an amount for your fare to go to a certain venue, you will be refunded the amount that you spent. 7. Voluntary Participation and Authorization Your decision to participate in this study is completely voluntary. If you decide to not participate in this study, it will not affect the services, or benefits of the project to which you are entitled. Consent Considering all the information given above, I voluntarily agree to participate in this evaluation: □ Yes □ No By affixing my signature below, I further authorize the use of my responses, and the video and audio recordings of the FGD/KII for purposes of learning, and/or presentation. Name of Participant (print): _______________________________________ Signature of Participant: _______________________________________ Date: ______________________ MRP External Evaluation Annexes) Page 227 | 286 Annex 10 ACTIVITIES THAT CONTRIBUTED TO MRP’S ADAPTABILITY AND RESPONSIVENESS Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 228 | 286 Annex 10 ACTIVITIES THAT CONTRIBUTED TO MRP’S ADAPTABILITY AND RESPONSIVENESS Activity Year Participants Sample outputs Pause and reflect session 2019 ECOWEB, Maradeca Refinement of the emerging impact and intermediate results of MRP Pause and reflect session 2019 Ecoweb, Maradeca Identification of key challenges related to the grants process; Grants team decided to establish common grant categories to help expedite grants review and processing Assessment of Out-of-School Youth (OSY) in BARMM 2020 BARMM Youth Youth’s perceptions on workforce opportunities and knowledge of existing civil society organization’s activities for OSYs; Data were used for programming, policy making and collaboration for youth development and empowerment; A sample collaboration was with the Hyundai Dream Center that trained youth scholars on automotive tech and servicing, customer communication, and work preparedness Orientation on new data collection system (From Souktel to Kobo) 2020 Ecoweb, Maradeca New data collection and storage system since after the bankruptcy of Souktel Pause and reflect session 2021 USAID, MRP senior managers, staff Identify challenges encountered; Validated learnings and successes to inform Year 4 implementation Conflict assessment study 2021 Ecoweb, Maradeca, Graduate students from Johns Hopkins University Recommendation that social cohesion activities to limit any further cleavages identified among different ethnic, religious, and socioeconomic groups MRP External Evaluation Annexes) Page 229 | 286 Activity Year Participants Sample outputs Rapid organizational capacity assessment (ROCA) 2021 193 CSGs Classification of the organizational capacities of CSGs as effective, functional and formative considering the areas of leadership and governance, program and project management, resource mobilization, finance and administration, external relations, collaboration and influencing. Policy Formulation and Advocacy Planning Learning Sessions 2022 118 CSGs Finalization of CSGs’ organizational vision, values, mission, and advocacy agenda; Sample policy instruments, such as resolutions and policy advocacy plans MLGU consultation meetings on project sustainability 2021- 2022 LGUs of Marantao, Baloi, Iligan City, Butig, Balindong, Pantao Ragat, Madalum, Tubaran, Pantar, Lumba Bayabao, Ditsaan Ramain, Poona Bayabao, Buadiposo Buntong, Tugaya Agreements with LGUs to sustain the project outcomes (MRP External Evaluation Annexes) Page 230 | 286 Annex 11 COMMUNITY ENGAGEMENT AND GRANT APPLICATION CYCLE Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 231 | 286 Annex 11 COMMUNITY ENGAGEMENT AND GRANT APPLICATION CYCLE66 66 As stipulated in MRP Grants Manual Community Grant Development Flowcharts Initial Community Entry Phases Below is the initial community entry process for both Quick Response and Standard CMGs: PHASE ACTIVITIES Entering the Barangay • Barangays identified through site selection process, CDFs mobilized; • CDFs orient barangay and municipal officials on MRP; • IDPs and host community and their leaders identified and contacted; • GA uploads documentation on FLUXX. • Formation of CSGs • • • • CDFs work with interested IDPs and host communities to form CSGs; CSGs choose their leaders, CDFs validate list of IDPs; USNO PA runs CSG member names through SAM, OFAC, UNSC; GAs upload vetting documentation on FLUXX. • Project Prioritization and Identification • • • • Partners conduct prioritization workshop, including project identification and prioritization. This process includes: CDF holds an FGD where priority projects are identified by the CSG as a group, with other community representatives; The CSG with the community representatives select from among the shortlisted projects, prioritizing from most to least desired among communities; CDF documents the prioritization process and the final list of prioritized projects, which is signed by the CSG leaders, CDF, CDM. GA uploads documentation to barangay card on FLUXX (MRP External Evaluation Annexes) Page 232 | 286 Standard CMG Micro-grants grant development process The figure below graphically represents the grant development process for Standard CMGs after the initial community entry phases. Beginning at the CN stage, the GA will send out a weekly internal grant update so all MRP team members are on the same page about the status of each grant. Additionally, the USNO Program Associate will randomly spot check grants on FLUXX throughout the grant cycle to ensure all required documentation is uploaded. PHASE ACTIVITIES Preparation of Concept Note Preparation of Grant Proposal Endorsement and Approval of Proposals Grant Kick-off • CSG, with assistance from CDF, drafts concept note(s). • Technical Leads review concept note and decide to endorse to full proposal for Standard CMGs or return for more information. • ECS determines if ESR is needed; if yes is checked, begins process. • GA receives the concept note and uploads to FLUXX. • GA and PDA prepare full grant proposal in FLUXX based on the concept note, ensuring full compliance. • GA uploads GA/PDA checklist; • GO reviews proposals and selects funding source(s); • GO sends to GM. • GM approves or approves with conditions for grants that are $5,000 and below and endorses grants that are above the $5,000 threshold to COP for approval or approval with conditions or for rejection. • GA or ECS uploads ESR to FLUXX, as required. • GO prepares IKA and submits to GM who reviews and clears. • GA or partners schedule(s) kick-off meeting with CSG to review, agree and sign the IKA along with other required certifications. • GA uploads IKA to FLUXX. • FLUXX status becomes “Granted-IKA uploaded” MRP External Evaluation Annexes) Page 233 | 286 Annex 12 COLLABORATIVE ACTIVITIES CONDUCTED BY MRP Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 234 | 286 Annex 12 COLLABORATIVE ACTIVITIES CONDUCTED BY MRP Agency Involved Nature of Collaboration Year Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) Minister for Social Services The Minister for Social Services allowed MRP to mobilize its Lanao del Sur municipal social welfare officers to assist IDPs in their respective jurisdictions. 2019 Task Force Bangon Marawi (TFBM) TFBM and DTI furnished MRP with the official list of businesses located in the MAA prior to the siege. The information greatly facilitated the processing of BRG proposals. 2019 Technical Education and Skills Development Agency (TESDA) TESDA provided MRP with list of recommended technical vocational institutes and assisted in providing technical training to livelihood grant recipients. 2019 Department of Labor and Employment (DOLE) DOLE assisted MRP during job fairs, advertising skills training, and employment programs to firms in Iligan, and helped in facilitating the approval of CSG applications for workers’ associations. 2019 Provincial Government of Lanao del Sur MRP worked with the PLGU of Lanao del Sur through its provincial agriculture office, its Technology and Livelihood Development Center (TLDC), provincial social welfare development office, and the cooperatives development office for the implementation of its programs in the province. 2019 Lanao and Regional Cluster of the Joint Child Protection and Gender￾Based Violence (JCPGBV) Working Group Through the JCPGBV, MRP can highlight the project’s gender integration efforts to BARMM and coordinate with other members of the working group for possible collaboration and complementation. MRP served as the Secretariat of the Lanao JCPGBV Working Group for 2019- 2020. 2020 Department of Trade and Industry (DTI) Lanao del Norte and ARMM offices, Bureau of Fisheries & Aquatic Resources (BFAR)- These agencies paved the way to the increase in the number of BRG referrals, as well as to the expansion of types of businesses that could be supported (including cooperatives) through BRG assistance. By collaborating with these agencies, 2020 MRP External Evaluation Annexes) Page 235 | 286 Agency Involved Nature of Collaboration Year ARMM, Provincial Agriculture and Provincial Technology & Livelihood Development Offices of Lanao del Sur MRP expanded its roster of BRG recipients to include agricultural cooperatives, women’s cooperatives (particularly those engaged in handweaving and food processing), and agri￾business enterprises. Task Force Bangon Marawi (TFBM)-Department of Trade and Industry (DTI) Assisted in the training of 150 CSG members from the remote Barangays of Poctan and Ragayan, Butig, Lanao del Sur on Basic Dressmaking, Basic Masonry, and Basic Carpentry. Under this collaboration, MRP provided the technical skills training while DTI-TFBM provided start-up toolkits. 2020 Philippine Disaster Resilience Foundation (PDRF), MSU-IIT-iDEYA, Department of Science and Technology, DTI, Sangguniang Kabataan Provincial Federation of Lanao del Sur Served as resources persons during the hands-on training, mentoring and coaching of 64 youth CSG leaders that facilitated youth-led design thinking and project development in preparation for the Youth Innovate Summit. 2020 M/BLGU) representatives, and religious leaders from the municipalities of Balindong and Lumba Bayabao in LDS, and Pantar, LDN Inclusion of municipal and barangay LGU representatives, and religious leaders in the TOT for community engagement and the CSG process in its workforce readiness interventions due to the rising demand for skills in community development facilitation. 2021 Bangsamoro Youth Commission (BYC) The BYC conducted a TOT on Voter’s Education and Youth Engagement to roll-out community￾level voter’s education and youth visioning workshops. 2021 Cooperative Development Authority (CDA), Provincial Cooperative Development Office, Municipal Planning and Development Office and Municipal Business Permit and Licensing Office These agencies served as resource persons during CSG Entrepreneurial Skills Training, Mentoring and Sustainability Planning, which led to concerned government offices commitment of support such as some free business permit applications for Year 1 as an incentive offered by some LGUs. 2021 Department of Interior and Local Government, Bangsamoro Autonomous Region of Muslim Mindanao The DILG and BARMM Commission on Elections served as resource persons during the MRP￾YouthLED Voter’s Education Training of Trainers. 2021 (MRP External Evaluation Annexes) Page 236 | 286 Agency Involved Nature of Collaboration Year (BARMM) Commission on Elections Commission on Elections The COMELEC served as resource person during Community Voter’s Education. The agency also assisted in the Youth Engagement and Special Registration, which led to the registration of 242 new voters and transferees. 2021 MRP External Evaluation Annexes) Page 237 | 286 Annex 13 MRP PERFORMANCE IN RELATION TO THE PROJECT INDICATORS Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 238 | 286 Annex 13 MRP PERFORMANCE IN RELATION TO THE PROJECT INDICATORS Annual Targets67 Y1 Y2 Y3 Y4 Intermediate Result 1: Economic conditions of IDPs and host communities improved. 2 Number of displaced business owners with new or restarted businesses No target set 3 Percent of trained displaced/HCMs gainfully employed - - 20% 25% Output 1.1: Business recovery opportunities expanded 4 Number of displaced business owners who benefitted from business recovery micro-grants 200 400 258 28 5 Number of displaced business owners trained in enterprise management 200 320 307 400 Output 1.2: Employment opportunities expanded 6 Percent of displaced/host community members trained in workforce readiness and life skills No target in % set* 80% 90% 90% Output 1.3: Livelihood opportunities expanded 7 Number of displaced/HCMs who benefited from livelihood micro- grants 800 1,260 2,550 3,541 Output 2.1: Social cohesion grants to host and displaced communities 10 Number of displaced/HCMs who benefited from social cohesion grants 110 29,457 28,792 106,659 67 Based on MRP MERL Validation dated September 22, 2022 MRP External Evaluation Annexes) Page 239 | 286 Output 2.2: Social cohesion trainings 11 Number of IDP/HCM trained on civic engagement 330 3,850 2,995 1,295 *target set was number of completers and not percentage of training completers (MRP External Evaluation Annexes) Page 240 | 286 MRP Performance68 No. INDICATORS Baselin e Life of Project Accomplishment Target Y1 Y2 Y3 Y4 Accom plishm ent (Oct 2021 - Mar 2022) Y4 (Oct 2021 - Jun 2022) 1 Manageable Impact: Self-reliance of internally displaced persons (IDPs) and host community members (HCMs) who were affected by the Marawi siege, improved. 2.63 End of Project Measure￾ment - - - - - Intermediate Result 1: Economic conditions of IDPs and host communities improved. 2 Number of displaced business owners with new or restarted businesses 0 - 12 214 143 Report￾ed Annually Report￾ed Annually 3 Percent of trained displaced/HCMs gainfully employed 0 refer to Annual Targets sheet 15% 17% 27% Report￾ed Annually Report￾ed Annually Output 1.1: Business recovery opportunities expanded 4 Number of displaced business owners who benefitted from business recovery micro-grants 0 refer to Annual Targets sheet 12 308 326 28 28 68 Based on MRP MERL Validation dated September 22, 2022 MRP External Evaluation Annexes) Page 241 | 286 5 Number of displaced business owners trained in enterprise management 0 refer to Annual Targets sheet 226 265 611 234 414 Output 1.2: Employment opportunities expanded 6 Percent of displaced/host community members trained in workforce readiness and life skills 0 refer to Annual Targets sheet 99% 97% 99% 99% 99% Output 1.3: Livelihood opportunities expanded 7 Number of displaced/HCMs who benefited from livelihood micro￾grants 0 refer to Annual Targets sheet 28 1,230 4,636 2,456 3,676 Intermediate Result 2: Social cohesion of IDPs and HCMs strengthened 8 Level of polarization towards the other identity group 3.01 End of Project Measureme nt - - - - - 9 Level of self￾perceived public representation 2.85 End of Project Measureme nt - - - - - Output 2.1: Social cohesion grants to host and displaced communities 10 Number of displaced/HCMs who benefited from social cohesion grants 0 refer to Annual Targets sheet 0 31,074 34,307 14,752 37,659 Output 2.2: Social cohesion trainings 11 Number of IDP/HCM trained on civic engagement 0 refer to Annual 335 3,085 3,798 1,409 1,604 (MRP External Evaluation Annexes) Page 242 | 286 Targets sheet Notes: Indicator 2: Custom Indicator, no yearly/life of project target; funding dependent Indicator 3: Custom Indicator; no targets in Y1 & Y2, succeeding targets were based on the previous year’s performance Indicator 6: Indicator was revised in Y2 onwards from “number of trained” to “percent trained” MRP External Evaluation Annexes) Page 243 | 286 Annex 14 COORDINATION, PARTNERSHIP, AND COLLABORATION ACTIVITIES OF MRP Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 244 | 286 Annex 14 COORDINATION, PARTNERSHIP, AND COLLABORATION ACTIVITIES CONDUCTED BY MRP Organization Contribution to MRP 2019 Task Force Bangon Marawi (TFBM) Furnished MRP with the official list of businesses located in the MAA prior to the siege through DTI. The information greatly facilitated the processing of BRG proposals. Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) Allowed MRP to mobilize its Lanao del Sur municipal social welfare officers to assist IDPs in their respective jurisdictions through the Minister for Social Services Department of Trade and Industry (DTI) Provided management training to CSG members, information to avoid duplication of assistance to IDPs; Facilitated the processing of Business Name applications; Provided training in enterprise management to CSG members who are awaiting receipt of their approved grants. Technical Education and Skills Development Agency (TESDA) Provided MRP with a list of recommended technical vocational institutes. Department of Labor and Employment (DoLE) Assisted with job fairs, advertising skills training, and employment programs to firms in Iligan; Facilitated the approval of CSG applications for workers’ associations Provincial Government of Lanao del Sur Helped launch MRP activities through Provincial agriculture office, Technology and Livelihood Development Center (TLDC), Provincial social welfare development office, and the Cooperatives Development Office Municipal and Barangay LGUs MLGUs of Butig, Marantao, PIagapo, and Kapai provided complementary resources to the CSGs in their respective areas; BLGUs assisted in organizing CSGs and implementing their projects. Provided MRP with venues for CSG meetings and data on IDPs 2020 Task Force Bangon Marawi (TFBM) As an active member of the TFBM Task Force, MRP participates in a quarterly coordination meeting among government line agencies and Mindanao Humanitarian implementing institutions MRP External Evaluation Annexes) Page 245 | 286 Organization Contribution to MRP to provided progress status updates and aim to synchronize project undertakings Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) Served as link in implementation of MRP activities such as BARMM-Health on COVID-19 response in Lanao del Sur and BARMM-Ministry of Agriculture Fisheries & Agrarian Reform on agriculture and aqua-culture priorities in Lanao del Sur Department of Trade and Industry (DTI) Provided technical expertise for the Business Development and Enterprise Development Course Department of Labor and Employment (DoLE) Assisted MRP for the registration of CSGs into Workers Associations for them to obtain legal entity status and to qualify for other government programs and interventions for workers associations Provincial Government of Lanao del Sur Partnered in organizing cooperative Pre-membership Education Seminars for CSGs, especially those who take the livelihood and enterprise development track through the Provincial Cooperative Development Offices and the Cooperative Development Authority Municipal and Barangay LGUs LCEs assisted in coordinating the delivery, distribution and deployment of assistance needed for the containment of the pandemic; barangay officials offered to provide washing facilities with soap for their constituents that are participating in MRP trainings 2021 Task Force Bangon Marawi (TFBM) Shared resources relating to information on recovery, reconstruction and rehabilitation, including initiatives to assist displaced communities of Marawi City Bangsamoro Autonomous Region of Muslim Mindanao (BARMM) Shared resources on Workforce Preparations, Voter’s Education, COVID-19 and Gender-related initiatives Bangsamoro Electoral Office (BEO) Served as resource persons on the Training of Trainers (TOT) on Voter’s Education and Youth Engagement Workshops in Lanao del Sur and in the community roll-out of voter’s education; Facilitated the registration of IDPs in their new locations Bangsamoro Youth Commission (BYC) Served as resource persons on the TOT on Voter’s Education and Youth Engagement Workshops in Lanao del Sur and in the (MRP External Evaluation Annexes) Page 246 | 286 Organization Contribution to MRP community roll-out of voter’s education and youth engagement workshops Ministry of Trade, Investment and Tourism (MTIT) Served as resource persons and mentors of the MRP organized Enterprise Development Training and Mentorship program for the CSGs; Supported business registration of CSGs in Lanao del Sur Department of Trade and Industry Assisted in implementing the BEDC; Supported business registration of CSGs in Lanao del Norte Technical Education and Skills Development Agency Assisted in implementing the Community-Based Technical Skills Trainings including monitoring of TVIs’ compliance with health protocols and blended learning strategy amid COVID-19 pandemic Inter-Agency Task Force on COVID-19 (Lanao del Sur) Strengthened partnership and collaboration with Lanao del Sur IATF and regular attendance at IATF meetings with other municipal mayors, heads of offices and hospital administrators. Provincial Government of Lanao del Sur Engaged in conversations regarding MRP partnerships and endeavors and continued support to MRP-relevant programs Iligan City Local Government Provided periodic updates on city-wide COVID-19 Executive Orders and guidelines; Provided barangay clearances for MRP domestic travelers Iligan City Health Office Provided technical support for COVID-19 Management and Prevention on workplace scenarios for the Program Management Office Local Government Units (Provincial, City, Municipality, Barangay) Assisted in planning and implementation of sustainability plan encompassing MRP goals in terms of strengthening CSGs on democratic participation including addressing gender equality and addressing gender-based violence; development of livelihood clusters and industries for maximum local economic development Barangay Governments Shared resources and guidance in monitoring of community￾based activities Equal Access International (EAI) Partnered with MRP for capacity development on community peace building, project design thinking for innovative solutions, project implementation and management, developing media and advocacy campaigns targeting the youth leaders in MRP communities MRP External Evaluation Annexes) Page 247 | 286 Organization Contribution to MRP Asia Foundation United/Voices for Peace Network Assisted in facilitating Voter’s Education Trainers Training and Youth Engagement for the roll out of orientation activities in line with MRP’s civic engagement initiative to help IDPs understand how to exercise their rights of suffrage KOICA Explored possible collaboration in Year 4; Discussed possible locations and target beneficiaries of and the kinds of assistance provided in identified thematic overlaps (livelihood support, psychosocial support services, gender and youth), and identifying initial activities that can be done to jumpstart the collaboration Action Against Hunger (AAH) Explored possible collaboration and complementation of initiatives to assist the IDPs and communities affected by the Marawi Siege on WASH (provision/repair of MRP community improvements for water facilities), trainings on protection monitoring as part of the ongoing capacity building of duty bearers of the Referral Pathways system) (MRP External Evaluation Annexes) Page 248 | 286 Annex 15 LIST OF SUSTAINABILITY COMMITMENTS FROM VARIOUS LGUS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 249 | 286 Annex 15 LIST OF SUSTAINABILITY COMMITMENTS FROM VARIOUS LGUs LGU Commitment 2021 Iligan City • To pursue its collaboration in establishing demonstration farms for horticulture and aquaculture for the IDP and HCM beneficiaries but with the goal of also benefiting non-direct beneficiaries in the long term. • To provide potential site for market hub through its Economic Enterprise Development and Management Office (EEDMO) • To jointly work for the implementation of hands-on training and entrepreneur’s mentorship program through CLGU-EEDMO • To create a Technical Working Group representing MRP and the CLGU to meet monthly and track progress of project implementation. • To support MRP interventions to prevent Gender-Based Violence (e.g., establishment and operationalization of CP-GBV Referral Pathway and awareness raising on GBV) Marantao • To look for possible shared production and marketing centers for CSGs (e.g., production and display center for garments and other light industries) • To help secure land to be used for the establishment of horticulture demonstration farm. (dressmaking/weaving) • To collaborate in the conduct of rapid market appraisal to guide agriculture production-related priorities. • To jointly conduct hands-on training, entrepreneur mentorship program and other civic engagement trainings activities. • To support MRP interventions to prevent Gender-Based Violence (e.g., establishment and operationalization of CP-GBV Referral Pathway and awareness raising on GBV). Baloi • To complement MRP in its Fish Hatchery and Horticulture Nursery projects. • To jointly conduct hands-on training, entrepreneur mentorship program and other civic engagement trainings activities (e.g., voter’s education and community peace building). • To support MRP interventions to prevent Gender-Based Violence (e.g., establishment and operationalization of CP-GBV Referral Pathway and awareness raising on GBV). Butig • To provide a 4-5-hectares of land with authority to use for the establishment of horticulture demonstration for a cluster of CSGs in Butig, Lanao del Sur. (MRP External Evaluation Annexes) Page 250 | 286 LGU Commitment • To create a TWG with representatives from both organizations to work on the establishment of a horticulture demonstration farm. • To jointly conduct hands-on training, entrepreneur mentorship program and other civic engagement trainings activities. • To support MRP interventions to prevent Gender-Based Violence (e.g., establishment and operationalization of CP-GBV Referral Pathway and awareness raising on GBV). • To facilitate in federating the 52 CSGs from 6 Barangays to formally become CSOs to represent in the Special Bodies Balindong • To work closely to align MRP livelihood priorities with existing MLGU industries (i.e., production and marketing of smoked fish) • To explore collaborative activities with MRP leading towards registration of qualified organizations to obtain legal entity and participate in the activities of Local Special Bodies. • To work with MRP trained youth leaders on Community Score Card to help them conduct project prioritization and development under the Bangsamoro Sagip Kabuhayan through the MSWD • To work closely for the implementation of CSG training and mentorship program for entrepreneurs. Pantao Ragat • To work closely with MRP for a potential complementation in the establishment of a market center for all CSG products (i.e., LGU to look for an existing facility that can be repaired by MRP to be used as a market hub/bagsakan center for agriculture and fishery products) • To collaborate in implementing business development and entrepreneurial training and mentorship program for MRP CSGs • To collaborate in the implementation of GBV prevention programs, including sharing with other LGUs strategies that might have prevented GBV from happening. Madalum • To assume full responsibility in sustaining all projects implemented in the municipality including municipal accreditation of qualified CSGs as per Vice Mayor Usman Sarangani on behalf of the mayor. • To work closely to access services from the Ministry of Agriculture, Fisheries and Agrarian Reform on soil analysis as part of interventions in developing horticulture. • To conduct BEP and CSG mentorship program. • To work closely with MRP in Gender Equality and GBV awareness raising campaign and in establishing a referral pathway. Tubaran • To support the implementation of MRP value chain priority livelihoods in horticulture and handwoven industries, including CSG capacity building/technical skills trainings and mentorship program MRP External Evaluation Annexes) Page 251 | 286 LGU Commitment • To support the implementation of Tangkal solid waste management (in-kind grant for segregation and training component). Separately, MLGU submits a proposal for potential cost sharing in the establishment of Residual Containment Area (RCA). • To support gender equality and GBV prevention campaign. Pantar Pantar Mayor Tago committed LGU support: • To support CSGs especially in marketing their farm products. Explore the possibility of establishing product consolidation center utilizing the gymnasium. • To explore complementation of LGU aquaculture project with MRP interventions. • To establish and operationalize GBV referral pathway. • To engage in awareness raising on gender equality and GBV. Lumba Bayabao • To issue an authority to use a municipal building/facility for CSGs’ common production area and display center of products. • To explore complementation of MRP projects and existing agricultural priorities especially those that support or can be integrated with rice, corn, vegetable, abaca production, livestock and poultry. • To conduct GBV champions workshop and dialogues with religious leaders Ditsaan Ramain • To monitor agriculture-related interventions and link with MAFAR projects and other privately owned demonstration farms. • To continue assistance in providing shared production facility for dressmaking. • To support the upskilling capacity of LGU staff on gender equality and GBV. • To conduct mentorship program for CSGs entrepreneurs 2022 Poona Bayabao • To support the sustainability of MRP-implemented livelihood projects in Poona Bayabao. • To collaborate in clustering CSGs for the purpose of implementing community learning facilities to improve social cohesion between IDPs and HCMs. • To collaborate in implementing technical skills and business sustainability planning workshop for CSGs implementing various livelihood micro-grants. • To accredit qualified CSGs as CSOs. • To support MRP interventions and awareness raising on Gender Development and GBV prevention Buadiposo Buntong • To support MRP projects and existing agricultural and fishery development priorities especially those that support or can be (MRP External Evaluation Annexes) Page 252 | 286 LGU Commitment integrated with the Municipality’s primary crops/commodities such as rice, corn and vegetables. • To collaborate in strengthening community-based organizations with LGU commitment to accredit MRP supported CSGs as CSOs. • To support MRP interventions and awareness raising on GAD and GBV prevention. Tugaya • To collaborate in systematizing mass production of woodwork like in the case of baor by outsourcing wood workers from other areas and bringing back the semi-finished baors for finishing touches and imbedding of the “perlas”. • To collaborate on VAWC and GBV awareness raising improving mechanism for case reporting. • To provide GAD space and once identified MRP can help in the improvement of the same. • To collaborate in the trainings and technology-transfer related undertakings. MRP External Evaluation Annexes) Page 253 | 286 Annex 16 QUANTITATIVE ANALYSIS RESULTS Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 254 | 286 Annex 16 Quantitative Analysis Results 1. Panel data analysis on level of perceived self-reliance Random-effects GLS regression Number of obs = 392 Group variable: ID2 Number of groups = 196 R-sq: Obs per group: within = 0.0049 min = 2 between = 0.0815 avg = 2.0 overall = 0.0469 max = 2 Wald chi2(5) = 17.64 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.0034 ------------------------------------------------------------------------------ Self_relia~e | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Time2 | -.0479006 .0400337 -1.20 0.231 -.1263652 .030564 Sex | -.0566735 .0446585 -1.27 0.204 -.1442026 .0308556 Age | -.0430772 .0534987 -0.81 0.421 -.1479327 .0617784 Identity | -.1265892 .0489679 -2.59 0.010 -.2225645 -.0306138 Location | -.0726753 .0466992 -1.56 0.120 -.1642039 .0188534 _cons | 2.728551 .0633041 43.10 0.000 2.604477 2.852624 -------------+---------------------------------------------------------------- sigma_u | .1186534 sigma_e | .39325312 rho | .08344047 (fraction of variance due to u_i) MRP External Evaluation Annexes) Page 255 | 286 ------------------------------------------------------------------------------ • The table suggests that there is no significant difference in the level of perceived self-reliance of the MRP beneficiaries during the baseline and endline assessments after controlling for the effect of the sex, age, identity group and location variables. This is supported by the p-value of the Time2 variable which is higher than 0.05. With COVID-19 occurring in between the baseline and endline assessment periods, this could mean that their perception about their ability to meet the basic needs of their family did not significantly decrease even with the onset of the pandemic. 2. Cross-tabulations and chi-square tests of independence involving employment-related outcomes. GROUP * ENGAGED IN PAID WORK/LIVELIHOOD CROSSTABULATION ENGAGED No Yes Total GROUP Comparison Group Count 133 114 247 % within GROUP 53.8% 46.2% 100.0% Intervention Group Count 196 241 437 % within GROUP 44.9% 55.1% 100.0% Total Count 329 355 684 % within GROUP 48.1% 51.9% 100.0% Chi-Square Test: Engaged in paid work/livelihood Value df p-value Sig. (2-sided) Sig. (1-sided) Pearson Chi-Square 5.114a 1 .024 Continuity Correctionb 4.761 1 .029 Likelihood Ratio 5.117 1 .024 Fisher’s Exact Test .026 .015 Linear-by-Linear Association 5.107 1 .024 (MRP External Evaluation Annexes) Page 256 | 286 • There is a significant difference in the proportion of those who indicated that they are engaged in paid work between the intervention and comparison group. This is supported by the p￾values of the significance test statistics which are less than 0.05 (e.g. Pearson chi-square). In particular, the results suggest that the proportion of those who have paid work is significantly higher in the intervention group. GROUP *RIGHTSKILLS_2 CROSSTABULATION RIGHTSKILLS_2 Total Disagree/strongly disagree Agree/strongly agree GROUP Comparison Group Count 82 163 245 % within GROUP 33.5% 66.5% 100.0% Intervention Group Count 85 344 429 % within GROUP 19.8% 80.2% 100.0% Total Count 167 507 674 % within GROUP 24.8% 75.2% 100.0% Chi-Square Test: Have the right skills to find a job Value df p-value Sig. (2-sided) Sig. (1-sided) Pearson Chi-Square 15.603a 1 .000 Continuity Correctionb 14.879 1 .000 Likelihood Ratio 15.244 1 .000 Fisher’s Exact Test .000 .000 Linear-by-Linear Association 15.579 1 .000 • There is a significant difference in the proportion of those who indicated that they have the right skills to find a job between the intervention and comparison group. This is supported by the p-values of the significance test statistics which are less than 0.05 (e.g., Pearson chi-square). This further implies that the proportion of those who have paid work is significantly higher in the intervention group. MRP External Evaluation Annexes) Page 257 | 286 GROUP*JOBCONFIDENCE_2 CROSSTABULATION JOBCONFIDENCE_2 Total Disagree/strongly disagree Agree/strongly agree GROUP Comparison Group Count 102 121 223 % within GROUP 45.7% 54.3% 100.0% Intervention Group Count 166 234 400 % within GROUP 41.5% 58.5% 100.0% Total Count 268 355 623 % within GROUP 43.0% 57.0% 100.0% Chi-Square Test: Confident to find a job Value df p-value Sig. (2-sided) Sig. (1-sided) Pearson Chi-Square 1.050a 1 .305 Continuity Correctionb .884 1 .347 Likelihood Ratio 1.048 1 .306 Fisher’s Exact Test .312 .174 Linear-by-Linear Association 1.048 1 .306 • The chi-square test results indicate that there is no significant difference in the proportion of the intervention and comparison group respondents who indicated that they are confident they will find new paid work if they lose their current job. This is supported by the p-values of the significance tests which are all greater than the level of significance which is 0.05. 3. Panel data analysis on level of polarization Random-effects GLS regression Number of obs = 392 Group variable: ID2 Number of groups = 196 R-sq: Obs per group: (MRP External Evaluation Annexes) Page 258 | 286 within = 0.0160 min = 2 between = 0.0546 avg = 2.0 overall = 0.0318 max = 2 Wald chi2(5) = 12.69 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.0265 POL_AVE | Coef. Std. Err. Z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Time2 | -.0610077 .0262176 2.33 0.020 -.1123932 -.0096221 Sex | .0033484 .0272052 0.12 0.902 -.0499728 .0566696 Age | .0541578 .0327432 1.65 0.098 -.0100178 .1183334 Identity | .0117803 .0295935 0.40 0.691 -.0462219 .0697826 Location | -.0687964 .0282429 -2.44 0.015 -.1241515 -.0134413 _cons | 3.022025 .038732 78.02 0.000 2.946112 3.097938 -------------+---------------------------------------------------------------- sigma_u | 0 sigma_e | .27873687 rho | 0 (fraction of variance due to u_i) • The results indicate that there is a significant difference in the level of perceived polarization of the MRP beneficiaries during the baseline and endline assessments. In particular, their perception that polarization exists in their community has significant decreased during the endline after controlling for the effect of the sex, age, identity group and location variables. 4. Panel data analysis on level of trust Random-effects GLS regression Number of obs = 392 Group variable: ID2 Number of groups = 196 R-sq: Obs per group: within = 0.0200 min = 2 MRP External Evaluation Annexes) Page 259 | 286 between = 0.1158 avg = 2.0 overall = 0.0662 max = 2 Wald chi2(5) = 27.36 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.0000 Trust | Coef. Std. Err. Z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Time2 | .0576639 .037088 1.55 0.046 -.0150272 .130355 Sex | -.0156487 .0384851 -0.41 0.684 -.091078 .0597806 Age | .0531769 .0463193 1.15 0.251 -.0376072 .1439611 Identity | .0364346 .0418636 0.87 0.384 -.0456166 .1184858 Location | -.19056 .039953 -4.77 0.000 -.2688664 -.1122535 _cons | 3.226377 .0547912 58.88 0.000 3.118988 3.333766 -------------+---------------------------------------------------------------- sigma_u | 0 sigma_e | .38062641 rho | 0 (fraction of variance due to u_i) • There is a significant difference in the level of perceived level of trust of the MRP beneficiaries during the baseline and endline assessments. The regression coefficient for Time2 suggests that their overall perception of trust has significant increased during the endline after controlling for the effect of the sex, age, identity group and location variables. 5. Independent samples t-test on level of trust between comparison and intervention groups Independent Samples T-Test: Level of trust t df p Cohen’s d Trust -6.047 682 < .001 0.481 Note. Student’s t-test. (MRP External Evaluation Annexes) Page 260 | 286 • The table suggests that the level of trust between the intervention and comparison group is significantly different. In particular, the level of trust of MRP beneficiaries is significantly higher than those who did not receive MRP interventions. 6. Panel data analysis on level of public participation Random-effects GLS regression Number of obs = 390 Group variable: ID2 Number of groups = 196 R-sq: Obs per group: within = 0.0005 min = 1 between = 0.0167 avg = 2.0 overall = 0.0085 max = 2 Wald chi2(5) = 3.30 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.6535 ------------------------------------------------------------------------------ Representa~n | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Time2 | .0032876 .0534898 -0.06 0.951 -.1081257 .1015504 Sex | -.0364973 .0555051 -0.66 0.511 -.1452854 .0722907 Age | .0053941 .0670318 0.08 0.936 -.1259858 .1367741 Identity | -.0158245 .06042 -0.26 0.793 -.1342455 .1025965 Location | -.0829701 .0575709 -1.44 0.150 -.1958069 .0298667 _cons | 2.758363 .0794098 34.74 0.000 2.602723 2.914003 -------------+---------------------------------------------------------------- sigma_u | 0 MRP External Evaluation Annexes) Page 261 | 286 sigma_e | .52663126 rho | 0 (fraction of variance due to u_i) ------------------------------------------------------------------------------ • The results indicate that there is no significant difference in the level of perceived public participation of the MRP beneficiaries during the baseline and endline assessments after controlling for the effect of the sex, age, identity group and location variables. While their perception has shown an increase, the magnitude of the increase is not statistically significant. 7. Independent samples t-test on level of public participation between comparison and intervention groups Independent Samples T-Test: Level of Public Participation t df P Cohen’s d Public Participation -13.982 682 < .001 1.113 Note. Student’s t-test. • The comparative analysis results indicate that the level of perceived public participation between the intervention and comparison group is significantly different. In particular, the perceived level of public participation of MRP beneficiaries is significantly higher than those who did not receive MRP interventions. (MRP External Evaluation Annexes) Page 262 | 286 Annex 17 FGD AND KII TRANSCRIPTS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 263 | 286 Annex 17 FGD and KII Transcripts RELEVANCE-RELATED EXCERPTS “For me, the grant that MRP provided to us was appropriate because it also suited our ability as women. In particular, the sewing machine that they gave us helped women in our area to earn income.” (KII, IDP Leader, Ditsaan Ramain) “The in-kind grant from MRP was really a big help to my pharmacy business. It was also timely for the sanitary needs of my customers during time of the COVID-19.” (KII, BRG Grantee, Marawi City) “MRP has helped a lot in providing programs for the IDPs and HCMs. They did a great job in addressing the needs of the beneficiaries. They worked with us closely in the implementation of the programs.” (KII, Buadiposo Buntong, LGU MPDC) “MRP asked people about their needs. The women expressed they needed support for dressmaking to earn income. During the graduation ceremony, I saw the products of women beneficiaries. The women worked hard and committed to support their families. I talked to the beneficiaries; they are the ones who give feedbacks with regards to the improvement that happened. We really appreciate what happened in the MRP, because the community is really the target, the other programs because they already have the design, let’s just say that these are the programs this is what to do, unlike the MRP community to choose. The beauty of MRP is that not only the IDPs from Marawi were targeted, but also the host communities. They made sure that the host community and IDPs can work together. I think this approach is important – that they both benefit from the projects. Because the other IDPs who were able to evacuate here have already settled here, we are not telling them to evacuate because they are already a member of the community.” (KII, MLGU, Bubong) “They [Plan International] did a lot of consultations at the outset such as what program is best, who are the intended beneficiaries, and how to go about it. What’s good with MRP is that from the start we already talked. In fact, during the early stages, they always give me updates. Until now, we consider Plan International as a significant partner especially on the business and livelihood.” (KII, Task Force Bangon Marawi) (MRP External Evaluation Annexes) Page 264 | 286 “MRP is in between a sunsetting and a new country strategy. But in both instances, support for communities affected by violent conflict particularly in Mindanao is included in the MRP results framework.” (KII, USAID) “The co-creation process brought together various stakeholders with multi-disciplinary expertise and experience. Most importantly, ECOWEB and MARADECA who are our implementing partners were part of the process. Their experience in implementing development programs in the area were captured in the process.” (KII, Plan International) “MRP was the first co-created activity of the mission. I found the exercise useful in terms of understanding the interventions that Plan International set out to do to address the problem set for MRP. On the other side, the expectations of USAID as a partner of Plan International and their cooperative agreement. So, it was a very useful exercise to really define the scope of work of MRP. There, we were able to define that MRP will have two pathways of support, and that would be economic recovery and social cohesion. The integration of those two objectives streams were discussed during the formation process.” (KII, USAID) “The concept of community solidarity groups was already identified from the get-go, even at the co￾creation stage. We already know that one of the mechanisms are supporting IDPs and we’re host communities in Lanao that we will work with community solidarity groups. We will organize IDPs and host communities together and become the recipient unit for receiving micro grants.” (KII, USAID) “In the last two years of MRP, we ramped up social cohesion, COVID response, and GBV. GBV funds came very timely because of COVID, and we saw an uptick in GBV. The pandemic restrictions affected the GBV rates negatively. So, GBV increased. It was very timely for us because we convinced the local government partners that one of the harmful consequences of COVID in our secondary issues is, other than the health risks are rise in GBV. So, it came very timely for us. We have been opportunistic, Plan International and us.” (KII, USAID) “Last year and this year, we received special funds for combating Gender-Based Violence. In 2020, soon after the pandemic erupted, I made the case a proposal for funding for COVID response. We saw that, especially during the first half of 2020, municipalities hosting larger numbers of IDP families have higher rates of COVID-19. So, we made a case for 1.2 million COVID-19 response funds. That is another color of money adaptation.” (KII, USAID) “For this assistance, adaptive management was very essential. For example, we recognize the color of money available. If the money comes from the stream of conflict mitigation, we develop assistance MRP External Evaluation Annexes) Page 265 | 286 along that stream. It influences target setting. But, year on year it’s demonstrable that the targets are met.” (KII, Plan International) “From the start of the pandemic, MRP actively coordinated with the Inter Agency Task Forces (IATF) of the different LGUs. Specific attention was accorded to participation in the Lanao del Sur (LDS) IATF as it covered the majority of MRP project areas. MRP was invited and participated in a few the LDS IATF meetings.” (KII, Plan International) “I always say to the people that infrastructure assistance from the government will end. You [IDPs and HCMs] are provided with livelihood assistance so you will not depend on government support. It fits the context because the Maranao’s are business oriented.” (KII, Task Force Bangon Marawi) “The first convergence of DTI with MRP, as the undertaking was economic, was on the area of legitimization of the business – registration, business name, input/lectures from DTI onwards to securing business permits.” (KII, DTI, Lanao del Norte) “I like MRP’s approach of CSG organizing. They included both IDPs and the members of the host communities. I believe that such organization is appropriate for socio-economic intervention.” (KII, DTI, Lanao del Norte) EFFECTIVENESS-RELATED EXCERPTS “At one point during the pandemic, the sales from the face masks were our main income. Our income kept our lives going during that very challenging time.” (KII, CSG Member, Barangay. Cabili, Iligan City) “If we compare our situation before and now, now is better. For example, my two sons who stopped school for two years are now back to school because of the income we have from the handicrafts that we make.” (KII, Business Recovery Grant Recipient, Marawi City) “At first, we were given livelihood, which included sewing machines, and there were fabrics. Those fabrics were used for us to make mokna (veil), and malong. We sold our produce, bought again fabrics to increase our income. It takes a long time to roll over. But we had five cycles. On one rolling, I get P2,000 plus.” (KII, HCM Leader, Ditsaan Ramain) (MRP External Evaluation Annexes) Page 266 | 286 “MRP’s assistance really helped my business. I was able to buy “bongo” (delivery truck) and a mini truck. As a result, I was able to send my children to school and meet my family needs.” (KII, BRG Recipient, Lumba Bayabao) “The members of our CSG have a chance to benefit from the projects we received from MRP. And they received trainings that helped them earn a living.” (KII, HCM leader, Women CSG, Buadiposo Buntong) “MRP has helped us through the provisions of sewing machines and trainings. The income we derived from our products augmented our families’ daily needs, especially for the school expenses of our children.” (FGD, Women CSG Members, Marantao) “The hijab we made from the fabric earned us Php19,950.00. So far, our profit has reached Php40,000.00. All members received their share. We are now on our fifth cycle. My life before MRP came was difficult. It’s hard, sir because all we do is hope and depend on my husband, a motor driver. We rent it, that’s our only source of income. So poor. When MRP came, somehow, somehow, I can say that we got a source to earn because, sir, our arrangement there, the member who sews and produces, they are given income. Personally, if I didn’t join MRP, maybe I was just at home and didn’t have a job. It’s really a bit difficult, you know, the motor driving of my husband, you can’t always earn a lot, especially now that there are so many motors, a lot of competition. When MRP came, it was already there, if personally, it really helped me a lot.” (KII, IDP Leader, Ditsaan Ramain) “It’s different sir, because when we didn’t get a bakery, we didn’t have much income, then when we got a bakery, it helped us a bit, because we met our needs a bit, and our children’s school fees. During special events like graduation a lot of people order from us. We are sharing this profit. At first, we were able to divide P500 per member. We have increased our capacity to buy food for our family.” (KII, CSG, Baloi) “MRP helped a lot. Because the fulltime housewives really have no job and no sewing machine. They didn’t just earn economically, but it also helped every household with small children, they just sewed their own children’s clothes. Those that can be sold in stores, was sold. Before MRP, only my husband had a source of income. It was hard. With the livelihood support I got from MRP; I am earning Php 300.00-500.00. I used it for medicine and food. This seems not that much, but it has helped a lot.” (FGD, CSG, Buadiposo Buntong) “I have observed that apart from the economic and financial benefit, the social interaction among us has also improved. Because without the MRP, the people here would not meet since there’s no reason for them to do so. The interaction with other people has been intensified because of the project activities.” (KII, HCM leader/Women CSG, Buadiposo Buntong) MRP External Evaluation Annexes) Page 267 | 286 “MRP has helped a lot so far. Because weaving first and foremost, it doesn’t happen all the time, for example 8 hours, so the body can’t do that. So, I asked for looms and sewing machines. When the sales of hand weaving were weak, that’s when we focused on sewing because we also know how to sew clothes and curtains. So, instead of relaying the income from hand weaving we still have income because of sewing. The impact of the intervention is not on me but on my community because I’m just helping them. Somehow, I have a little salary but my focus is really to help my community. At first it was difficult because there were no materials, no equipment. Now it’s easy because aside from the grants I received from MRP, I also have friends from other organizations that help our coop.” (KII, BRG Recipient, Marawi City) “During the first two years, you could probably measure the impact of the project not directly on the economic development right away but on the appreciation and acceptance of the community that the assistance for them is anchored on their expressive. Definitely, in the earlier part there are a few who have assistance and may not have been able to nurture it. But they realize that the assistance for them is not time. They appreciated that the help given to them is something that they can benefit in the long term and with other members of the community not only personally. Later, in terms of economic benefit we have observed improvements. For example, when we visited a household at first there was no electric fan. When we revisited the same household, they have already purchased one.” (KII, Plan International) “One manifestation of economic improvement is when we visit them during beneficial monitoring, they have improved their livelihood grant and they have improved their houses. For example, they have added a certain section in their house. We have also seen they have purchased additional furniture and appliances. There are instances that I visited far flung areas. I observed house improvements. Given the distance from hardware, it would be difficult to bring construction materials to their areas. So, we observed they have disposable income that they have probably incurred from the benefit they have from the livelihood grants they received from the project. Also, there is always inventory in their small sari-sari stores. Since we do the monitoring every six months, we have observed those changes over time.” (KII, Plan International) “One of the changes I can say is that I have established a close relationship with other people I am not close to because we meet here two days a week because we need to sew. It’s much better now that I’m involved in MRP because I seem to enjoy it a bit because I always have someone with me. When I didn’t join the MRP, I was just at home. Now that I have joined MRP, I can sew and sell bed sheets. The things I don’t know, I can learn from my colleagues.” (KII, HCM Leader, Ditsaan Ramain) “I think they used to be at home, now they have something to work on. Those women helped the children’s education. If before only their husbands were the earners, they were dependent on their husbands, but now they have helped too. In Ragayan, we used to give only what we could see, but now they have bought their own sewing machine equipment faster. Their Madrasa was renovated from Plan International. Because they had another project in social cohesion, that is a group because (MRP External Evaluation Annexes) Page 268 | 286 their social cohesion is making money, so they are helping the children in the Madrasa.” (FGD, Maradeca) “They [MRP] did a good job, and I was one of them because all the people here, the ones you interviewed before, are the only ones who know how to sew. Before, only a few here knew how to sew. I observed many are interested in sewing because we were taught how to do it during the training. Somehow the women would make money. And I’m happy too. They bought fabrics, made bedsheets, and sold them to earn an income. Something has changed in them now, especially the IDPs, then they didn’t seem to have anything, I’m not saying they don’t have anything to eat, but now they are interested in the project, it seems that their life has improved a bit.” (KII, Barangay Captain, Ditsaan Ramain) “I can say that economic condition of the beneficiaries improved, because before when they were in Marawi, they had a small business, but now, they have doubled. Their stores have grown more, unlike when they were in Marawi which was small. They have income there in Marawi but their income didn’t go down when they were here. So, I can say that there is also an improvement.” (KII, MPDC, Buadiposo Buntong) “Economic improvement is on meeting the expenses of the family. Their livelihood helped with the daily expenses for their children. For example, grocery shopping daily. Also, before they say problems. But now they’re going with a smile. No more crying. That’s in their reaction compared before and now, maybe that’s also manifestations that they seem to have recovered at the same time their economy has changed.” (FGD, Maradeca) “We have no issues with the IDPs here because they are our relatives. Some of them have old houses here.” (FGD, Women CSG Member, Buadiposo Buntong) “We accept IDPs because they are Muslims, and Maranaos. They are IDPs now but we experienced their situation in the past.” (KII, Barangay Captain, Ditsaan Ramain) “Based on our experience, we don’t have conflicts between HCMs and IDPs. They work together in our community. Our mayor didn’t leave for the whole 3 years and they go to the IDPs, we are all with them, and we really go with the mayor, sometimes the LGU sleeps here, because we really look at the IDPs. There is really no discrimination here by saying that you are just IDPs, no. Because life has become more comfortable here for IDPs, so we have more IDPs. But now there are only a few more than 300 families left, because the others returned to Marawi City, and the others went everywhere. But at that time, we were shocked, because even the other municipalities went here, not because we were saying that we were from Buadipuso Buntong, they were joking with us, when MRP External Evaluation Annexes) Page 269 | 286 they said Buadipuso Buntong was loving, and our service was heartfelt program.” (KII, LGU MPDC, Buadiposo Buntong) “Before MRP, we didn’t pay much attention with each other, especially since I just evacuated here, I don’t know them, I’m ashamed to talk to them, my house is just at the end there, I really don’t have any friends. I just go here when I want to buy in the store. I really don’t know anyone. Then, when MRP started, there were meetings, and we became close. Now, we help and support one another. One of them, sir, is what I’m telling you when something happens, and someone dies? We go there and help that way, and when there is a wedding, everyone is invited. Then there are seminars here, and everyone will be informed.” (KII, IDP Leader, Ditsaan Ramain) “In the beginning those IDPs expected that only they would be helped. They really thought that the HCMs would not be helped. But when they found out that the HCMs were with them, it was okay with them. Even the barangay chairman became friends with the IDP and the host. Overall, I think the relationship between IDPs and HCMs is tight because the process is facilitated for both.” (FGD, Maradeca) “The CSGs have made requests from the municipal LGUs. They start to coordinate directly with them. Compared to before, they handle this now on their own. The relationship between LGUs and CSGs is open. This paves the way for greater LGU support.” (FGD, Maradeca) “When there are activities, for example when the projects are received, the chairwoman and the barangay councilors are there as witnesses. The relationship is close because the whole barangay is planning, talking as a group.” (KII Buadiposo Buntong, HCM leader, CSG women) “Before MRP, only my husband had a source of income. It was hard. With the livelihood support I got from MRP; I am earning Php 300.00-500.00. I used it for medicine and food. This seems not that much, but it has helped a lot.” (FGD, Women CSG Member, Buadiposo Buntong) “The beauty of MRP is that not only the IDPs from Marawi were targeted, but also the host communities. They made sure that the host community and IDPs can work together. I think this approach is important – that they both benefit from the projects.” (KII, Municipal Administrator, Bubong) “After being displaced, we never thought we would have the opportunity to be part of a community again, let alone vote for our chosen candidates in any election.” (Youth leader, Lumbatan) (MRP External Evaluation Annexes) Page 270 | 286 “Yes, there is a change in the participation of IDPs and HCMs in the barangay. Like what I said before, they will be fine. Before, they didn’t know, but now they seem to know, no matter what they participate. The women you interviewed are recognized as women’s association in the barangay. Apart from their sewing, the women participate in the affairs of the barangay.” (KII, Barangay Captain, Ditsaan Ramain) “The IDPs and HCMs are even more eager to participate because they know that they are already part of the community and they are being targeted of a certain project, so they are more intrigued, “let’s listen” when the LGU or the barangays have programs and projects they are really attending. They [MRP] made sure that the host community and IDPs can work together. IDPs who evacuated here have already settled in our place, and they are already members of the community.” (KII, MLGU, Bubong) “We have more participation now in community affairs Sir. Most members of our CSG are women only the president and secretary are not. Also, our relationship with the host community members is okay Sir, because our neighbors here are kind, they accept us.” (KII, CSG, Baloi) “We are helping the youth and women federations in their accreditation in the municipality. They already have by-laws and policies which are the requirements for accreditation. They can become members of our local special bodies. The women can be registered as cooperative and the youth can be accredited by National Youth Commission. This is important for their representation.” (KII, MLGU, Bubong) “Yes, I have observed that they have increased their capacity to be self-reliant because they already have newly acquired skills, and then, the others already have skills, they say they sew at home but it’s even more enhanced, they are now accepting even uniforms, they sew. That I appreciate Maradeca the way they do, I believed in the saying that, if you give a man a fish, you feed him a day, but if you teach him how to fish you could feed him for lifetime. So good, they have acquired the skills, and they have the tools to do whatever their skills are.” (KII, MLGU, Bubong) “We still have a plan to go back to Marawi because we have a place there. My parents’ house was in my name, and then there was still a place.” (KII, IDP leader/farmer, Balindong) “I think that is one reason [inclusion of HCMs] why some IDPs feel comfortable in the local community and they have an on-going livelihood. But they will eventually return to Marawi because they are from the place.” (KII, Task Force Bangon Marawi) MRP External Evaluation Annexes) Page 271 | 286 “There are instances where the sharing of resources to IDPs became an issue. With MRP, the host communities are included. I think that is one reason why some IDPs feel comfortable in the local community and they have an on-going livelihood.” (KII, Task Force Bangon Marawi) “When we ask them, they really want economic support. That’s anchor for the recovery. But we know that we have to invest in social cohesion because they will stay in their new communities for an extended time. So, in a way we’re trying to facilitate the integration of IDPs in their new communities in Lanao.” (KII, USAID) “In operation, there is a bit of complexity in gauging the performance of the project since there’s no life of project for the most part. So, for this particular assistance, it’s really more of adaptive management and the recognition of the color of money. It is because the source of funds will dictate the kind of assistance that will be provided to beneficiaries. If the money comes from the stream of conflict mitigation, we may develop assistance along that line. There is that kind of effect in target setting. But, year on year, the meeting of the targets is very demonstrable.” (KII, Plan International) SUSTAINABILITY-RELATED EXCERPTS “They implemented activities that help sustain the business/livelihood of beneficiaries. Examples are the series of trade fairs and business fora. These provided beneficiaries opportunities to sell their products and establish link to the market. Through the value chain approach, we really endeavored to have their products sold even outside of Marawi.” (KII, Task Force Bangon Marawi) “MRP is conducting monitoring. Then they will look at our businesses to see if they still have the capital or not, then they will look at our products to see if they still have. There are also times they take us to attend seminars like about financial, about business. The seminars helped us to sustain our business. As a CSG we also thought that when we buy fabric, the seamstress has a salary so that she has an interest in sewing, then the one who doesn’t sew is the one who sells, and the seller also has a salary. We give a percentage so that the capital is not lost, no matter how small it grows.” (KII, HCM Leader, Ditsaan Ramain) “First, we ensured that the LGUs own and commit support to sustain the gains of the project. Second, we have introduced the CSGs to the other development actors in their communities. We are not only co-workers, but they also benefit from the networks and links of the implementing partners. We also have youth groups that are very active in social entrepreneurship. And there is that conscious push for them to be strengthened and integrated into the ongoing initiatives of the government at the provincial level. So, we made sure that in every opportunity with engagement with our provincial government, the youth groups of MRP will be included. Most of them have their own specific advocacy issues such as peace building and social entrepreneurship.” (KII, Plan International) (MRP External Evaluation Annexes) Page 272 | 286 “There is an overall commitment that we generated from five pilot LGUs in relation to referral pathways for gender-based violence and VAWC. In particular, the utilization of the 5% fund allocation for GAD. During the start of the program, we were involved in the review of their plans. Our participation focused on the establishing or strengthening of the referral pathways. Right now, they are already at a stage of having a written formal resolution for the fight in the five municipalities that I think they are within this year they are going to table the presentation to be included in the budgeting cycle come October.” (KII, Plan International) “We at the LGU are also encouraging them, because when we federated those organizations, they are not yet registered in CDA or now CBS. For those youth, they are not registered with the National Youth Commission, so we are extending our helping hands through our cooperative officer so that we can register them to have a legal entity that is their organization. Because they already have by-laws, those are the requirements, then the organization, there they are. If there are gender and development programs, then we will target them.” (KII, MLGU, Bubong) “For the BRG distribution of in-kind inventories, DTI complemented by training the individual entrepreneurs along the areas of business and financial management, marketing – pricing, costing.” (KII, DTI, Lanao del Norte) “My colleagues want to be able to do a seminar on coop as well so that they can understand better what a cooperative is. It is important because sometimes they don’t seem to understand me; if I tell them the ways we do, I have a hard time because they don’t seem to believe. If we become Coop, many will help with the sale. They seem to be interested in us being a coop so that they don’t get lazy so that they understand what we’re doing.” (KII, HCM Leader, Ditsaan Ramain) “The intention of forming the group [CSG] is not for it to become a formal organization immediately. It’s really to simplify and make it more efficient for grants to be delivered. As it turns out, because the groups are composed of people, some have evolved to more formal groups. They now have a formal platform to pursue their aspirations. The grant served as anchor for that design.” (KII, Plan International) “The beauty of MRP is that not only the IDPs from Marawi were targeted, but also the host communities. They made sure that the host community and IDPs can work together. I think this approach is important – that they both benefit from the projects.” (KII, Municipal Administrator, Bubong) MRP External Evaluation Annexes) Page 273 | 286 “There are instances where the sharing of resources to IDPs became an issue. With MRP, the host communities are included. I think that is one reason why some IDPs feel comfortable in the local community because they have already assimilated there and they have an on-going livelihood.” (KII, Task Force Bangon Marawi) “As a women community solidarity group, we committed at the start to sustain what MRP gave to us. We knew that what they gave would really help us. For me, we will really continue this sir, because we are women. The women are engaged in sewing, and even if we are just at home and we can continue sewing. I see that as an opportunity to earn money. So, do you hate that? No right? As a woman, that’s really a big help.” (KII, IDP Leader, Ditsaan Ramain) “Yes, the programs were appropriate because we are the ones who decided that we want to sew. I hope our business grows, not just in sewing. What we want is for us to be able to buy space in the markets so that we can display our products there so that people can easily buy them.” (KII, HCM Leader, Ditsaan Ramain) “We develop policies on how to manage our investment and income. We set some rules on the percentage sharing of income between those who are involved and the rest of the CSG members. 20% of the income will go back to our capital investment, and 10% is also shared with all members even if they are not working on the sewing machines. This way they continue to participate in our CSG. We believe we need to do this to sustain what we received from MRP. Through this, we hope to increase our operations.” (KII, IDP Leader, Bagoinged, Ditsaan Ramain) “Giving them agency not only the grants. The grants will give them extra motivation to stay especially if their livelihood grants turned out to be productive. But the agency, they were provided with technical support, technical assistance for the projects. They will also upscaled, they have additional learnings like basic accounting. They also have other trainings like how to source out new buyers, how to network. They will benefit from those. The mentoring they received like how you conduct yourselves when you talk to an LGU official because you’re asking for funding, it builds their self-confidence. Whether or not they stay, we’re pretty much confident that they will be productive members of wherever they will be residing in the future.” (KII, Plan International) (MRP External Evaluation Annexes) Page 274 | 286 Annex 18 PROCESS AND SCRIPT FOR VALIDATING SURVEY RESPONDENTS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 275 | 286 Annex 18 Process and Script for Validating Survey Respondents 1. Call the assigned respondent using the phone number in the Validation Reference Sheet (VRS). In case, the respondent has no phone number or if the phone number in the VRS can’t be contacted, please do the following: a. Look for a reference person who belongs to the same CSG, has a phone number, and who belongs to the same municipality and barangay with the respondent you are trying to contact. You can use VRS or the main database to do this. Case 1, to try to contact Respondent #1 (No. 58 in VRS), the phone numbers of No. 54-57 can be used. Case 2, to try to contact Respondent #2 (No. 42 in VRS), other members of Kasalimbago ko Masa sa Brgy. Basingan Farmers (men) CSG in the main database, like Respondent #3 can be used. However, only Maam XXX will have copy of the main database. You need to contact her for possible reference person when Case 2 happens. b. Contact the reference person. As a guide you may say: “Magandang araw po. Ito po ba si _______________? Ako po si ___________ connected sa Panagora na kasalukuyang nag-eevaluate sa MRP, MARADECA at ECOWEB. Miyembro po kayo ng CSG na ____________? Kilala niyo po ba si ____________ (sabihin ang pangalan ng respondent). Gusto sana namin siyang mainterview kaso hindi po namin siya macontact. Puwede po kaya makahingi ng cellphone number niya? O kahit anong paraan paano siya makontak?” 2. Upon contact with the subject respondent, establish rapport first and foremost. As a guide, you may say: “Magandang araw po. Ikaw po ba si _______________? Ako po si ___________ connected sa Panagora na kasalukuyang nag-eevaluate sa MRP, MARADECA at ECOWEB. Miyembro po ba kayo ng CSG na ____________? Tama po ba na kayo ay may natanggap na supporta galing sa ____________ (sabihin kung anong NGO siya under base sa VRS)? Okay salamat po. Kasama po kayo sa listihan ng mga benipisyaryo na iinterbyohin sana namin. Okay lang po ba may kunting katanongan ako sa inyo?” 3. When the timing is right, ask the following questions: a. Tama po ba ang iyong pangalan ay ___________________? Kung hindi, ano po ang tama ninyong pangalan? b. Ang address niyo po ba ngayon ay _________________ (basahin ang Municipality, Barangay, at Province sa Validation Form)? Kung hindi, ano po ang address niyo ngayon? c. Meron po bang palatandaan diyan sa inyo na makakatulong po sa pagpunta namin sa interview? (MRP External Evaluation Annexes) Page 276 | 286 d. Meron pa ba kayong ibang cellphone number na puwedeng makontak kung sakaling hindo kayo makontak sa number na ito? Puwede ko po marecord ang number? e. Kung sakali hindi po kayo puwede, sino po ang puwede naming mainterview? f. Ano po ang contact number niya? g. Tama po ba na kayo ay member ng _________________ (basahin ang CSG name sa VRS)? Kung hindi, anong CSG po kayo member? h. Nakatanggap po ba kayo supporta galing sa Ecoweb/Maradeca/MRP? i. Puede naming kayo mainterview ngayong _______________ (Date to be determined by the coordinator)? 4. Record responses to the abovementioned questions using the Automated Respondents’ Validated Data Form (ARVDF). Meaning, use the ARVDF to record corrections or updates on the name, address, phone number, CSG membership, and other relevant contact information. 5. Thank the respondent. As a guide, you may say: “Maraming salamat po. Sana mameet ko po kayo ngayong ________ (sabihin ang preferred date ng interview) _________ (sabihin ang preferred time ng interview) para sa ating interview.” 6. After the completing the validation process, submit the ARVDF online. MRP External Evaluation Annexes) Page 277 | 286 ANNEX 19 SUGGESTED REVISIONS BASED ON PILOT-TEST OBSERVATIONS Marawi Response Project External Performance Evaluation (MRP External Evaluation Annexes) Page 278 | 286 Annex 19 Suggested Revisions Based on Pilot-test Observations. Items Observations/Recommendation Revision - There should be question that would classify respondents as: business stopped; business restarted Include an item that would classify respondents as: business stopped; business restarted Item 1 There is no need to ask the name. Omit item about name. Item 2 There is no need to ask the date of birth. Omit item about date of birth. Item 3 This is not a question. Convert to a question. Ilang taon ka na ngayon? Item 4 Acceptable N/A No question on civil status. Ano ang iyong estado ngayon? Ikaw ba’y may asawa, hiwalay, biyuda/biyudo o single? Item 5 This is not a question. Convert to a question. Ano ang iyong tribu? Item 6 This is not a question. Convert to a question. Ano ang address ng iyong business? Item 7 This is not a question. Convert to a question. Could be aligned to endline. Break to age ranges. (see revision). Ilan ang nakatira sa inyong tahanan na 29 taong gulang pababa? Ilan naman ang 30 taong gulang pataas? Item 8 Could be transferred towards the end or omitted. Convert to a question. Ano ang iyong cellphone number? Item 9 This is not a question. Convert to a question. Saang lugar ka nanggaling? Item 10 Translate to Tagalog. Anong pinakamataas mong naabot sap ag￾aaral? 0 Hindi naka pag-aral 1 Elementary level 2 Elementary graduate 3 High school level 4 High school graduate 5 College level 6 College graduate MRP External Evaluation Annexes) Page 279 | 286 Items Observations/Recommendation Revision 7 Vocational graduate 8 Master’s degree/Doctor’s degree level 9 Master’s degree/Doctor’s degree graduate 10 Others __________________ Item 11 Vague. Convert to a question. Improve options. They could not understand option 2. No suggestion this time. Item 12 Convert to a question. Provide examples to options. Ano ang kategoriya o klase ng iyong business? 1 Trading (ex. RTW, accessories, grocery) 2 Service (ex. Printing) 3 Manufacturing (ex. 4 Agriculture/fisheries Item 13 Convert to a question but can be omitted. Ano ang naka rehistro na pangalan ng iyong negosyo? Item 14 Convert to a question. Options are okay. Sa anong ahensiya ng gobyerno naka rehistro ang iyong negosyo? Item 15 Convert to a question but can be omitted. Ano ang kasaluyang address ng iyong negosyo o pangkabuhayan? Item 16 Convert to a question. Takes time to compute. Ask the year previous business started instead. Respondents asked if before Marawi siege. Anong taon nag umpisa ang dating mong negosyo bago ang Marawi siege? Prior to Item 17, ask what type of grant is received. Anong ang klase ng grant na-avail mo sa MRP/Ecoweb/Maradeca? 1 Business recovery micro grant 2 Livelihood micro grant 3 Social cohesion micro grant Prior to Item 17, ask what form of grant was received. Anong ang natanggap mo sa grant? 1 Cash 2 In-kind (ex. Grocery, gamit sa negosyo) Item 17 Convert to a question. Magkano ang kabuoang halaga ng grant na iyong natanggap? Item 18 Convert to a question. Kailan mo ito natanggap? (MRP External Evaluation Annexes) Page 280 | 286 Items Observations/Recommendation Revision Item 19 The phrase “paano mo ginawa” is vague. The options indicate “what happened” than “how did you go about”. Change the item to “what happened”. Anong nangyari sa iyong negosyo pagkatapos mong natanggap ang grant? 1 Hindi ko sinimulan 2 Huminto ilang buwan at ngayon ay sarado na 3 Humina ng ilang buwan hanggang ngayon 4 Humina/huminto ng ilang buwan pero ngayon ay nag-operate na 5 Ibinenta ko sa ibang negosyante Item 20 Question is a bit long. Shorten the question. Add lockdown as option since it is frequently mentioned. Ano ang mga dahilan kung bakit nagsara/humina ang iyong negosyo o kabuhayan sa kabila ng grant? (Puede marami ang sagot.) 1 Isyu sa kapayapaan at siguredad 2 Isyu sa pamilya 3 Hindi na manage ng maayos 4 Kulang ang trabahante 5 Kulang ng dagdag na pondo 6 Kulang sa technical na kakahayan 7 Malakas ang competition 8 Lockdown dahil sa COVID 19 9 Others ___________________ Item 21 Simplify the question. Nakakuha ka ba ng grant mula sa ibang institusyon o organisasyon? Item 22 Translate to Tagalog. Kung OO, galing saan? 1 Kamag-anak o kaibigan 2 Gobyerno 3 NGO 4 Simbahah 5 Others ________________ Item 23 Simplify the question. Nagloan ka ba sa ibang institusyon o organisasyon? Item 24 Translate to Tagalog. Kung OO, galing saan? 1 Banko 2 Lending/pawnshop 3 Financier 4 NGO 5 Others ______________ MRP External Evaluation Annexes) Page 281 | 286 Items Observations/Recommendation Revision Item 25 Simplify the question. Make 4Ps an example of government assistance Saan mo ngayon kinukuha ang pangtustos sa pangangailang mo at ng iyong pamilya? 1 Full time na trabaho 2 Part time na trabaho 3 Asawa o sa ibang kamag-anak 4 Remittance galing sa kamag￾anak na nasa ibang bansa 5 Gobyerno (ex. 4Ps) 6 Others _____________________ Item 26 The question appeared vague to respondents. The phrase “sa itaas” is difficult to understand. Align options with endline survey. Anong pangangailangan ang hindi kayang tugonan ng panustos na sinabi mo sa dating tanong? 1 Pagkain 2 Non-food items (includes clothing, hygiene kits, etc.) 3 Gamot o bayad sa ospital 4 Bayad sa loan 5 Pag-aaral ng mga anak 6 Monthly bills (e.g., electricity, water) 7 Pagpapa-ayos ng bahay 8 Upa/renta ng bahay 9 Others _________________ Item 27 Simplify the question. May utang ka ba ngayon? Item 28 Improve question. Add business capital as option. Kung OO, ano ang dahilan kung bakit ka umutang? 1 Pagkain 2 Non-food items (ex. damit, hygiene kits, etc.) 3 Gamot o bayad sa ospital 4 Bayad sa loan 5 Pag-aaral ng mga anak 6 Monthly bills (e.g., electricity, water) 7 Pagpapa-ayos ng bahay 8 Upa/renta ng bahay 9 Others _________________ Item 29 Questions is vague. Improve question. Add “started business when grant received” as option. Anong sitwasyon ng iyong negosyo noong natanggap mo ang grant? 1 Walang pa akong negosyo; nagsimula agad ako ng matanggap ang grant (MRP External Evaluation Annexes) Page 282 | 286 Items Observations/Recommendation Revision 2 Walang akong negosyo; nagsimula ako ilang buwan pa ng matanggap ang grant 3 Sarado; sinimulan ko agad ng matanggap ang grant 4 Sarado; sinimulan ko ilang buwan pa ng matanggap ang grant 5 Nagsimula na ako ulit; pagkatapos ng ilang buwan/taon natanggap ko ang grant Item 30 Questions and options are okay. Sa kasalukuyan, napalawak mo ang iyong negosyo o kabuhayan? Item 31 Improve question. Kung HINDI mo napalawak, ano ang ginawa mo iyong negosyo o kabuhayan? 1 Pareho lang a status o operation 2 Sinama ko sa ibang negosyo Item 32 Improve question. The option “increase inventory” is not clearly understood. Kung napalawak mo, ano ang iyong ginawa? 1 Nagtayo ako ng ibang puwesto 2 Dinagdagan ko ang inventory, produkto o serbisyo 3 Bumili ako ng ibang negosyo 4 Others _________________ Item 33 The options do not address the question about how many. The options are more of the type of employee. Improve options. Sa kasalukuyan, ilang trabahante mero ka? If needed, add a question about the type of employee. Anong klase ang iyong mga trabahante? 1 Kamag-anak na walang bayad 2 Kamag-anak na may bayad 3 Full time na may bayad 4 Part time na may bayad Item 34 The item is vague. No suggestion this time. Item 35 Improve item. Convert to question. Paano ilalarawan ang iyong negosyo? 1 Malaki pa ang lugi 2 Lugi pero kunti nalang 3 Wala pang kita pero hindi naman lugi (break even) 4 Nag uumpisa ng kumita 5 Malaki na ang kita kumpara dati MRP External Evaluation Annexes) Page 283 | 286 Items Observations/Recommendation Revision Item 36 Simplify translated question. Natutugonan ba ng iyong negosyo o kabuhayan ang pangunahing pangangailangan mo at ng iyong pamilya? Item 37 Translate to Tagalog. Align options with endline survey. Kung OO, anong pangangailangan ang natutugonan nito? 1 Pagkain 2 Non-food items (ex. damit, hygiene kits, etc.) 3 Gamot o bayad sa ospital 4 Bayad sa loan 5 Pag-aaral ng mga anak 6 Monthly bills (e.g., electricity, water) 7 Pagpapa-ayos ng bahay 8 Upa/renta ng bahay 9 Others _________________ (MRP External Evaluation Annexes) Page 284 | 286 ANNEX 20 NON-DISCLOSURE AND CONFLICT OF INTEREST AGREEMENTS Marawi Response Project External Performance Evaluation MRP External Evaluation Annexes) Page 285 | 286 Annex 20 NON-DISCLOSURE and CONFLICT OF INTEREST AGREEMENTS 1) John Stephen Curada, Team Leader 2) Roberto Palo, MSME Specialist 3) Orson Salgado, Social Cohesion Specialist (MRP External Evaluation Annexes) Page 286 | 286 - 4) Heizel Rapista, Project Assistant 5) Louie Allen Lacsamana, Data Management Assistant