Mid-Term Evaluation of the Budikadidi Development Food Security Activity in the Democratic Republic of the Congo (DRC) [Insert title of report] FEBRUARY 2020 IMPEL | Implementer-Led Evaluation & Learning Associate Award ABOUT IMPEL The Implementer-Led Evaluation & Learning Associate Award works to improve the design and implementation of Food for Peace (FFP) funded development food security activities (DFSAs) through implementer-led evaluations and knowledge sharing. Funded by the USAID Office of Food for Peace (FFP), the Implementer-Led Evaluation & Learning Associate Award will gather information and knowledge in order to measure performance of DFSAs, strengthen accountability, and improve guidance and policy. This information will help the food security community of practice and USAID to design projects and modify existing projects in ways that bolster performance, efficiency and effectiveness. The Implementer-Led Evaluation & Learning Associate Award is a two-year activity (2019-2021) implemented by Save the Children (lead), TANGO International, and Tulane University in Haiti, the Democratic Republic of Congo, Madagascar, Malawi, Nepal, and Zimbabwe. RECCOMENDED CITATION IMPEL. (2020). Mid-term Evaluation of the Budikadidi Development Food Security Activity in the Democratic Republic of the Congo (DRC). Washington, DC: The Implementer-Led Evaluation & Learning Associate Award PHOTO CREDITS Cover Photo by Eric Mafuta DISCLAIMER This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the Implementer-Led Evaluation & Learning (IMPEL) award and do not necessarily reflect the views of USAID or the United States Government. CONTACT INFORMATION IDEAL Activity c/o Save the Children 899 North Capitol Street NE, Suite #900 Washington, DC 20002 www.fsnnetwork.org info@fsnnetwork.org PREPARED BY: Mid-Term Evaluation of the Budikadidi DFSA in the DRC Table of Contents i TABLE OF CONTENTS Table of Contents...........................................................................................................................i Figures and Tables ....................................................................................................................... iii Acronyms .................................................................................................................................... iv Executive Summary.................................................................................................................... viii 1. Introduction ...........................................................................................................................1 1.1 Background and Setting ............................................................................................................1 1.2 DFSA Goals and Activities..........................................................................................................1 2. Overview of Evaluation ...........................................................................................................4 2.1 Purpose .....................................................................................................................................4 2.2 DRC DFSA MTE Objectives.........................................................................................................4 2.3 Methodology.............................................................................................................................5 2.3.1 Process Overview.....................................................................................................5 2.3.2 Methodology............................................................................................................5 2.3.3 Data Collection Methods .........................................................................................6 2.4 Evaluation Team Composition ................................................................................................11 2.4.1 Limitations..............................................................................................................11 3. Findings – Foundational Purposes ......................................................................................... 13 3.1 SP F.1: Environment Favors Multi-Sectoral Development toward Food & Nutrition Security for All Community Members...........................................................................................................13 3.1.1 Objective 1: Quality of Program Service Delivery ..................................................13 3.1.2 Objective 2: Evidence of Changes..........................................................................15 3.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................16 3.1.4 Objective 4: Coordination, Collaboration, and Convergence ................................17 3.1.5 Objective 5: Early Evidence of Sustainability .........................................................18 3.1.6 Objective 6: Appropriateness and Quality of the R&I Approach...........................19 3.1.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross-Cutting Themes .............................................................................................................19 3.2 SP F.2: Reduce Systemic Gender Barriers to Food and Nutrition Security .............................20 3.2.1 Objective 1: Quality of Program Service Delivery ..................................................20 3.2.2 Objective 2: Evidence of Changes..........................................................................21 3.2.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................22 3.2.4 Objective 4: Coordination, Collaboration, and Convergence ................................22 3.2.5 Objective 5: Early Evidence of Sustainability .........................................................23 3.2.6 Objective 6: Appropriateness and Quality of the R&I Approach...........................23 3.2.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross-Cutting Themes .............................................................................................................24 3.3 SP F.3: Communities Are Resilient to Common Shocks ..........................................................24 3.3.1 Objective 1: Quality of Program Service Delivery ..................................................24 3.3.2 Objective 2: Evidence of Changes..........................................................................26 3.3.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................26 3.3.4 Objective 4: Coordination, Collaboration, and Convergence ................................27 3.3.5 Objective 5: Early Evidence of Sustainability .........................................................27 3.3.6 Objective 6: Appropriateness and Quality of the R&I Approach. ..........................28 4. Findings – Purpose 1: Chronic Malnutrition in children under five sustainably reduced ........... 29 4.1 SP 1.2: Households Practice Optimal HNW Behaviors During the 1000-day Period ..............29 IMPEL | Implementer-Led Evaluation & Learning ii Table of Contents 4.1.1 Objective 1: Quality of Program Service Delivery..................................................29 4.1.2 Objective 2: Evidence of Changes..........................................................................31 4.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................32 4.1.4 Objective 4: Coordination, Collaboration, and Convergence ................................33 4.1.5 Objective 5: Early Evidence of Sustainability .........................................................34 4.1.6 Objective 6: Appropriateness and Quality of the R&I Approach...........................35 4.1.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross-Cutting Themes .............................................................................................................35 4.2 SP 1.4: Communities and HHs Maintain a Clean Environment Conducive to Good Health and Nutrition ..........................................................................................................................................36 4.2.1 Objective 1: Quality of Program Service Delivery ..................................................36 4.2.2 Objective 2: Evidence of Change............................................................................42 4.2.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................42 4.2.4 Objective 4: Coordination, Collaboration, and Convergence ................................43 4.2.5 Objective 5: Early Evidence of Sustainability .........................................................44 4.2.6 Objective 6: Appropriateness and Quality of the R&I Approach...........................44 4.2.7 Objective 7: Appropriateness and Effectiveness of Interventions Focusing on Cross-Cutting Themes .............................................................................................................45 5. Findings – Purpose 2: Household Inclusive Social and Economic Well-Being Improved ............ 47 5.1 SP 2.1, 2.2, 2.3 Inclusive Social and Economic Well-Being......................................................47 5.1.1 Objective 1: Quality of Program Service Delivery ..................................................47 5.1.2 Objective 2: Evidence of Change............................................................................51 5.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design..................................52 5.1.4 Objective 4: Coordination, Collaboration, and Convergence ................................52 5.1.5 Objective 5: Early Evidence of Sustainability .........................................................53 5.1.6 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross-Cutting Themes .............................................................................................................53 6. Findings – Monitoring, Evaluation, Accountability, and Learning (MEAL) ................................ 55 6.1 Contextual Factors ..................................................................................................................55 6.2 Strengths.................................................................................................................................55 6.3 Missed Opportunities..............................................................................................................57 7. Findings – Refine & Implement (R&I)..................................................................................... 59 7.1 Contextual Factors ..................................................................................................................59 7.2 Strengths.................................................................................................................................59 7.3 Missed Opportunities..............................................................................................................61 8. Findings – Other Focus Areas................................................................................................. 62 8.1 Operations and Management.................................................................................................62 8.2 Sustainability ...........................................................................................................................63 8.3 Social Behavioral Change ........................................................................................................63 9. Recommendations................................................................................................................ 64 Annex 1: Budikadidi Activity Descriptions.................................................................................... 78 Annex 2: Evaluation Objectives and Questions............................................................................. 82 Annex 3: Field work and Analysis Calendar .................................................................................. 85 Annex 4: Evaluation Team Members, Roles ................................................................................. 86 Annex 5: Evaluation Sites and Locations In Order Of The Evaluation Visit ..................................... 87 Annex 6: Data Collection Instruments.......................................................................................... 88 Annex 7: Consent Scripts/Forms................................................................................................ 158 Mid-Term Evaluation of the Budikadidi DFSA in the DRC Table of Contents iii FIGURES AND TABLES Figure 1: Poor quality road to a study area...................................................................................12 Figure 2: Queueing at water facility. ............................................................................................19 Figure 3: Jerry cans lined up at water source................................................................................38 Figure 4: Water point schedule....................................................................................................39 Figure 5: Setting up a permaculture home garden........................................................................51 Figure 6: Suggestion box..............................................................................................................56 Figure 7: Theory of Change, Foundations Purpose – before and after R&I.....................................60 Table 1: Budikadidi Project Purposes.............................................................................................2 Table 2: Description of types of respondents by sector and data collection method........................8 Table 3: Evaluation respondents, methods used, sector focus, and location of interviews.............10 IMPEL | Implementer-Led Evaluation & Learning iv Acronyms ACRONYMS AC Animateurs Communautaires BA Barrier Analysis BCZ Bureau Central de la Zone de Santé BEO Bureau Environmental Officer BF Breast Feeding CAC Cellules d’Animation Communautaire CARG Conseiller Agriculture Rurale de Gestion CBO Community Based Organization CCTS Commodity Tracking System CDF Congolese Francs CDCS Country Development Cooperation Strategy CG Care Group CGIAR Consultative Group on International Agricultural Research CIMMYT International Maize and Wheat Improvement Centre CLTS Community Led Total Sanitation CPN Consultation Prenatal CPS Consultation Pre-Scolaire COP Chief of Party CMAM Community Management of Acute Malnutrition CMC Community Marketing Center CNES Comite Nationale d'Ethique de la Santé CODESA Comité de Développement Sanitaire (Health Development Committee) CRS Catholic Relief Services CSA Climate Smart Agriculture CSO Civil Society Organization CT Community Transformation CTW Community Treatment Worker CU5 Children Under 5 years of age DFSA Development Food Security Activity DFID Department for International Development DIP Detailed Implementation Plan DPS Division Provincial De La Santé DRC Democratic Republic of Congo Mid-Term Evaluation of the Budikadidi DFSA in the DRC Acronyms v EA Environmental Assessment EBF Exclusive Breast Feeding EHA Essential Hygiene Actions ET Evaluation Team EWS Early Warning Surveillance FAW Fall Army Worm FFP Food for Peace FFW Food for Work FG Focus Group FGD Focus Group Discussion FH Faithful House FLG Farmer Leader Group FSL Food Security and Livelihoods FtF Feed the Future FY Fiscal Year GBV Gender Based Violence HH Household HIMO Haute Intensité de Main-d’œuvre HNI Human Network International HTSP Healthy Timing and Spacing of Pregnancies HZ Health Zone ICCM Integrated Community Case Management ICRAF International Council for Research in Agro-Forestry IDI In-Depth Interview IDPs Internally Displaced Persons IEE Initial Environmental Examination IGA Integrated Governance Activity IHP Integrated Health Project IITA International Institute of Tropical Agriculture ILRI International Livestock Research Institute INERA Institut Nationale d’Etude et Recherche Agronomique IPAPEL Inspection Provinciale de l’Agriculture, Pêche et l’Elevage ITAPEL Inspection Territoriale de l’Agriculture, Pêche et l’Elevage IP Implementing Partners IPM Integrated Pest Management IMPEL | Implementer-Led Evaluation & Learning vi Acronyms IPTT Indicator Performance Tracking Table IRB Institutional Review Board IYCF Infant and Young Child Feeding LF Lead Farmer LFG Lead Farmer Group LMs Lead Mother's LOA Life of Activity MCZ Medicin Chef de Zone MEAL Monitoring, Evaluation, Accountability, and Learning M&E Monitoring and Evaluation MAM Moderate Acute Malnutrition MDM Medecins du Monde LM Leader Mother MOA Ministry of Agriculture MOH Ministry of Health MOU Memorandum of Understanding MT Metric Tons MTE Mid-Term Evaluation MUAC Mid-Upper Arm Circumference NCBA-CLUSA National Cooperative Business Association CLUSA International OCHA Office for the Coordination of Humanitarian Affairs OFT On-Farm Trial OP Organisation Paysanne PACE Participatory Agricultural Cascade Extension PIRS Performance Indicator Reference Sheet PGs Permagardens PLW Pregnant Lactating Women PMT Program Management Team PNC Police Nationale Congolaise (Congolese National Police) PROSANI Programme Santé Intégrée RACOJ Réseau de l' Association Congolais de Jeune R&I Refine and Implement RCT Randomized Controlled Trial RECO Relais Communautaires (community health workers) REFED Réseau de Femme pour le Développent Mid-Term Evaluation of the Budikadidi DFSA in the DRC Acronyms vii RFA Request for Applications RFLSA Evaluation Rapide de la Sécurité Alimentaire et des Moyens de Subsistance SUAP Safer Use Action Plan SAM Severe Acute Malnutrition SBCC Social and Behavior Change Communications SC Stabilization Centres SENASEM Service Nationale de Semence SG Saving Group SGBV Sexual Gender Based Violence SILC Savings and Internal Lending Communities SOW Scope of Work TA Technical Assistance TOC Theory of Change TOPS Technical and Operational Performance Support TOT Training of Trainers UNICEF United Nations Children’s Fund USAID United States Agency for International Development VEA Village Ecole Assainis WASH Water Sanitation and Hygiene WMC Water Management Committees WMYAT Women Men and Youth of All Tribes WRA Women of Reproductive Age YSGs Youth Savings Groups IMPEL | Implementer-Led Evaluation & Learning viii Executive Summary EXECUTIVE SUMMARY Background In FY 2016, the U.S. Agency for International Development's (USAID) Office of Food for Peace (FFP) issued an award to the Catholic Relief Services (CRS) Democratic Republic of Congo (DRC) to implement a Development Food Security Activity (DFSA). It is a five-year project (October 2016 to September 2021) with a budget of $44 million, being carried out in the Kasaï Oriental province. The project, named Budikadidi, which means "self-sufficiency" in Tshiluba, is implemented by a consortium of partners with expertise in governance, agriculture, youth, gender, and nutrition. Kasai Oriental is the smallest and most densely populated province in the DRC. Agricultural and mining constitute the primary economic activities. Poverty is widespread, particularly in rural areas, where agricultural production is inadequate, artisanal mining renders little profit, and unemployment is high. In addition, health and nutrition indicators are poor. A recent SMART survey in Kasai Oriental found over 59% of children under 5 to be stunted. Strong patriarchal systems favor male dominance, which impacts on gender relations at all levels of the population causing female discrimination and gender inequality. Recent militia activities have caused unrest and internal displacement within the region, contributing to an already fragile situation and creating additional strain on weak government services and systems. While conflict prompted a humanitarian response, Kasai Oriental has received relatively little development assistance as compared with other areas in the (DRC). Budikadidi uses a multi-sectoral approach to deliver a package of interventions aimed to build local capacity, strengthen service-delivery systems, and increase accountability, as well as reduce structural, cultural, and gender-based barriers to change. Activities involve vulnerable populations and aim to provide technical assistance in agricultural production and livelihoods, management of natural resources, nutrition and health, WASH, and resilience. Also, there are critical cross-cutting interventions related to themes such as governance, gender, and youth. Budikadidi is one of the DFSAs piloting a refine and implement (R&I) approach, which involves a preliminary period of formative and applied research and pilot interventions aimed at improving activity design, then followed by full implementation. The goal of Budikadidi is to ensure that nutrition and food security for households improves to achieve sustained nutrition, food security, and economic well-being outcomes. Participants include approximately 426,420 community members living in 85,300 households located in 474 villages in three rural health zones (Miabi, Cilundu, and Kasansa). The project consists of two purposes, with an overarching foundational purpose. These are: ● Foundational Purpose (FP): Communities empowered to sustain improvements to food security and nutrition. ● Purpose (P) 1: Chronic malnutrition in children under five sustainably reduced. ● Purpose (P) 2: Household inclusive social and economic well-being improved. Methodology In August and September 2019, a midterm evaluation (MTE) of Budikadidi was conducted by Tulane University, under the umbrella of the Implementer-Led Evaluation and Learning award (IMPEL). The overall goals of the MTE were to examine the programmatic and operational approaches with the original, approved plan; assess the quality of program service delivery; evaluate the effectiveness of the project approach; and identify contextual factors that are contributing to intended objectives, Mid-Term Evaluation of the Budikadidi DFSA in the DRC Executive Summary ix results, and impacts. The MTE findings informed recommendations aimed to modify strategies to maximize project goals and impact on participants, improve the quality and effectiveness of program activities, identify activities for scale-up, and increase the likelihood of sustainable and positive effects on communities and individuals in the implementation areas. Another critical component was to assess the first-ever application of the R&I model. The ET constituted a six-member multidisciplinary team of experts representing Tulane University, the Kinshasa School of Public Health (KSPH), FFP Washington and the Kinshasa mission. Team members had expertise in qualitative research design, implementation and analysis, program evaluation, agriculture, livelihoods, food security, governance, maternal and child health, nutrition, WASH, monitoring and evaluation, and social and behavioral change communications, Also, the DFSA AOR participated as an observer. The team had support from a local agronomist and four data collectors who had backgrounds in qualitative research methods and fluency in Tshiluba and French, with one having basic English skills. The evaluation employed a mix of qualitative methods, which was complemented by a document review and secondary quantitative data from program monitoring and baseline surveys. Fieldwork began with initial meetings and briefings in Kinshasa and in Kasai, followed by field data collection. The evaluation concluded with a ‘ground-truthing’ workshop to present and discuss initial evaluation results and a debriefing with the USAID mission. Findings Since the official start of Budikadidi on October 1, 2017, the project has demonstrated progress on its strategic objectives and activities are generally being well received by participants. Some key areas where Budikadidi has succeeded to date include:  Contributed to the establishment or revitalization of community animation cells (CAC) in all villages. Setting up the CAC was transparent and democratic, allowing all members of the village to participate in elections and preventing village leaders and elites from taking control.  Established care groups which adhere well to international standards. The program team and communities -some of which have taken the initiative to enroll new families - are to be applauded for their efforts to ensure the core strategy grows with the population and that newly created families (due to new marriages), returnees, and other migrants are included as activity participants. The establishment of a solid, well-integrated governance and CG approach can be attributed, at least in part, to the technical assistance and ongoing oversight and monitoring provided by field staff living in DFSA communities who are trained in technical areas and program implementation.  Established a strong community-based structure for water. Both village chiefs and CAC members are involved in governance and accountability of the water supply.  Equipped and transferred knowledge to Natural Leaders and LMs on WASH concepts and practices. Signs of social cohesion are evident in some communities where youth are being galvanized to construct latrines for vulnerable community members  Used study results from the refinement year to make substantial changes to the project's theory of change and implementation strategies during the refinement year.  Established a strong routine monitoring system that is an effective project management tool used by staff at all levels of the organization. IMPEL | Implementer-Led Evaluation & Learning x Executive Summary The evaluation also found several areas where the project has an opportunity to improve and learn, informing Budikadidi as well as other DFSAs. These can be loosely grouped in four categories:  Timeliness of outputs  Activities need to be scaled up or re-focused  Specific lessons learned from the R&I approach  Linkages between activities/project purposes and harmonization of approaches between consortium members needs improvement. Timeliness of outputs A range of factors both within and outside the control of the IP have continued to contribute to a slow start since the official start of Budikadidi on October 1, 2017. Delays in implementation of activities of key USAID funded collaborators, including USAID Integrated Health Program (IHP) and Integrated Government Activity (IGA), have affected health, nutrition and governance activity specific sub purposes, such as SP 1.1 (Early pregnancies & forced marriages reduced) and 1.3 (All household members make use of high quality, accessible health services). Aspects of the agriculture approach, which is intended to contribute to food security, income, and nutrition, are struggling to reach scale and viability. Some components, such as the market and permagarden approaches, appear still to be in the "refine" phase. The permagardens (PG) method, which intends to ensure year-round production of vegetables by and for households, has been widely adopted by communities. However, CRS recent reports indicate that at least 15,600 permagardens have been implemented, which falls quite short of the 38,332 target. Furthermore, the ET found that the PG activities often failed to follow technical approaches and had other quality related shortcomings. Thus far, Budikadidi has not been successful in implementing activities that increase diversity in crop production and livelihoods which will allow communities to absorb shocks. It is critical that the technical staff and leadership assess the current agricultural strategy and establish a set of approaches that can be implemented on a large enough scale to deliver anticipated results as planned. Those components that do have potential for wide range uptake should be scaled up quickly, while others that do not show promise should be eliminated. Budikadidi must both accelerate implementation of agriculture activities to make up for delays and better ensure that appropriate technical oversight is provided by field agents and followed by participants. Inputs critical for activities and promised by the project, such as seeds, need to be provided in a timely fashion to avoid further delays and to gain project credibility Need for some activities to be scaled up or re-focused The high demand for potable water in the Budikadidi project area far exceeds the budgetary allocation for water system installations. The population need for water sources is placing extreme stress on the water systems. The project has invested in and recently installed water systems, but they are failing to meet SPHERE standards related to queueing time and the average liter per person per day. This raises concerns about the durability of the borehole foot pumps and whether the new pumps will reach their lifespans. There is a perception among project staff, the health system, and communities that child malnutrition is only (severe) acute malnutrition. Children outside of the red zone for MUACs measurements are not considered malnourished or at risk for malnutrition, and communities believe the overall nutrition situation is improving, and in some cases declared the problem solved, even with high levels of stunting. This may undermine uptake of improved nutrition practices as participants may believe there is no need Mid-Term Evaluation of the Budikadidi DFSA in the DRC Executive Summary xi for them to change practices if their child is already "green." The project must reinforce recognition of chronic malnutrition among project staff, the health system, and communities. Gender norms in Kasai are deeply entrenched and will take a long time to change. Budikadidi needs to explore opportunities to better ensure that those activities that appear to have the potential to make a change and gain traction in reducing systemic gender barriers, such as literacy and CG, continue post implementation and thus have a broader and long-term effect Lessons learned from the R&I approach CRS should be commended for their good use of the R&I process to inform and adapt their program design. There are key lessons from this experience that may help inform future R&I implementation. The context in the Budikadidi implementation areas changed rapidly from the time design activities took place and today. The conflict that began in 2016 and the cholera outbreak of 2018 required the project to revise their targeting and activities to address critical needs. Some project staff expressed concern that the findings of the refinement year studies would become irrelevant due to this rapidly changing context. Selection of studies should be strategically prioritized and the number of studies limited in future R&I DFSAs, reducing the management burden, and allowing more time for quality studies to be conducted. This will increase the likelihood that the findings of the studies are used to adapt the project design. The evaluation team agrees that R&I, particularly at this early stage, could benefit by gathering information across DRC DFSAs to determine which formative studies elicited most relevant information regarding activity design and implementation and why. Additionally, the R&I approach should continue to integrate adaptive management throughout implementation. The ET observed differences between the technical sectors, and between senior and junior-level staff within sectors, in their plans for and commitment to adaptive management. Given that both R&I and adaptive management are new to USAID, there is an opportunity to promote both approaches jointly. Linkages between activities/project purposes and harmonization of approaches between consortium members The project's theory of change shows linkages between the agriculture/income pathways and the nutrition pathways, providing an opportunity to create synergies. However, the health and nutrition team (especially below senior management level) does not have enough familiarity with the targeting, strategies, and timing of the livelihoods work to understand how it should be supporting health and nutrition at the household level. Strengthening linkages across purposes—particularly between P1 and P2—is also critical to establish resilience. Interviews revealed that some community members contributing to CAC activities participated voluntarily, while others received monetary payments or gifts in-kind. On the receiving end, communities perceive this as disjointed and unfair systems. While incentives were designed to be motivational, they appeared to demotivate those who did not receive them. Research suggests that the size of incentives relative to others’ incentives can be more important to recipient satisfaction than absolute size or fairness of the incentive. Going forward, CRS should ensure that all new incentives/motivation structures are discussed with the full project team (rather than left to the discretion of sectors), and community representatives, so that potential perceptions of "unfairness" can be addressed before they create problems. IMPEL | Implementer-Led Evaluation & Learning xii Executive Summary The Budikadidi consortium includes an important mix of international and local implementing partners with a range of expertise and experience critical to the success of the project activities. These organizations have different organizational and salary structures. During the evaluation, it became apparent that salary disparities of field agents maintaining similar roles and responsibilities leads to demotivation and discontent among those with less compensation. There is also an opportunity to strengthen communication/linkages with local government. Weaknesses in collaboration and information sharing with government entities at the zonal level identified during the evaluation could jeopardize longer-term sustainability. Recommendations This evaluation has yielded recommendations, presented at the end of the report for each of the project purposes, for Monitoring, Evaluation, Accountability, and Learning (MEAL), for the Refine and Implement approach (R&I), as well as cross-cutting recommendations in strategic and operational areas. The evaluation team presented the preliminary recommendations during a validation workshop with Budikadidi consortium members, FFP, and other stakeholders at the end of the field data collection, providing an early learning opportunity. Additionally, draft findings and recommendations were used in a planning workshop held in December of 2019 to help inform the future development of the project. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Introduction 1 1. INTRODUCTION 1.1 Background and Setting Kasai Oriental is the smallest and most, densely populated province in the Democratic Republic of Congo (DRC). Poor infrastructure makes some areas in Kasai Oriental inaccessible during the rainy season. Agricultural and mining constitute the primary economic activities. Poverty is widespread, particularly in rural areas, where agricultural production is inadequate, artisanal mining renders little profit, and unemployment is high. Strong patriarchal systems favor male dominance, which impacts on gender relations at all levels of the population causing female discrimination and gender inequality. Recent militia activities have caused unrest and internal displacement within the region, contributing to an already fragile situation and creating additional strain on weak government services and systems. While conflict prompted a humanitarian response, Kasai Oriental has received relatively little development assistance as compared with other areas in the (DRC). 1.2 DFSA Goals and Activities In FY 2016, the U.S. Agency for International Development's (USAID) Office of Food for Peace (FFP) issued an award to the Catholic Relief Services (CRS) Democratic Republic of Congo to implement a Development Food Security Activity (DFSA). The project, named Budikadidi, which means "self￾sufficiency" in Tshiluba, is being carried out in the Kasaï Oriental province. Budikadidi is implemented by a consortium of partners. The partners have expertise in governance, agriculture, youth, gender, and nutrition. Included are the National Cooperative Business Association CLUSA International (NCBA￾CLUSA), Caritas, Réseaux Femmes et Développement (REFED), and Réseau des Associations Congolaises de Jeunes (RACOJ). The project uses a multi-sectoral approach to deliver a package of interventions aimed to build local capacity, strengthen service-delivery systems, and increase accountability, as well as reduce structural, cultural, and gender-based barriers to change. Activities work with vulnerable populations to provide technical assistance in agricultural production and livelihoods, management of natural resources, nutrition and health, WASH, and resilience. Also, there are critical cross-cutting interventions related to themes such as governance, gender, and youth. Budikadidi is one of the DFSAs piloting a refine and implement (R&I) approach, which involves a preliminary period of formative and applied research and pilot interventions aimed at improving activity design, then followed by full implementation.. Activities are designed to assist Congolese to identify opportunities to learn, mitigate, adapt, invest, and thrive. Budikadidi results are intended to contribute to USAID/DRC's Country Development Cooperation Strategy (CDCS), which supports the DRC's long-term transition to more effective and empowering development. The goal of Budikadidi is to ensure that nutrition and food security for households improves to achieve sustained nutrition, food security, and economic well-being outcomes. Participants include approximately 426,420 community members living in 85,300 households located in 474 villages in three rural health zones (Miabi, Cilundu, and Kasansa). The activity theory of change (TOC) includes the following purposes (P) and sub-purposes (SP): IMPEL | Implementer-Led Evaluation & Learning 2 Introduction Table 1: Budikadidi Project Purposes Foundational Purpose (FP): Communities empowered to sustain improvements to food security and nutrition. SP F.1: Environment favors multi-sectoral development toward food & nutrition security for all community members SP F.2: Systemic gender barriers to Food and Nutrition Security reduced community members SP F.3: Communities are resilient to common shocks Purpose (P) 1: Chronic malnutrition in children under five sustainably reduced. SP 1.1: Early pregnancies & forced marriages reduced SP 1.2: Households practice optimal HNW behaviors during the 1000-day period SP 1.3: All household members make use of high quality, accessible health services SP 1.4: Communities & HHs maintain a clean environment conducive to good health & nutrition Purpose (P) 2: Household inclusive social and economic well-being improved. SP 2.1: Men and women share household responsibilities and decision making equitably. SP 2.2: HH income increased. SP 2.3: HHs have access to diverse, appropriate foods for all members at all times Setting up local governance structures designed to ensure multi-sectoral development toward food and nutrition security for all community members is pivotal to the foundation and long-term sustainability of the approach. The aim is to have participants involved in and benefit from a range of complementary activities that, over time, become independent of project inputs and self-reliant. The violence that erupted in the Kasai region in 2016 forced Budikadidi to alter DFSA implementation plans, which were initially designed to execute activities in both Kasai Central and Kasai Oriental. Ongoing security concerns in Kasai Central influenced a decision to implement activities only in Kasai Oriental. This change occurred after the project had already set up offices in Kasai Central, causing delays in the start of R&I, recruitment of staff, and the development of working relationships with local and international partners. With this change, the project added a third health zone in Kasai Oriental, not initially included in the DFSA. Since the official start of Budikadidi on October 1, 2017, a range of factors both within and outside the control of the IP have continued to contribute to a slow start. These factors include delays in meeting USAID environmental compliance requirements, procurement challenges, and technical staffing challenges and changes. Contextual factors including the focus on mining for economic revenue and the social consequences associated with mining, inadequate agricultural production, and limited crop diversity, limited economic opportunities, extreme gender inequality, poor communication and road infrastructures, and minimum previous exposure to development projects and approaches, present significant obstacles in implementing effective activities. Also, assistance provided by humanitarian Mid-Term Evaluation of the Budikadidi DFSA in the DRC Introduction 3 agencies involving the distribution of food and cash in project health zones, which generally last about three months, detract from project activities as community members focus on benefiting from the aide. Monetary and in-kind assistance appear to alter expectations and undermine the development process. Another challenge is that residents living in villages affected by the violence in 2016 are reestablishing living structures and trying to meet basic needs and therefore have different development requirements. The inability of existing government structures to consistently provide mandated services, such as basic curative and preventive health care, has impacted negatively on the efficiency of implementation of Budikadidi activities. The slow starts of key USAID funded collaborators, including USAID Integrated Health Program (IHP) and Integrated Government Activity (IGA), have affected health, nutrition and governance activities specific sub purposes, such as SP 1.1 (Early pregnancies & forced marriages reduced) and 1.3 (All household members make use of high quality, accessible health services), which are not evaluated in detail. A detailed description of the project activities by sector is found in Annex 1. In August and September 2019, a midterm evaluation (MTE) of Budikadidi was conducted. This report presents findings and recommendations based on the Budikadidi MTE. IMPEL | Implementer-Led Evaluation & Learning 4 Overview and Evaluation 2. OVERVIEW OF EVALUATION 2.1 Purpose A mid-term, process evaluation (MTE) of the CRS Budikadidi DFSA was carried out in the Kasai Oriental province of DRC. The overall goals of the MTE were to examine the programmatic and operational approaches with the original, approved plan; assess the quality of program service delivery; evaluate the effectiveness of the project approach; and identify contextual factors that are contributing to intended objectives, results, and impacts. The MTE findings informed recommendations aimed to modify strategies to maximize project goals and impact on participants, improve the quality and effectiveness of program activities, identify activities for scale-up, and increase the likelihood of sustainable and positive effects on communities and individuals in the implementation areas. A critical component of this MTE was to assess the first-ever application of the Food for Peace “Refine and Implement” (R&I) model, which involves a preliminary period of formative and applied research and pilot interventions aimed at improving activity design, then followed by full implementation. Objective 5 of the MTE examined R&I according to each technical sector with the aim to complement existing evidence gathered through the R&I studies conducted. 2.2 DRC DFSA MTE Objectives The MTE’s specific objectives are to: 1. Review the quality of program service delivery related to the different themes and systems in addressing chronic food insecurity and child malnutrition with the targeted clients, taking into account contextual changes that may have occurred since the inception of the activities. 2. Identify evidence of changes (positive and negative, intended, and unintended) associated with program interventions. The interventions included identifying factors that appear to promote or hinder food security and safety. 3. Assess the quality, relevance, and efficacy of the DFSA design, taking into account whether activities are contextually appropriate, address critical needs, and maintain standards that can impact positive change. Evaluate relevant programmatic principles such as whether approaches are human-centered, evidence-based, systems-oriented, focused on the most vulnerable, designed for multiple interventions to target the same household, and focused on quality. 4. Assess the degree and benefits of efficient coordination, collaboration, and convergence with locally-based DFSA partners and external organizations that are critical to achieving DFSA goals and purposes. 5. Assess early evidence of sustainability produced by the DFSA activities, thereby determining the extent to which outcomes, systems, and services are designed and being implemented to continue after the DFSA ends. 6. Assess the appropriateness and quality of the R&I approach and activities and how the R&I approach has been used to inform strategies and activities. 7. Determine the appropriateness and effectiveness of interventions focused on cross-cutting themes for the activities. Activities include cross-cutting interventions designed to improve gender and equity in decision making to achieve food security outcomes and targeting youth to Mid-Term Evaluation of the Budikadidi DFSA in the DRC Overview and Evaluation 5 improve their access to, participation in, and benefit from DFSA interventions. Analysis conducted on the effectiveness of activities designed to strengthen governance at the local level, in an overall effort to empower communities to sustain improvements in food security and nutrition, and environmental risk and mitigating the impacts of climate change. 8. Recommend adjustments to program implementation or design and explain how these changes would improve program outcomes and sustained impact. Based on these objectives, the evaluation team (ET) developed key evaluation questions enumerated in the evaluation protocol Annex 2. 2.3 Methodology 2.3.1 Process Overview The evaluation encompassed three phases, involving preparation, data collection, and an analysis and report writing period. Preparations began in July 2019 and included a review of the draft statement of work, team member selection, document review, protocol development, site selection, logistics planning, and instrument development. During this period, technical experts communicated with FFP technicians in Washington and Kinshasa about sector approaches, project designs, approaches, and interventions employed by the DFSA, contextual issues in the DRC, and evaluation data collection methods. The team leader also communicated with the DFSA chief of party (COP) and technical staff to prepare for the fieldwork. During this period, the DFSA team was requested to send a matrix delineating ongoing activities carried out in DFSA communities, as well as lists of more sporadic activities such as trainings, demonstrations, or market days scheduled in DFSA areas at the time of the evaluation. An initial workshop, which was attended by three teams preparing for MTEs in the DRC, was held on August 12th and 13th in Kinshasa to review the data collection design and methods and specific technical components and to consolidate instruments. The evaluation in Kasai took place from August 15th to September 3rd, 2019, which included a briefing of activities, data collection, and a validation workshop. A debriefing presentation on the evaluation process and preliminary findings of the three MTEs, including the MTE carried out in Kasai, was held at the USAID mission with FFP representatives on September 27th . 2.3.2 Methodology The mixed methods approach consisted of a combination of documentation review and a mix of qualitative data collection procedures. While the evaluation primarily drew on qualitative data, the ET members also used available quantitative data (e.g., ongoing monitoring data, annual reporting data, and secondary data) to answer the evaluation questions. The team employed complementary qualitative methods designed to capture information from a breadth of respondents filling diverse roles in project implementation and collaboration and to ensure data triangulation. The evaluation methodology guided development of the detailed evaluation tools found in Annex 6. The evaluation began with a desk review of relevant project reports, including the background and inception documents, such as the Request for Applications (RFA), reports of the formative research carried out during the R&I year, annual and quarterly project reports, baseline studies, and training materials, to gain an understanding of the DFSA designs, details of activities and key actors involved, status of ongoing implementation, contextual factors specific to the DFSA area, and the food security situation. Another important set of documents reviewed were the Pipeline Resource Estimate Proposals IMPEL | Implementer-Led Evaluation & Learning 6 Overview and Evaluation (PREP), which include a package of materials describing already implemented and planned activities. The desk review also included an inventory of other reports and studies that could provide relevant information to the project themes or contextual information about the DRC and the Kasai region, as well as reviews of DFSA specific documents such as reports and materials available at the field level. The team reviewed monitoring data collected by the implementing partners, as well. An initial meeting took place at the Project Headquarters. The DFSA COP and technical team members briefed the ET on the formative research and implementation of field activities, different technical themes, adaptations in implementation, and the use of monitoring and data for program modifications. The briefing session, which lasted about four hours, included discussions during which the ET was able to get further clarification on ongoing project implementation. Before data collection, the evaluation team lead led a one-day training to introduce the evaluation objectives, design, and data collection methods to the local data collectors. Most ET technical leads also attended the training to become more familiar with the evaluation approach and qualitative techniques and methods and to review and refine evaluation instruments with the local team data collector. The interview guides and other tools can be found in Annex 6. 2.3.3 Data Collection Methods During the mission, the team applied key informant interviews, in-depth interviews, observations, and focus group discussions — a description of the provided methods is below. Key informant interviews were administered to a range of technical experts based in Washington, Kinshasa, and the provincial capitals where activities were taking place. Key informants were selected based on their technical expertise and role in FFP activities. The ET carried out key informant interviews with USAID staff providing technical support, the IP COP, and key technical team overseeing technical program development, implementation and monitoring, and representatives of partner organizations or government agencies collaborating on-field activities. Information was collected on the design and development of project activities, including the formative research phase, status of implementation and future activity plans, personnel recruitment and performance management, and actors involved in the different activities, with efforts made to understand the implementing partners' abilities to plan, lead, execute, coordinate and monitor activities. During interviews, the ET assessed understandings of evidence-based decision-making and gathered information on the quality, appropriateness, and utilization of the formative research studies. The ET collected information on capacity strengthening activities, supervision structures, and behavioral change strategies. The team explored contextual factors, such as those related to post-conflict and resettlement, environmental changes affecting agriculture production, and humanitarian assistance that had occurred since the inception of activities. Implementation changes or demand for program activities may have evolved as a result of these contextual factors or other unforeseen constraints, and the overall progress towards reaching desired objectives, including post-implementation sustainability. The ET examined collaboration, coordination, and information sharing with partners and other stakeholders involved directly or indirectly in project activities and critical to achieving project goals. Sector ET leads carried out key informant interviews in French or English. ET members either took handwritten notes or entered information directly in a computer. In-depth interviews were conducted with influential community members providing oversight to project activities such as leaders of the CAC, participants implementing activities in community settings, Budikadidi coordinating staff and field agents involved in providing oversight and input to different Mid-Term Evaluation of the Budikadidi DFSA in the DRC Overview and Evaluation 7 technical themes and project activities, and collaborators representing government sectors and other implementing partners. Initial desk reviews, in-country briefings, and information gathered during the key informant interviews with team leaders and technical advisors guided selection of informants. Topics explored included their roles in activities, training received, and the perceived quality of the training, materials, or supplies received to carry out their work, and utilization of tools and instruments distributed. Questioning also focused on supervision structures, monitoring, and evaluation of project activities, participation of community members in DFSA interventions, and the adaptation of positive practices as a result of project activities and messages. The ET examined perceptions of the DFSA activities and overall approach, challenges faced in implementing activities, recommendations to improve the quality of the interventions, and perceptions of and satisfaction regarding their roles. Team members also assessed collaboration, coordination, and information sharing of lessons learned with actors representing associations and organizations. ET members conducted interviews in French or local language depending on the preference of the respondent. When required local data collectors served as translators. Technical leads took handwritten notes with local data collectors also recording notes. Observations were made of local village structures, activities, and practices such as farming fields, water sources, sanitation practices, livelihood activities, and male-female interactions. During direct observations, the ET assessed activities and practices promoted through DFSA interventions related to farming, market garden, and permagarden techniques, WASH (e.g., handwashing, location, and condition of latrines, cleanliness around households), and child feeding practices. The ET also observed scheduled activities such as, CG meetings, CAC meetings, counseling sessions or household visits led by LMs, and training on value chain strengthening. Focus group discussions were conducted (e.g. caregivers of children under 24 months, LMs, farmers, Faithful House couples, members of listening clubs, and representatives of water management committees) in villages to assess involvement in activities; information received, perceptions of activities, and changes in behaviors (intended and unintended). During group discussions, the ET evaluated the level of exposure to activities, quality of the interventions, potential social or other consequences associated with activities and any initial signs of sustainability-related to addressing food insecurity and nutrition. Team members also assessed integration and coordination of activities from the perspective of community members. The ET identified contextual factors that may be affecting the implementation of interventions or participation of community members, as well as barriers and facilitators to achieving desired objectives and longer-term sustainability. The ET led focus group discussions in local language, with a local data collector serving as moderator and translator during the sessions. Another data collector took handwritten notes during the session. All key informant and in-depth interviews and focus group discussions were audio recorded. IMPEL | Implementer-Led Evaluation & Learning 8 Overview and Evaluation Table 2: Description of types of respondents by sector and data collection method Sector Key informant interviews In-depth interviews Observations FGD Governa nce COP, governance lead, RACOJ staff, DPS partners, Budikadidi coordinators Village chiefs, CAC members, RECOs, Lead Mothers, youth group leaders, CODESA, medical chief officers, health center staff, community animators, agricultural territorial specialists, territory administrators, Budikadidi facilitators*, Budikadidi field agents* CAC meetings, community action plan CAC members, youth group members Gender Gender lead, SBC lead, REFED staff, Budikadidi coordinators CAC members, listening club leaders, literacy instructors, Real Father mentors, Lead Mothers, youth group leaders CAC meetings, FH meetings FH couples, listening group members, SILC members, Lead Mothers Resilien ce COP, governance lead, Budikadidi coordinators Village chiefs, CAC members, RECOs, Lead Mothers, youth group leaders, CODESA, medical chief officers, health center staff, community animators, agricultural territorial specialists, territory administrators Value chain meeting, community action plan Health and nutritio n COP, health and nutrition lead, nutrition manager, PRONANUT staff, DPS staff, Tufts consultant Lead Mothers, medical chief officers, community animators, nutrition supervisor, health center staff, Budikadidi coordinators, health promoters, community relays, CODESA CG meetings, childcare practices Lead Mothers, mother and father participants WASH Health and nutrition lead, WASH manager, DPS WASH representative WASH representative BCZ, water committee members, WASH promoters, Lead Mothers, natural leaders, masons, water technicians, household heads Water points, latrines, handwashing stations, cleanliness of yards, dish racks, water receptacles, signs of open defecation Lead Mothers, mother and father participants, water committee members Mid-Term Evaluation of the Budikadidi DFSA in the DRC Overview and Evaluation 9 Sector Key informant interviews In-depth interviews Observations FGD Agricult ure Agriculture lead, environmental lead, CLUSA staff ITAPEL and IPAPEL staff, environmental agent, Budikadidi agriculture supervisor, rural development agents, experts overseeing PG, participants implementing PG, participants doing road work Permagardens, market gardens, nurseries, road rehabilitation, value chain training OP members, market garden members, SILC group members, youth group members M&E COP, MEAL manager, health and nutrition lead, nutrition manager, Accountability and Learning (A&L) Officer, CRS, accountability manager MEAL officers, Database and ICT4D specialist Demonstrations of mobile data collection system, nutrition registers, CAC visual charts, suggestion boxes, spreadsheets for hot line calls R&I COP, gender lead, health and nutrition lead, nutrition manager TOC Sample Village sites identified before traveling to the province. Due to the short period allotted for the evaluation, the ET aimed to visit six villages. A matrix delineating project activities and distances to the central zonal city, as well as consultations with Foreign Staff Nationals (FSNs) in Kinshasa regarding priority activities, guided selection of evaluation villages. The goal was to capture variability in the interventions carried out in village sites located in the three DFSA health zones. The ET assumed that activities in villages closer to the central zonal town would receive more supervision and be higher performing in terms of quality of activities and the number of participants reached than those villages located in remote areas. Considering the status of ongoing project-specific activities, the ET purposively selected three villages (under 15 km) and three villages far (between 15- 40 km) from the main town, capturing a mix of priority activities, with one close and one far village identified in each health zone. When inter-village sharing of interventions or services existed, such as water points or CG meetings, the ET assessed activities offered in nearby locations. Data collected on field activities focused on six villages. In four villages, data collection lasted two days, while in the final two villages it lasted one day. In one instance, the ET failed to reach the village initially identified because a truck blocked the road. In this case, the ET evaluated the village closest to where the road was blocked. The ET also collected data in health zones and provincial capitals with activity collaborators and government officials. The ET informed the COP and health zone coordinators about IMPEL | Implementer-Led Evaluation & Learning 10 Overview and Evaluation the choice of MTE villages the day before the initial field visit. Annex 5 lists villages visited during the evaluation. The evaluation design targeted a range of government officials, implementing partners, and Budikadidi staff to ensure data triangulation with data collected from 492 respondents (249 women, 243 men). The table below presents the number of data collection events by the method, sector focus, and location. The ET collected data in the three DFSA health zones with Budikadidi coordinators, field agents, collaborators, and participants; in Mbuji Mayi with Budikadidi staff, partners, and collaborators; and in Kinshasa with CRS and FFP personnel. Table 3: Evaluation respondents, methods used, sector focus, and location of interviews Description of data collected Total N Methods Key informants 39 In-depth informants 84 Observations 34 Focus groups 28 TOTAL 185 Sector Focus Agriculture 36 Gender 7 Governance 32 Nutrition and health 33 Management and operations 10 M&E 13 Multisectoral 4 Road work 1 Social behavioral change 3 Youth 6 WASH 47 TOTAL 192* Location Miabi (Health Zone) 45 Kasansa (Health Zone) 56 Cilundu (Health Zone) 45 Mbuji Mayi (Province Capital) 36 Kinshasa (National Capital) 3 TOTAL 185 *Seven interviews covered two sectors, and are counted twice in this section of the table. Analysis ET members took handwritten notes or entered information directly on the computer during data collection. Nightly briefing sessions were routinely held to review daily findings and plan for subsequent data collection. These sessions involved a preliminary analysis of each technical sector, focusing on strengths, missed opportunities, contextual factors, and sustainability. The validation workshop, held toward the end of the data collection period with the ET and implementing partner staff presented an opportunity to share and discuss emerging, preliminary findings with the Mid-Term Evaluation of the Budikadidi DFSA in the DRC Overview and Evaluation 11 implementing partner and develop joint preliminary recommendations. Data analysis continued post￾mission when technical leads used content analysis while reviewing field notes and writing up results and recommendations. The team lead, who maintained detailed interview notes, interview transcripts and nightly briefing summaries, used triangulation to validate the content of the sector write-ups. During data analysis, some ET members also listened to the audio recordings. The ET employed data triangulation to validate results through a combination of data sources such as multiple in-depth interviews, key informant interviews, group discussions, and observations. Ethics ET members obtained informed consent from all key informant and in-depth interview respondents and focus group participants before data collection. Consent forms can be found in Annex 7. 2.4 Evaluation Team Composition The ET constituted a six-member multidisciplinary team of experts representing Tulane University, the Kinshasa School of Public Health (KSPH), FFP Washington and the Kinshasa mission, and CRS. Team members had expertise in qualitative research design, implementation and analysis, program evaluation, agriculture, livelihoods, food security, governance, maternal and child health, nutrition, WASH, monitoring and evaluation, and social and behavioral change communications, with each member leading at least one evaluation thematic sector (see Annex 4). Also, the DFSA AOR participated as an observer. The team had support from a local agronomist and four data collectors who had backgrounds in qualitative research methods and fluency in Tshiluba and French, with one having basic English skills. Each regional team member accompanied an ET sector lead during data collection, serving as a translator from Tshiluba or French as needed, and providing critical insights into the local context and practices. Towards the end of the evaluation, after several sector leads had departed, local team members served as primary data collectors. The team lead had a background in qualitative research and maternal, child health and nutrition, and extensive experience in the DRC. 2.4.1 Limitations There was a limited work period before arrival in DRC and once in country. This impacted essential preparations related to literature and document review, work with sector specialists on data collection approaches and instrument development, and interfacing with the implementing partner. The limited preparation time also resulted in uncertain expectations for each team member’s work duties prior to the field mission. Given the short fieldwork period, it was difficult to carry out in-depth qualitative training, which likely affected the quality of data collection, particularly during the initial days of the evaluation. The expertise, availability, and composition of the evaluation team varied. Some team members were not proficient in French and needed to rely on other evaluation members for interpretation when carrying out interviews with non-English speakers. Some team members had limited previous evaluation experience, particularly involving qualitative data collection and analysis. Some technical leads had limited availability during the data collection period. Additionally, the United States government (USG) travel and security policies shortened the data collection period for some ET members. Other USG restrictions related to ground transport and where staff could stay also affected nightly debriefing sessions. IMPEL | Implementer-Led Evaluation & Learning 12 Overview and Evaluation Figure 1: Poor quality road to a study area. Photo credit: Lauren Blum Due to security concerns, the ET carefully followed the CRS rules related to the timing of travel outside of Mbuji Mayi. Some evaluation communities selected were remote, and, due to poor roads, required significant travel time to be reached — these factors limited interview time in some sites. Due to the limited time for fieldwork and the availability of French-speaking ET members with evaluation expertise, the ET was not able to review all interventions promoted by the project with equal intensity. The ET prioritized interventions receiving most significant project investments and those considered most important to achieving strategic objectives, and most directly coinciding with team members’ technical expertise. The ET recognizes that this trade-off resulted in less attention to some interventions, including delayed pregnancy and small grants. This report includes some observations on these interventions, but they are not a major focus of the analysis. The timeframe and related challenges during the write-up phase limited the extent of systematic analysis of the qualitative data. The team lead faced challenges assembling draft write-ups and other inputs on the different report thematic sectors from ET members. Many of the sector reports were in French; translation to English required time to maintain the content and quality. As a result, compiling quality sector write-ups took far longer than anticipated. In addition, the team lead participated in another evaluation immediately after this MTE, forcing her to postpone data analysis and write up until completion of data collection for the other evaluation. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 13 3. FINDINGS – FOUNDATIONAL PURPOSES The overall foundational purpose of Budikadidi is “Communities empowered to improve and sustain food security and nutrition for members.” This section presents an in-depth analysis of the data related to each of the three sub-foundational purposes. For each sub-purpose, the first seven evaluation objectives are addressed. The eighth evaluation objective, recommendations, are consolidated in the recommendations section at the end of the report. 3.1 SP F.1: Environment Favors Multi-Sectoral Development toward Food & Nutrition Security for All Community Members 3.1.1 Objective 1: Quality of Program Service Delivery Budikadidi contributed to the establishment or revitalization of community animation cells (CAC) in all villages. Setting up the CAC was transparent and democratic, allowing all members of the village to participate in elections and preventing village leaders and elites from hijacking the process and taking control. In villages visited, the village chief, although not a formal member of the CAC, worked in close collaboration and was informed, verbally or in writing, of all CAC activities and achievements. Support from the local politico-administrative authorities, especially village and sector chiefs, has helped the process to go smoothly. The main reason for setting up new CACs was to implement the revised government approach, whereby CACs are multi-sectoral structures, which is different from the original approach, which centered on RECOs. The diversity of Budikadidi activities required a more multi-sectoral framework so the revised government approach coincided well with the project approach. New processes were undertaken to set up new CACs. Although democratic, some CACs the ET visited suffered initially from the non-inclusion of existing RECOs and led to the existence of two CAC structures. Another problem was that the process went through a village assembly, which did not take into account the location and number of community households, which the government approach does. Specifically, the government approach mandates that RECOs, considered primary CAC members, are responsible for following 50 neighboring households. Implementation by Budikadidi of the new approach, not well known by local community members and government authorities, initially created confusion and potential duplication of efforts as illuminated by the ET finding that two CACs existed simultaneously in villages. During the evaluation, the ET found that CACs implemented by Budikadidi were multi-sectoral and appropriately designed to provide oversight and coordinate development activities implemented by different technology sectors. CACs elected a president and cabinet, with each appointed member assigned a specific role according to the area of sector development. Through the project, CAC members have received training to strengthen capacities, particularly regarding improving managerial and administrative capabilities, and defined roles. Training involved the development of internal regulations (ROI), statutes, the development of community action plans, and the use of management tools. Budikadidi provided documentation to all the CAC structures during training, including CAC procedure “People (from Budikadidi) came to the village, they were interested in working with the village chief, they sensitized the village members. They said that they came to help us, that they wanted to work with the community members. They asked us to elect people from the village who will form the groups they will work with.” – CAC President IMPEL | Implementer-Led Evaluation & Learning 14 Findings – Foundational Purposes manuals and management tools. Procedure manuals contain information about the principles and rules that should govern the operation of CAC. Management tools include the cash register, the cash register receipts, inventory documents, attendance lists, report register to function according to the CAC strategy and work plan. Information collected from IP consortium members and DFSA participants suggests that training was of high quality, enabling CACs to function transparently and inclusively. All CACs visited used management tools and functioned as planned, with evidence of community involvement in decision-making within governing bodies. For example, when deciding what crops to grow, the CACs received information on the different crops promoted by Budikadidi and selected which crops to plant. All CACs visited had established an action plan, which is generally followed. The CAC holds regular, typically weekly meetings to discuss village-level activities. The success of CACs appears to largely depend on the characteristics and leadership of CAC members and the involvement of RECOs. Better functioning and integrated CACs more often included existing RECOs, who generally have greater leadership and governance capacity developed through previous experience and training, possessed higher literacy skills and were more proficient with management tools. Active involvement of village chiefs was advantageous, giving credibility to the structure and avoiding power conflicts that could otherwise have impeded relations between the village chief and CAC president. In several villages, the ET identified CAC members who participated in multiple activities (e.g., SILC, OP, LM) and thus facilitated essential links between different technical sectors. The evaluation found that CACs have been implemented at varying speeds and used approaches that could potentially undermine the commitment of members. Interviews revealed that some community members contributing to CAC activities participated voluntarily while others received monetary payments or gifts in-kind. For example, experts overseeing PG received 50 USD for ensuring a certain number of gardens were planted, and LMs were given pagnes (local cloth) to wear when conducting activities, while literacy instructors did not receive special incentives. While incentives were designed to be motivational, they appeared to demotivate those who did not receive them. In addition, the ET uncovered instances when CAC members, such as participants of water committees, may have special access to resources made available through the project. Respondents complained that certain CAC members control sectoral activities for their personal benefit. In the fragile Kasai Oriental context, which is plagued by poverty and corruption and has experienced recent conflict, such discrepancies in allocation of incentives can contribute to intra-community tensions and damage the reputation of the IP. In fact, violent protests led by DFSA participants disgruntled about promised compensation for roadwork caused a temporary setback to the project. Since the approach is designed to strengthen community governance and ownership and lead to community development and self-reliance, the model is intrinsically based on voluntarism and should not involve monetary or gifts as compensation. Another element that has impacted on the approach relates to the expectation expressed by some communities or sub-populations that the project will provide monetary or other assistance, thus slowing the transition to self-reliance and the general progression of implementation activities in these populations. Some communities or sub-groups within these communities maintained the expectation that NGOs and/or internationally funded activities will provide monetary or in-kind assistance. While this mindset is widespread in DRC, particularly in war affected areas that have been the focus of humanitarian assistance, it potentially undermines community members to take “We received training on how to develop the action plan and how to use the tools provided by Budikadidi. The training was three days long and it was provided by the Budikadidi field agents. All of the CAC members were trained.” – CAC Member Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 15 ownership of the development process. These types of expectations occurred more in communities relying on mining activities. In contrast, people that did not anticipate assistance proactively took charge, embracing community governance and instilling early signs of change towards autonomy and self-reliance. The information collected shows that Budikadidi has focused on village-level governance bodies. Correspondingly, in villages visited, the CAC and its sub-structures generally functioned well. However, there has been little focus on governance structures beyond the village. For instance, the ET did not find evidence of strengthening of the Bureau Centrale de Zone de Santé (BCZS), CODESAs, or the Territorial Inspection of Agriculture, Fisheries and Livestock (ITAPEL). In addition, at the zonal level CACs have failed to form some critical linkages with other key actors or associations such as CODESA or the CLD. As a result, many CACs do not connect with the health area health center or leader of the groupement, which, in some instances, caused conflict between village chiefs and chef de groupement. They deficiencies, at least in part, reflect the late start of key USAID collaborators in the form of USAID IHP and IGA, both mandated to strengthen institutional capacity of government workers and positioned to create a natural link between CACs and local structures such as CODESA or CLD. Also, some sectors under the CAC structure lack relations with state structures that are mandated to oversee their activities according to government standards. In several interviews, including interviews with health officials (IT), Community Animators (AC), District Medical Officers, and members of the Provincial Health Division and the territorial agronomist representing ITAPEL, respondents reported being unaware of and not involved in activities. The ET recognizes that some government officials may be incentivized to conceal aspects of their involvement in Budikadidi. In-depth analysis by the ET shows that, while Budikadidi has succeeded in involving certain government structures and officials in activities, efforts are needed to ensure consistent government participation across sectors. Budikadidi teams in the health sector have started to correct this by involving ITs and ACs in activities as trainers, but the involvement of governmental authorities representing other sectors lag behind. Mechanisms for coordination and monitoring of activities have been set up. CAC leadership reports CAC activities to the village chief, but also to the Budikadidi polyvalent (in charge of governance), who enters ongoing activities in the CommCare software. However, monitoring data is not routinely transmitted to the different government sectors, and as a result, the information is not included in national statistics. In the health sector, CAC activities could be sent to the CODESA President, discussed during monthly health center monitoring meetings, and incorporated in the monthly activity report (SNIS) that health centers transmit to health zones. These data are subsequently included in the DHIS2 national-level monitoring data. 3.1.2 Objective 2: Evidence of Changes All DFSA villages set up or revitalized CACs designed to govern and provide oversight to community structures. Budikadidi has contributed to the establishment and structuring of several community groups contributing to local governance and serving to mobilize community activities including farmer organizations, SILC groups, water committees, Faithful Couples, LMs, youth associations and listening clubs. All community groups initiated, although independent in their operations, report to the CACs, which are supported by village chiefs. Expectations and collaboration between village chiefs and CACs appear strong, as reported by village leaders and CAC presidents. All CAC members, members of the community, and opinion leaders interviewed could describe Budikadidi activities and identify CAC structures and leadership. The majority could explain the functioning of the CAC and the various sub-structures such as water committees, SILC, OPs, and CG guiding activities. Members of CAC participated in many trainings aimed to increase capacity related to IMPEL | Implementer-Led Evaluation & Learning 16 Findings – Foundational Purposes governance, management, and leadership. Specifically, training focused on Internal Regulations (ROI), the drafting of statutes, community planning, and the use of management tools. All CACs visited had established community action plans (CAP), delineating information on the implementation and coordination of all activities supported by Budikadidi in villages. The establishment of the CAC governance body and associated structures serves to strengthen civil society efforts related to community development and increase participation and ownership of sector activities. By strengthening the leadership and governance capacity of CAC members, providing the needed management tools, and ensuring ongoing monitoring and oversight, Budikadidi clearly drives changes related to community involvement, empowerment and proprietorship. Several of the governance structures evaluated included women representatives who accounted for about 25% of CAC members, thus coming close to the 30% target established by the Budikadidi project. The project represents an increase in female representation in a context marked by strong male dominance and the absence of women in leadership roles. However, in most cases, women filled less powerful and more female-oriented roles, such as treasurer secretary, advisor, or hostess, a position created by CACs involving the person in charge of greeting members and serving food. Respondents indicated that people have more confidence in women as treasurers, a responsibility involving keeping money but with limited decision-making authority regarding spending, Key informants reported that it is rare to find a CAC with a female president or even vice-president, who in the Congolese context, are the decision makers. At the end of December 2019, Budikadidi monitoring data showed more positive results, with women representing 31% (1,822 of 5,951) of CAC members. Budikadidi data also reports that 11% (112 of 1,033) of CAC presidents and vice-presidents and 72% (389 or 544) of CAC treasurers and assistant treasurers are women, suggesting an increase in female involvement and leadership since the time of the evaluation. Representation of youth on CACs varied across evaluation villages, with some CACs formally including youth leadership and others not. None of the villages evaluated had youth members in a primary CAC leadership role. 3.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design In the context of the DRC, where government policy encourages community participation, the model put in place by Budikadidi is highly relevant and effective, embracing central tenets of the government policy, which promotes a democratic and transparent process in the governance of community activities and their coordination. The establishment of operational CACs has allowed village members to take ownership of the community development process. However, the expectation to receive monetary or in-kind assistance, which during the evaluation community respondents periodically expressed by a request for funds or other resources to support activities likely reflects prior exposure to assistance that may interfere with community commitment towards and sustainability of the development process. Evaluation interviews revealed that local government officials are frequently unfamiliar with the newly established CAC approach. While the ET recognizes that this likely reflects a general problem with the dissemination of the revised government approach, the ramifications are pertinent to Budikadidi. Polyvalent, the field agents responsible for providing oversight to CACs, acknowledged that weakness was related to the lack of understanding and involvement by local government representatives, most of whom had been working in the area prior to the inception of Budikadidi, in the CAC structure and its components. Perhaps because the polyvalent has many villages and activities to supervise, they are overstretched and appeared unable to address this constraint. The failure to establish formal linkages with local government structures relegated the CAC to function as an independent governance body. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 17 This has also resulted in a lack of clarity regarding which government entities are responsible for ensuring appropriate CAC oversight and supervision. As the central governance structure in communities, the CAC should connect to CODESAs in the health area, and through CODESAs, to the Local Development Committee in the territory and health zone. Unfortunately, the establishment of functioning CODESA was supposed to be carried out by the USAID IHP project, which was not operating at the time of the evaluation. USAID IHP has a mandate to provide resources to the IT to support and supervise CODESA. While the handbook on community dynamics describing governance of CACs highlights the multisectoral nature of the approach, the government entity responsible for ensuring oversight to multisectoral approaches is not defined, forcing the CAC to remain under the supervision of the MOH through health center staff. However, this supervision is not adequately provided. 3.1.4 Objective 4: Coordination, Collaboration, and Convergence CRS worked with consortium partner organizations, specifically CARITAS and RACOJ, to set up the CACs. CLUSA field agents provide oversight and supervise the CACs. The CAC is a federated structure comprised of collaborative associations and groups existing at the village level. CACs are responsible for coordinating structures working on the different Budikadidi sector activities. CRS has made extensive efforts to establish collaborative efforts with the government, implementing partners and UN agencies at the national and provincial levels, particularly in the health sector. At the provincial level, a steering committee involving different provincial government bodies has been established to provide project oversight. CRS initiated a multisectoral committee to facilitate information sharing and coordinate activities in communities. CRS participates in regular meetings with other government sectors, particularly those working in health, nutrition, and WASH. An agreement with USAID IHP defining how the organizations will work together on activities related to health, nutrition, and WASH is in effect. Apart from IHP USAID, no other collaborative efforts are being carried out with USAID-funded projects. However, CRS has initiated talks with IGA, a likely collaborator on governance activities, and plans to commence collaboration in the future. The ET encountered contradictory accounts from people regarding the involvement of local government entities and the CAC. Delegates from several government entities, including the MOH at the health zone level and the Ministry of Agriculture, Fisheries, and Livestock (ITAPEL), reported they were not involved and did not know, at least officially, about the CAC and its sub-structures. On the other hand, Budikadidi staff claimed that they had informed local government officials about their activities and involved some in the implementation of the CAC. While the ET recognizes that government officials may have alternative motives for claiming lack of involvement in activities, evaluation observations demonstrated that critical links with government bodies (e.g. CODESA, Local Development Committee (CLD)) are not established. Poor linkages with the health sector are in part due to the late start of USAID IHP, which has the mandate to provide health systems strengthening and coordination of facilities. As a result, key government health representatives responsible for coordinating community participation, such as CODESA, the health center team, and the health zone team, were not involved at the inception of DFSAs. Due to the delay in the implementation of USAID IHP, Budikadidi staff have attempted to make adaptations to facilitate linkages. However, at the time of the evaluation, the government entities did not have a real connection with the CACs, nor did they have an understanding of interventions necessary to take on the activities implemented by Budikadidi. Budikadidi has not yet shared routine data related to its activities with other implementing partners or government structures such as the MOH. Lack of adequate dissemination of information on project activities and lessons learned, coupled with insufficient involvement of local government entities, may IMPEL | Implementer-Led Evaluation & Learning 18 Findings – Foundational Purposes cause other partners to be poorly informed about the CAC structures, sometimes causing confusion and potentially impeding coordination of activities with other partners. For example, the ET learned that another implementing partner, CHEMONICS, which was involved in the distribution of mosquito nets, had recently set up new CACS in Kasai Oriental even in villages where Budikadidi worked. In one evaluation village, respondents consistently reported that another implementing partner working in a nearby village was paying daily wages to Budikadidi trained LMs, which could potentially disrupt the motivation of other volunteers. While the ET recognizes the complexities of field work and is not blaming CRS for these occurrences, these examples highlight the need for transparent information sharing and good coordination at the field level. 3.1.5 Objective 5: Early Evidence of Sustainability The foundation of the Budikadidi approach focused on strengthening governance and community capacity to build self-reliance and autonomy in community development, with sustainability at the center. Budikadidi has deployed resources not only to set up CACs but also to train members in critical aspects of governance, providing basic management tools and materials to function effectively. While CAC members do not have a specific sustainability strategy, they had been sufficiently prepared to take over governance activities. Members elected as CAC members and representatives of CAC structures were viewed as credible and legitimate, with their positions eliciting honor and respect, thus ensuring long term commitment. During interviews, their roles were equated to "prime minister" or minister of the village chief. In particular, village CAC presidents, maintained widespread recognition for their leadership and role in leading meetings, implementing action plans, and receiving reports from representatives of the CAC structures such as the Water Management Committees, LMs, Listening Clubs, SILCs and OP. Critical links have been created between the CACs and their sub-structures, enhancing negotiation skills required to collaborate and coordinate with the multiple existing structures. The formation and implementation of CACs and their associated activities has forced members to improve capacities and to build partnerships designed to mobilize financial and human resources required to carry out activities in their community action plans. The majority of CAC members and people involved in the government structures exhibited good knowledge of guiding governance practices. All respondents knew that Budikadidi had three- years remaining before the end of the project mandate. Most community members were involved directly or indirectly in the process of implementing CAC and its structures such as water committees, CG, SILC, or farmers' organizations, and all reported that the process was democratic, inclusive, and transparent. Implementation of the governance structures has been bolstered through training of field agents, who have gained practical knowledge and become local resources to guide implementation of activities. At the time of the evaluation, several activities appeared to be self-managed and reliant, no longer requiring ongoing support of the project. These included many CACs, but most notably, SILC groups, which, once developed quickly, became operational. SILC members have organized themselves to manage fees collected and assess risk related to their savings, provide credit to members, and manage crises or internal conflicts, and generally no “The project will stop, we have been informed that the project will stop. . . We think that we will continue, we have our plan of action and we will continue the activities even if Budikadidi stops. We will take care of ourselves...” - CAC member Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 19 longer require a field agent to supervise activities. The least sustainable activities during evaluation were those that require resources and supervision, such as farmers' organizations involved in market gardening, who expected seeds and assistance in the management of market garden workplaces and sale of produce. Limited sharing of information on project activities with partners and involvement of local government entities potentially undermines sustainability. Figure 2: Queueing at water facility. Photo credit: Eric Mafuta 3.1.6 Objective 6: Appropriateness and Quality of the R&I Approach The CACs and governance structures are designed to follow new community health policy enacted by the MOH in the DRC. Formative research during the R&I phase was not used to directly guide the design, formation, or operations of CACs, although the gender analysis highlighted potential female roles in governing bodies. Budikadidi officials reported interacting with provincial government representatives to gather information and directives on the recently revised government CAC approach. They noted that new directives were often ignored by the MOH officials at the provincial level, leading to misunderstandings between the project management and government entities. Interviews revealed that Budikadidi has used M&E data to inform attempts to strengthen the functioning of CACs by using. An example of evidence-based adaptations entails the decision to involve government health representatives (IT, AC), whose absence in activities was identified as a weakness, in training, and subsequently to use them as trainers to support the polyvalent. 3.1.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross￾Cutting Themes The CAC provides a central governing body that brings all community activities together under one coordinating structure and serves to provide community members, through CAC delegates, a more in￾depth understanding of the ongoing implementation of activities and their results. CACs elaborated on the community action plan, which delineates key steps and provides a timeframe related to village development to follow. Women and youth representing LMs and youth groups are members of the CACs and report on their activities. Women accounted for about 25% of CAC members, although they generally filled less powerful roles. Youth involved in listening clubs, road rehabilitation and WASH activities are actively engaged. The CAC and their structures have provided a platform for women and IMPEL | Implementer-Led Evaluation & Learning 20 Findings – Foundational Purposes youth to have a voice in development activities. Their involvement strengthens efforts to enhance food security and general village development, and in the case of youth, sustainability. 3.2 SP F.2: Reduce Systemic Gender Barriers to Food and Nutrition Security 3.2.1 Objective 1: Quality of Program Service Delivery The employment of multiple activities (e.g. FH, Real Fathers, Listening Clubs, CGs) targeting different community members on gender issues is a strength, allowing the project to intervene at many levels of community life. While Budikadidi aims to have 30% of governing bodies represented by women, they were approaching but not yet meeting this goal in the evaluation villages. Women generally maintain traditional female roles with little power such as treasurer or secretary; there is little evidence of women leading government bodies or even involved in decision-making in these structures. Literacy centers are set up in villages across the project. At the end of second quarter of FY19, 5,832 of the 19,140 targeted to enroll in literacy in FY19 were taking literacy classes. Centers are run by communities, with classes conducted in schools or makeshift structures and held outside, forcing frequent cancellation during the rainy season. The volunteer instructors are generally teachers who live in the village and organize teaching sessions once a week. In the villages visited, the vast majority of literacy classes were comprised of women, with several women interviewed reporting having learned to read and write in these centers. While the project distributed some early training and support for the management of literacy centers, in the evaluation villages, many essential provisions such as chairs and tables, or even manuals in Tshiluba, had not been provided, making it difficult for centers to function. The ET found that the monthly fees, generally involving 50-100 Congolese francs (CDF) (.03-.06 USD) to cover costs involved, such as the purchase of chalk and paper and to motivate teachers, were typically not paid, preventing local procurement of these essential materials. One village had trained instructors, but three months later when the evaluation took place literacy classes were still not underway because Budikadidi had not provided blackboards. In another village, an instructor had allegedly impregnated one of his students, raising local concerns about the cultural appropriateness of allowing male-female interactions and potentially undermining the reputation of the activity. While literacy activities have allowed women to gain some reading, writing, and numeracy skills, these skills are not highly useful without access to financial resources and livelihood activities. Other activities, such as the SILC and farmers' organizations, should enable women to increase access to financial resources, but the slow start to agricultural activities has limited progress. Loans obtained through SILC are primarily used for immediate household and family needs. Budikadidi initiated the establishment of FH couples and Real Father groups. The project has exceeded the 1,003 target of active Real Father mentors, who are trained to encourage positive masculinity and male involvement in household decision making. FH groups include 15 married couples from neighboring villages who participate in a series of sessions designed to teach couples to work together, improve household relations, and make decisions around food security and nutrition more inclusive. Messages, which are tailored to the local context, promote involvement of women in household discussions and decision-making related to the education, health, and nutrition of children. Couples are exposed to the themes associated with positive masculinity and the roles both parents can play to ensure more equitable intra-household relations, with men encouraged to spend more time with their wives and to assist with issues related to childcare. The approach also aims to reduce domestic violence against women. The ET found that the number of couple participants in the FH is low, with Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 21 many husbands reported to refuse to continue to attend sessions due to concerns that activities are shifting household power dynamics in favor of women. Men involved in mining and living outside of the community, were not being reached. Village-level activities involving youth groups, including youth associations and listening clubs, are designed to strengthen the psychosocial skills of youth and adolescents. Participants of listening clubs benefit from mentoring provided by a youth counselor who is trained and supervised by the polyvalent facilitator. Through weekly listening sessions of radio messages and discussions, young people of different ages and gender are exposed to gender-sensitive topics related to the consequences of early pregnancy and marriage, marriage-related legislation, and gender-based violence. They also receive information on the stages of puberty and what to expect as they experience emotional and physical changes. Listening clubs provide a good platform to discuss shared responsibility and joint decision￾making as youths prepare for the future. While message content is of high quality, the numbers of youth reached is limited. In some communities, opposition of mothers to their daughters’ involvement decreased female participation. It is unlikely that this type of sensitization and information-sharing approach will instill sociocultural transformation. Interestingly, in one village, the participation of girls was higher and even encouraged by parents. This community had an active anti-violence youth group, which was raising awareness on laws prohibiting early marriage and demanding arrests of men impregnating minors and girls out of wedlock and families enforcing early marriage. In addition, legislation to protect minors was being enacted by police to prevent early marriage and pregnancy. 3.2.2 Objective 2: Evidence of Changes A range of activities address systemic gender barriers, reaching a broad spectrum of community members. However, there is little evidence that Budikadidi is close to achieving the target that 80% of community members maintain positive attitudes and practices related to gender or that activities are impacting on long-term behavioral change. In this regard, FH couples commonly refuted that interventions are reducing gender barriers. For example, most women participating in group discussions reported that their husbands continue to give them little respect, highlighting that men do not value women and are opposed to shared responsibility and joint decision-making. Data triangulation confirmed that most men control household resources, even those earned by women, and continue to make decisions about household expenses, including money required for food consumption. Women respondents indicated that they conceal money from their husbands to ensure that the money is available for critical household needs. Men were reported to spend household income on alcohol and extramarital affairs. Kasai Oriental and Occidental have the highest prevalence of polygamy in the DRC at 31% (DHS 2013-14), occurring whether or not adequate resources are available. The fact that family clan members live nearby allows older members to exert pressure to adhere to family norms, perpetuating firmly entrenched obstacles to gender-based behavioral change. However, some signs of change were noted. For instance, a few women and men reported improvements in household communication and husband’s willingness to engage them in decision making, particularly related to the education, health, and nutrition of children, but not to household food security. In Kasai, men own agricultural land and make decisions regarding the types of staple crops to plant. Men are strictly in control of staple crop harvests, despite the fact that women do most of the field work, with the exception of the initial clearing of the fields. The firm role men play in controlling staple crop production will be difficult to change. Women, on the other hand, maintain gardens and control vegetable crops harvested. IMPEL | Implementer-Led Evaluation & Learning 22 Findings – Foundational Purposes During interviews, LMs did not cite objections by their husbands that they lead health, nutrition, and WASH activities, indicating that their husbands have given them freedom to circulate neighborhoods, which is unusual in this context. Through CAC structures, many women have become involved in governing structures and activities previously dominated by men. Several SILC groups are predominantly female. However, participants reported that money from savings and loans is more often used for immediate household needs such as school payments and medical care and less for creating small business activities that could impact on food security. Youth activities focusing on altering attitudes and practices leading to gender discrimination and facilitating healthy male-female interactions at a young age were implemented in all the villages visited. Youths interviewed demonstrated an understanding of and support for the messages shared. 3.2.3 Objective 3: Quality, Relevance, and Efficacy of the Design Budikadidi has combined multiple activities aimed to reach different sectors of the population to address gender barriers. Given the complexity of the problem, this mix appears to be appropriate, with most activities attempting to address key constraints women face. Female respondents of reproductive age and Budikadidi field agents interviewed generally reported that the activities were important and relevant, although implementation will be required over a long time to obtain results. Women participants stressed the importance of their involvement in household decision-making and approved efforts enabling them to have a voice. Lead Mother roles as CG leaders give women positive and high visibility. There were signs that some members perceive activities designed to alter the imbalanced power dynamics guiding relationships between men and women as disruptive, threatening, and negative. In particular, middle age or older men interviewed commonly viewed activities aimed to instill gender change as inappropriate. While Budikadidi reports about 25% male attendance, the vast majority of literacy classes in evaluation villages were attended by women, with men refusing to participate because they were embarrassed to be in classes with women or be seen by school children participating in literacy and numeracy sessions. Some mothers prohibited their daughters from participating in listening clubs. Selection of goats for distribution to SILC groups opposes gender equality efforts. In the Kasai context household resources including small livestock are controlled by men, although smaller animals, such as rabbits and guinea pigs, may be managed by women, boys and girls.1 Correspondingly, women respondents indicated that they prefer receiving rabbits, which have a short reproductive cycle, and can be used to supplement the family diet. In some communities, there is a high out-migration of adult male members engaged in mining not reached through activities. Activities aimed to reduce early pregnancy and forced marriage included listening clubs, strengthening youth associations, and the involvement of grandmothers. Involving grandmothers in counseling promoting later marriage and pregnancy of household youths coincides with their family role as youth advisors. Young males, who have been identified as proponents of gender equality, are using their leadership roles in youth activities to promote efforts to reduce gender inequality. While participation of girls in listening clubs was relatively gender balanced, in some villages, mothers discouraged the engagement of their daughters, stating that these activities would not yield positive results. 3.2.4 Objective 4: Coordination, Collaboration, and Convergence Activities addressing gender barriers are carried out through a consortium of partners with FH couples supported by REFED, CLUSA, and Caritas and RACOJ and CRS providing oversight to youth activities. At 1https://www.fsnnetwork.org/sites/default/files/ENGLISHBudikadidiGenderAnalysisFinalReport2017compressed.pdf Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 23 the local level, activities are supported and supervised by the polyvalent facilitators and CACs. The ET did not collect information on government collaboration on these activities. In the future, the project will train religious and traditional leaders on gender issues, and these leaders will be incorporated in community-based gender transforming activities. 3.2.5 Objective 5: Early Evidence of Sustainability Gender discrimination prevails in Kasai, with social norms guided by traditional patriarchal systems and customs ensuring male dominance and placing severe disadvantages on women. Deep-rooted social tenets dictate that women play secondary roles to men, enforcing restrictions on female movement and interactions with men, even involving women speaking in front of men. These gender-based social norms drive a preference for schooling of male children, causing females to have much lower school attendance and literacy rates. Socioeconomic systems also encourage female marriage, which is often forced, at a very young age. These traditional norms will require years to change and will likely need longer-term strategies. In other contexts where female discrimination is widespread, increasing school enrollment of girls and prolonging female education, improving opportunities for income-generating opportunities, and ensuring policies and programs designed to increase marriage age and reduce adolescent pregnancy, have been shown to decrease early female marriage and pregnancy and improve gender equality. Activities designed to address gender barriers that show signs of cultural change and broader sustainability include participation of women in governance bodies and the skills gained through literacy activities. The use of LMs to sensitize other women on health, nutrition, and WASH, gives a large number of women the opportunity to accumulate knowledge. The important responsibilities given to LMs to impart key knowledge give thousands of women positive visibility. It is less clear whether FH couples and listening club activities will instill long term results. In the context of reducing gender barriers related to food and nutrition security, Budikadidi has put an emphasis on training members of governing structures and participants, to ensure that activities are high functioning. Activities that require project investments and inputs, such as literacy groups, will likely discontinue. While activities such as literacy and CG involve local community members who are volunteers, it is questionable whether these activities will continue without adequate supervision. Therefore, community supervision structures should be established, and in the case of LMs, incorporated in government structures. To ensure sustainability, CRS should explore integrating LMs as RECOs in their villages. 3.2.6 Objective 6: Appropriateness and Quality of the R&I Approach The gender analysis provided critical contextual information regarding gender-related issues that affect household food security and nutrition. Results informed strategies designed to enhance gender equality related to intra-household decision making and modifications in the theory of change. The research highlighted the dominance of men in household decision-making and control of family resources, underlying the need to identify ways to improve female involvement. Younger married men demonstrated greater interest in learning about reproductive health and family planning and ensuring the health and wellbeing of their family members. The research showed an inferior status of women is associated with poor perceptions of their daily work, which primarily involves farming, household chores, and childcare, lack of female involvement in the cash economy, high rates of illiteracy, and local customs requiring that women receive permission from their husbands to engage in activities outside the household. Inability to read and write coupled with restrictions on movement limits women's ability to engage in work other than farming. Findings guided the content of the FH approach and highlighted IMPEL | Implementer-Led Evaluation & Learning 24 Findings – Foundational Purposes the importance of literacy programs. The research underscored that women would be accepted in leadership positions, particularly related to agriculture, childcare, and religion. The gender analysis also showed that grandparents traditionally play a primary role in teaching adolescents about reproductive and sexual health, making grandmothers logical advisors concerning messages designed to reduce early pregnancy. Men (paternal uncles and fathers) are the prime decision-makers regarding early marriage of girls, which is driven by household economics and the rewards of dowry. While the gender analysis highlighted the danger of dowry, the project has not made attempts to alter dowry practices. Adolescent boys objected to gender inequalities and promoted the need to improve female rights. Also, adolescent girls were found to have high ambitions, which disregarded traditional gender roles. While power dynamics favoring men are deep rooted, these findings underlined the potential of youth as future change agents in gender equality. 3.2.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross￾Cutting Themes While governance activities allow women to participate in traditionally male platforms, women are relegated to roles that do not involve decision making. Moreover, most women living in rural areas are not well prepared to speak in male-dominant forums. Participating in SILC, OPs, and market gardens allows women to access financial and other resources independent of their husbands. However, at the time of the evaluation, agricultural activities have made poor progress, generating little revenue. The Real Father and Faithful Couples approaches and youth activities encourage male-female dialogue related to community matters and promote intra-household discussion and joint decision making. However, reducing gender barriers necessitates long term actions involving the development of income￾generating opportunities for women and improving female financial capacity and autonomy. It is unlikely that traditional practices related to early marriage and pregnancy will change without significant economic changes and enforcement of gender-based policies and programs. The late start of USAID IHP, which will promote modern family planning methods and provide a full package of contraceptives in health facilities, has held back progress associated with gender initiatives, mainly related to birth spacing and delay of early pregnancy. 3.3 SP F.3: Communities Are Resilient to Common Shocks The ET did not assess resilience activities specifically related to cross-cutting themes associated with gender or youth, so objective 7 (appropriateness and effectiveness of interventions focused on cross￾cutting themes) is not presented here below. 3.3.1 Objective 1: Quality of Program Service Delivery Seven types of shocks that could affect communities were identified through R&I studies, including inter-community conflicts, drought, torrential rains and associated phenomena, climate change, disease and epidemics that affect humans, animals and plants, and insufficient application of laws such as those related to the legal marrying age and migration. Budikadidi implements a combination of activities designed to mitigate shock and increase community resilience. The ET found that some communities had organized negotiation mechanisms for conflict management and mediation. In some 2019-2020 community action plans, a training activity on dispute resolution for CAC members had been implemented. Unfortunately, the ET was unable to assess the quality of the conflict management approaches. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 25 The ET obtained information on early warning systems of shock or disaster. The ET observed rain gauges in the zonal coordinating offices used to track rainfall in areas supported by Budikadidi. At the village level, LMs received training to take arm measurements and are successfully using MUACs to monitor the nutritional status of children between 6 and 59 months. Arm measurements are also carried out by members of the CAC and RECOs, with children detected to be acutely malnourished referred to a health center. Nutritionally rich recipes and cooking demonstrations contribute to community resilience, although the delay of agricultural activities and limited linkages between P1 and P2 restricts availability of high-quality foods. As part of early warning systems, Budikadidi teams regularly collect prices of major foodstuffs in markets. However, this information is not shared at the community level. Hygiene, sanitation and water activities aim to prevent disease transmissible through oral-fecal and water routes that can lead to epidemics. Budikadidi has contributed to the rehabilitation of existing and construction of new water points and the formation and training of water committees on water source management. The ET found that the water management teams have been adequately trained and have the appropriate tools to function. Budikadidi has also established teams that routinely monitor community latrines and sanitation. While efforts in the hygiene, sanitation and water sector are notable (see SP 1.4), Budikadidi is confronted with obstacles to meet the high demand for potable water including difficulties in reaching aquifers for the drilling of wells, inability of the newly installed water systems to meet demands due to the technology used and population density, and non-use of newly installed water points due to costs or time involved in obtaining water. The evaluation also found unwillingness of some communities to build latrines, particularly improved latrines. Uptake of handwashing stations is poor due to lack of durability or the stealing of hardware. Budikadidi played a critical role in prevention and control efforts related to the 2018 cholera epidemic that occurred in project health zones and the project had developed a well-coordinated strategy with the local government and other implementing partners to address future outbreaks. . Complementary agriculture activities, including OP groups, market gardens, permagardens, animal husbandry, and value chain are intended to diversify food production, contribute to income, improve nutrition, and increase resilience, but are struggling to reach scale and viability (see SP 2). A range of factors contributed to a slow start, including environmental compliance delays, procurement challenges, technical staffing changes, late distribution of seeds, inadequate skills of field staff, and activity design. Budikadidi is promoting community-based natural resource management involving training and promotion of improved techniques aimed to ensure sustainable and diversified agricultural production, such as use of locally available fertilizers and diversified cropping systems. Delays in the provision of seeds adapted to local agricultural conditions and preferred by households has limited crop production. Also, participants often failed to follow appropriate techniques; for instance, permagardens were not dug at the appropriate depth and poor water management has impacted on market gardens. During the dry season, many permagardens were abandoned due to inaccessibility of water sources or the infiltration of domestic animals. Rumors suggesting that PGs, which are shaped like tombs, were linked to high child mortality, have gone unaddressed and led to abandonment. Some field agents supervising agricultural activities indicated that they had not received adequate training and could not therefore provide appropriate information or good oversight to ensure improved practices and sustainable techniques are followed. Many CAC officials and OP and market garden participants claimed to be unaware of key messages related to sustainable agriculture and resilience, such as discouraging slash and burn techniques and alternating crops. Other important technical approaches such as composting of organic materials and integrated biological pest control had not been introduced in the evaluation villages. These interventions still require extensive Budikadidi technical inputs, materials and supervision to reach project objectives. IMPEL | Implementer-Led Evaluation & Learning 26 Findings – Foundational Purposes Failure to consistently distribute locally preferred animals (rabbits or guinea pigs) for household breeding has undermined resilience efforts. Prior to the evaluation, Budikadidi distributed goats purchased outside the area, with a large percentage decimated by disease. The project is currently working to strengthen value chains which involve SILC groups investing in livestock, marketing agents participating in the purchase and sale of animal feed, and training of community actors in provision of veterinarian services. In one village where Budikadidi had trained a veterinary assistant, community members were unaware that veterinarian services were available. SILC groups have received adequate training to be functional in villages across the project communities. SILC participants reported that SILC credit is more often used to provide small loans for immediate family needs and less frequently used to start income-generating projects critical to livelihood diversification and resilience. Overall, Budikadidi has been unsuccessful in implementing key agricultural activities and diversifying livelihoods to allow community members to absorb shock. Budikadidi staff realize the fragility of the situation and are working to improve resilience activities. 3.3.2 Objective 2: Evidence of Changes All villages visited had functioning CACs, which provided central guidance for resilience activities. Nutritional surveillance and activities aimed to improve maternal and child feeding practices are being carried out by LMs representing over 341 Care Groups. While water points have been rehabilitated and new water points installed, supply of potable water is insufficient, forcing communities to continue to access unsafe water sources. The ET noted that latrines are being built, but uptake of improved latrines is slow due to social norms and economic constraints. Handwashing stations commonly lacked ash or soap, thus negating improvements in handwashing behavior (see section on WASH). In the majority of villages, participants had initiated permagardens, but many were subsequently abandoned. Several villages visited were employing techniques that opposed techniques promoted by Budikadidi to improve soil fertility and increase crop production such as slash and burn. Budikadidi trained several auxiliary veterinarians to treat animals and establish linkages with veterinarians and businesses selling animal products, but they do not appear to be widely used by participants. SILC groups have been developed, with over 40,000 participants, and show signs of autonomy. Participants in some villages reported receiving guinea pigs, rabbits or goats distributed by Budikadidi, and in some instances, community SILC groups initiated raising animals on their own. Value chains are being strengthened, but limited progress in OP and market garden production has restricted their effect in improving market systems. The ET did not find evidence that enforcement of laws related to issues such as early marriage or migration, identified as one of seven potential shocks, are being addressed. 3.3.3 Objective 3: Quality, Relevance, and Efficacy of the Design The combination of interventions established for resilience is well designed to help absorb the potential shocks identified. For factors that cannot be addressed through project interventions, such as climate change or drought, monitoring systems have been put in place to identify warning signs and evaluate their potential impact. Environmental protection efforts such as reforestation and promotion of improved stoves are appropriate and highly needed measures to slow deforestation. Children identified as acutely malnourished are sent to health centers for further assessment and treated for childhood illnesses. Most health centers in the project area do not have regular supplies of food supplements (Ready-to-Use Therapeutic Food (RUTFs)), Participants were discouraged that Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Foundational Purposes 27 malnourished children referred to health centers were not given RUTFs, which are locally perceived as “treatment’ for malnutrition. WASH activities such as improved access to potable water and achieving ODF status, which is 80% in project communities, is critical to disease prevention and control, including cholera, which is endemic in the Budikadidi region. The mix of agricultural activities is critical to increased food production, but requires much additional technical oversight and inputs to be effective. When PGs and market gardens were successfully planted, participants noted their importance in terms of household food consumption, and in the case of market gardens, revenue. In some villages, poor planning of market garden production caused overabundance of certain vegetable crops, leading to reduced or no profit. SILC groups initially experienced problems with non-reimbursement off credit, but subsequently received training on risk analysis aimed to address the problem. At the time of the evaluation, participants claimed that SILC activities had been primarily used for immediate household needs, but less to provide credit to launch small business activities and diversify livelihoods, which is essential to resilience. Animal husbandry must focus on preferred animals with a short reproductive cycle to enhance resilience. Some early warning initiatives, such as the installation of a rain gauge or collection of market food prices, appear to be designed to maintain government services and have not been achieved or used by communities. A significant challenge is the involvement of local government actors in activities, but also the transmission of information such as data on nutritional status, rainfall levels, agricultural production, and market food prices to government sectors. 3.3.4 Objective 4: Coordination, Collaboration, and Convergence Consortium members bring different expertise to work in an integrated fashion to implement resilience￾related activities. Coordination is realized through planning and monitoring meetings at different levels of the project organization. Budikadidi has carried out most interventions aimed at improving community resilience through CACs and their training and activity implementation. Local-level nutritional surveillance data is collected by LMs, RECOs, and CAC members, but it is unclear whether data are transmitted to health services and utilized by Pronanut as planned. No mechanism has been established to facilitate the transmission of other early warning system data such as rainfall data or food prices to government entities. Collaboration with local agricultural agents and sharing of agricultural data is also limited. SILC data should be transmitted to the rural development or small enterprise ministry; however, there are no representatives of these ministries at the local level. 3.3.5 Objective 5: Early Evidence of Sustainability Resilience activities fall under several sectors, especially governance, health, and agriculture. Sector level training of participants and capacity building of CAC structures in the execution of activities aims to enhance sustainability. A formal, coherent development sustainability plan is not in place, which limits the monitoring and evaluation of resilience activities. While Budikadidi updates annual sustainability plans related to different sectors in PREPs, the plans do not appear to be adequately conveyed to staff at all levels. Project activities could benefit from a well-documented plan that is communicated to staff at all levels. Budikadidi headquarters staff are familiar with the objectives of sectoral activities, which contribute to their focus on resilience and understanding of tasks that constitute elements of sustainability. However, specific questions were not asked as to whether field agents link their sectoral activities to community resilience. Government entities frequently claimed not to have been involved in the IMPEL | Implementer-Led Evaluation & Learning 28 Findings – Foundational Purposes implementation of activities and only to be aware of interventions through meetings or reports they received from Budikadidi staff, which has negative implications for sustainability. While the ET recognizes that, in a context where government salaries are frequently not paid or are very low, these reports may reflect the desire of government officials to benefit from per diem through participation in Budikadidi activities. Nonetheless, these negative reports do not bode well for long-term sustainability of activities. Participants reported that they had received training and appropriate support regarding management tools and inputs to accomplish tasks for activities such as CG, WASH, and SILC, preparing them to continue activities post implementation. Agricultural activities, a central component of resilience, were extremely behind schedule, and some inputs had not been adequately provided, restricting sustainability. 3.3.6 Objective 6: Appropriateness and Quality of the R&I Approach. Budikadidi conducted R&I studies to assess shocks that could affect communities and to guide resilience strategies. Seven types of shocks were identified, including inter-community conflicts, drought, torrential rains, and associated phenomena, climate change, disease and epidemics that affect humans, animals and plants, and insufficient application of laws such as those related to the legal marrying age and migration. The studies were of high quality and effective in identifying contextually appropriate and useful information to develop a combination of relevant interventions to address resilience. The TOC has taken into account ongoing changes related to resilience activities. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 29 4. FINDINGS – PURPOSE 1: CHRONIC MALNUTRITION IN CHILDREN UNDER FIVE SUSTAINABLY REDUCED This section presents findings on the evaluation objectives related to SP 1.2 and SP 1.4. Evaluation findings from SP 1.1 (early pregnancies and forced marriages reduced) are presented under SP F.2. Additionally, as discussed in the limitations section, SP 1.3 (all household members make use of high quality, accessible health services) was not evaluated as IHP, who has a mandate to improve the quality and accessibility of health services, had not yet begun at activities at the time of the evaluation fieldwork. 4.1 SP 1.2: Households Practice Optimal HNW Behaviors During the 1000-day Period 4.1.1 Objective 1: Quality of Program Service Delivery. Participants reported CG as their primary channel for receiving health and nutrition content from the project; most also had attended one or more cooking demonstrations. However, as it was not possible to observe cooking demonstrations during the evaluation, the quality assessment focuses primarily on the CG strategy. Budikadidi’s activities adhere well with CG standards2 . Importantly, the ratios of households per LM and the size of CG are well within norms, enabling participatory and engaging CG meetings, frequent home visits (95% of targeted households are visited monthly), and routine supervision (promoters are supervised at least monthly, while lead mothers are supervised about twice a year). Most LMs are equipped with visual materials (boites a images) to support their household counseling work; materials are ample, and images are large enough to use in a group setting and well designed to show the behaviors; however, some images are blurry. In a few geographic locations, LMs have yet to receive all their materials due to procurement delays and the increased number of LMs compared to original estimates (the number of households in the project area has been increasing with the influx of people returning to the community after fleeing conflict) – these gaps affect less than 10% of LMs. The project management was aware of the gaps and working to rectify them as quickly as possible. The program team and communities -some of which have taken the initiative to enroll new families - are to be applauded for their efforts to ensure the core strategy grows with the population and that newly created families (due to new marriages), returnees, and other migrants are included as activity participants. CG meetings observed as part of the evaluation were engaging and highly participatory. Promoters particularly appreciate receiving copies of the lesson plans, which they say improves fidelity in their sessions with LMs. The facilitation modules are well designed to include not only information transmission but also a dialogue around likely barriers in the community (identified through the barrier analysis) and how they can overcome. However, in practice sessions seem to emphasize "what to do" more than "how to do it." Examples of very effective small changes – such as stirring a spoonful of palm oil into vegetables in a palm tree dense community or planning fieldwork to accommodate exclusive breastfeeding – were mentioned by some LMs and participating households. It was not clear whether 2 http://caregroupinfo.org/wp-content/uploads/2014/10/care_group_criteria_november_12_2010-1.pdf IMPEL | Implementer-Led Evaluation & Learning 30 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced these were their ideas or were coming from the project. Despite these positive examples, participants much more frequently cited basic knowledge, rather than practical changes, as the key things they learned from the project. Overall, households showed very strong familiarity with three of the project's five priority nutrition behaviors (dietary diversity, separation of children from animal feces, and exclusive breastfeeding). Households cited the other two priority behaviors (consumption of iron-rich foods and birth spacing methods) less frequently. As mentioned above, the project opted to deliver birth spacing methods through the Faithful House platform as an option of methods requires both husband and wife's participation. However, Faithful House operated at a relatively small scale compared to CG in the project area; therefore, these messages had limited reach relative to those transmitted through CG. The project team is collaborating with IHP, which will be targeting the couples promoting natural family methods with training to promote and refer couples for additional types of family planning services. IHP is also providing additional birth spacing activities, including supplying health centers with modern methods entailing the full government family planning package of methods throughout the project area. One promoter began the meeting by asking about recent events in the community and challenges the LMs were facing, followed by a group dialogue to identify solutions to challenges raised by other LMs. This was an excellent practice that offers a strong opportunity for LMs to build problem-solving skills. While nearly all project promoters are RECOs with years of experience in health promotion, there is variable quality among promoters, particularly in terms of their engagement, with some promoters skipping meetings or not providing the expected amount of supervision. Low engagement occurs in part because project promoters serve on a part-time volunteer basis, making it difficult for the project team to enforce work responsibilities as stringently as with staff. CRS has established reasonably robust supervision systems – including: Supervision of LMs by promoters. Each promoter is mandated to supervise 10 LMs every month, which, given about 60 LM per promoter, equates to two supervision visits per LM per year. In practice, some promoters do more supervisions, while others do fewer. Supervision of promoters by field agents, each field agent supervises each promoter monthly. There is a high adherence to this schedule. Supervision of field agents by project supervisors, each supervisor supervises each field agent monthly with high adherence to this schedule. Periodically (about once a year), the project also specifically invites the local health system to participate in these supervision visits. Project staff have developed and use paper-based supervision/quality improvement checklists, based on tools developed by the TOPS project, which score supervisees on a range of facilitation skills such as beginning with a recap of the previous session, asking questions, ensuring participation of everyone, providing complete and correct information, etc. Supervision scores are regularly tabulated and reviewed by project management to identify trends but have not been used to separate volunteers who are not performing/improving. All promoters interviewed as part of the midterm evaluation were able to name not only their field agent, but also the zone supervisor, and in many cases the Caritas and CRS Nutrition managers based in Mbuji Mayi. Based on the responses of supervisees, the quality of supervision varies a bit across the project: some personnel receive quality supportive supervision characterized by observation and private feedback on strengths and areas to improve, while in other cases supervision visits are dedicated to correcting or adding on to the supervisees work. Current supervision visits are always announced in advance and never unscheduled, a practice that project management recognizes limits utility of supervision, particularly with promoters who may have low engagement. At the time of the evaluation, project management was considering the feasibility of introducing unscheduled supervision. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 31 While the CG structure is in place across the project with a seemingly strong foundation, there are varying levels of engagement of LMs and participating households between communities. The project team recognizes some missteps that have led to the demotivation of LMs in some areas. For example, at project onset, not all communities adhered to the idea that the households should select their LMs (some community leaders selected LMs), which led to LMs not being recognized or respected by their peers as leaders. This problem was quickly identified by the project team (after about 10% of communities started CG) and strong efforts were made to ensure selection processes were followed correctly in subsequent communities. However, LM selection was not systematically redone in all areas where it was not conducted according to plan3 . Later, the project provided fabric for LM uniforms that led to community jealousy and backlash against LMs in some areas, with participants refusing to welcome LMs for home visits, believing they were benefitting unfairly from the project or withholding benefits that should go to households. These issues may be contributing to varying satisfaction among LMs; the quality and engagement of promoters may also be an essential factor, meaning where promoters are less engaged, the performance and motivation of LMs is likely negatively impacted. The different incentive structures across project sectors also appear to be undermining LMs motivation. LMs feel demotivated by their relative lack of incentives compared to other community-level project agents. For example, "local experts" were paid for PG construction, which some LMs perceive as unfair, given that they did the work to convince households to become involved in PG. Another example is youth leaders who receive transport subsidies to attend full-day training, while LMs invited to short local sessions where transport is not necessary. LMs in many communities indicate that their supported households believe they are receiving benefits from the project (though LMs are volunteers) and that the benefits are not shared with supported households. In some communities where this has been particularly acute, LMs have resigned or stopped conducting home visits as they were being turned away by families. At the household level, due to women’s heavy time burden for domestic responsibilities including farm work, cooking, cleaning and childcare, and limited ability to attend events without their husband’s permission, the project team made a strategic decision to focus on home visits, with neighbor women’s meetings organized less frequently (about once per quarter). While this appears to be a well-informed decision taken on the basis of the project’s initial gender analysis, it is likely leading to trade-offs as neighbor women do not benefit as much from the support structure provided by their peers and the fun, engaging atmosphere of a well-executed meeting. 4.1.2 Objective 2: Evidence of Changes Familiarity with the project-promoted health and nutrition information across the project communities is striking – even in communities where LMs report challenges with peer households. Participating households cited nearly all project lessons, including hygiene/sanitation (one of the most frequently cited), dietary diversity, use of health services for women and children, reduction of women's workload during pregnancy, healthy timing and spacing of pregnancy, and infant and young child feeding practices (especially exclusive breastfeeding, timely introduction of complementary foods, diversity). Content on meal frequency for children and the importance of iron for pregnant women is rarely cited. Beyond community members, Promoters (as mentioned above, often RECOs) and project 3 In what felt that households would not feel comfortable choosing other women than those already selected by their community leadership, and that encouraging them to do so, could sew discord in an area with already tenuous social cohesion following recent conflict. IMPEL | Implementer-Led Evaluation & Learning 32 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced field staff have also benefited from this increased knowledge that they will be able to continue to disseminate beyond the end of the project in subsequent roles. Households report they have adopted improved practices, particularly use of health facilities, exclusive breastfeeding, etc. These are not visible like the WASH activities that involve infrastructure development (latrines, dish drying racks, tippy taps), which is seen across communities. However, the upcoming annual survey will provide an opportunity to confirm uptake of practices. While households report improving many practices, they consistently cite that they are struggling with recommendations on dietary diversity and meal frequency. Most people report eating and were observed preparing only foo-foo and cassava leaves. Likewise, participants report feeding children only 1-2 times per day, which is consistent with DHS findings that only 1 in 5 children in the region receive minimum meal frequency. These levels of diversity and frequency are far below nutritional recommendations. Participants indicate that poor access to food (production levels and income) presents a significant obstacle to dietary diversity and meal frequency (time burden is also cited as a challenge for meal frequency). Most participating women report that they have not received project support to increase their food production or income. They do recognize that they have benefited from PG, and say they have been helpful, but do not feel this is sufficient to address their food needs. Likewise, many are members of SILC groups but have not (yet) seen substantial impact on their food security from these activities. Only a minority of women who participated in focus group discussions as part of this midterm evaluation were members of the OPs or indicated that they had learned an agricultural technique that could meaningfully increase food production. 4.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design Nutrition is a clear need in the Budikadidi targeted communities, with a majority of children experiencing at least one form of malnutrition and limited practice of recommended nutrition￾supporting behaviors in households and communities. While there are periodically short-term emergency nutrition projects, and some health systems support, Budikadidi is one of the only actors supporting improved nutrition at the community level on a long-term basis. The Budikadidi theory of change is quite robust and outlines many relevant pathways for improving nutrition and health outcomes. Each of these pathways appears to be relevant in project communities. However, the theory of change does not have a clear way of identifying the relative weight/importance of each of the pathways in addressing the problem. In selecting approaches, the project worked to cover pathways and, tried to ensure some balance. This approach does not necessarily help prioritize the pathways that are likely to have the most significant impact on nutritional outcomes. I And, though the theory of change is very complex, it lacks detail on the specific behavioral changes that are needed – for example it refers to “essential nutrition actions” without disaggregating how each may have different drivers. While Budikadidi did identify priority behaviors, they seem to have been selected in the interest of hitting each of the theory of change pathways, rather than based on relative importance of the pathway (or behavior) to achieving high level impact. In particular, the project may be inadvertently underemphasizing infant and young child feeding practices (which have a strong body of evidence to support their direct impact on nutrition) in order to ensure balanced coverage for practices where the pathways to improving nutrition are longer or evidence is mixed (e.g. prevention of environmental enteropathy). For example, the project has prioritized diversification of young children's diets as a key behavior. However, there are also large potential (and perhaps more rapid) gains that could improve feeding frequency/quantity (which are not as dependent on purchase/production of new food items). The project may be able to achieve greater gains by focusing on with more intensive efforts on the pathways/behaviors that show the most promise and Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 33 opportunity and sequencing and layering behaviors so that families can master a few new practices before being asked to make further changes. Content of health and nutrition messages is relevant and generally robust; however, as mentioned above, more attention could be given to promoting practical small changes that families can make (less attention on what to do, more attention on how to do it). Also, despite prioritizing just a handful of key behaviors in their social and behavior change strategy documents, content for CGs tends to be more comprehensive and knowledge-focused rather than intensively addressing the few priority behaviors in depth. Compared to hygiene promotion work – which had a specific community engagement effort through Community-Led Total Sanitation – nutrition has generally drawn less community-level enthusiasm as the content is delivered one-on-one to households without an explicit community forum. While this approach has managed impressive reach – ensuring that nearly every household is benefiting from health and nutrition activities – it has created less of a buzz than hygiene/sanitation and could benefit from more engagement of community leaders. LMs are represented CACs in all communities, and some indicated that CACs are working on health/nutrition. In some communities visited, LMs indicated that health and nutrition topics were not part of routine CAC meetings as other project elements were, a potential missed opportunity which may be linked also to the ability of women in some communities to exert public leadership. 4.1.4 Objective 4: Coordination, Collaboration, and Convergence Budikadidi coordinates with the regional Pronanut office and regularly attends the multisector coordination meetings and Division Provincial de Santé quarterly planning/review meeting in Mbuji Mayi to collaborate with other nutrition actors. The project works with the local health systems – primarily the Medicins Chefs de Zones (MCZs) and the Animateurs Communautaires (ACs) working in the Bureau Centrale de la Zone de Sante (BCZ) – in the project's health and nutrition activities. The MCZ and ACs have participated in project training, helped to develop and validate curricula and modules for CG, serve as co-facilitators for training promoters, and periodically participate in supervision of field activities. In some areas, Pronanaut provincial staff and health zone personnel were also involved in the studies conducted as part of the refine and implement process, including inputting into protocols and attending restitution meetings. Pronanaut personnel have participated in the review and validation of SBC approaches, including health and nutrition messages. Some health zones have recently engaged nutritionists, who have not yet been integrated systematically into project activities. Budikadidi also provided a 4-day training to 94 health facility personnel (Infirmiers Titulaires and Infirmiers Titulaires Assistants) from 47 health facilities on providing quality essential care for young children in line with national protocols, and provided refresher training to over 2100 ReCos on themes relevant to the project (such as MUAC screening, hygiene, and nutrition). While the project left the selection of promoters to communities, the overwhelming majority of promoters are also RECOS, which means the additional capacity created through the project will remain within the health system. Health facilities and health zone management do not report systematically receiving project data or reports, which is a missed opportunity. Quality of government health services is notoriously poor, which is likely affecting the utilization of health facilities and the quality of care received by those participants accessing government health services. The project has been proactive in coordinating with the USAID IHP project, including holding frequent coordination meetings in Kinshasa and Mbuji Mayi, ironing out plans to bring complementary rather than duplicative activities through a memorandum, and conducting joint field missions. Though the team has felt frustrated at times by the slow start-up of IHP activities on the ground, the overall collaboration approach is a real strength and offers USAID strong value. However, the team has IMPEL | Implementer-Led Evaluation & Learning 34 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced struggled to coordinate with humanitarian actors operating in the same health zones (except cholera focused programming where collaboration was felt to be strong). In addition to emergency food security projects offering food distributions, there have been many short term (often just 3 month) emergency community management of acute malnutrition (CMAM) projects providing ready to use therapeutic foods. Health zone leadership has the responsibility to ensure the coordination of all actors operating in the zones. However, in practice, they are often not proactive in communicating about the existence of other actors in the areas (perhaps out of fear that one actor may choose to shift targeting if needs are perceived to be covered). Because the emergency projects are short term, they may end before agreements reached on models of coordination. Divergent approaches to resource transfers across donors and weak coordination has threatened Budikadidi’s success in several ways. Humanitarian actors are engaging and offering financial incentives to LMs and promoters (while Budikadidi does not) but also by shifting community attention to only acute malnutrition (not chronic malnutrition as Budikadidi seeks to address) and generating demand for external food commodities to treat malnutrition (instead of locally produced food to prevent malnutrition) and reinforcing handout mentalities. The project team continues to strive to improve coordination directly, and by encouraging health zones to facilitate coordination, Budikadidi has also shared the coordination gap with IHP, which has a systems strengthening mandate. 4.1.5 Objective 5: Early Evidence of Sustainability Community members, front line staff, government stakeholders, and senior-level management and technical staff are all aware of Budikadidi’s status as a project with a fixed end date. Households will retain the knowledge they have gained through the project, and several improved behaviors are likely to continue without additional support (women, for example, cite exclusive breastfeeding as a practice they will replicate with future children regardless of project intervention). Budikadidi has maximized the likelihood that practices will be sustained by selecting behaviors that do not rely on external inputs. Many LMs report gaining confidence in themselves and their ability to contribute to their communities, which are also long-term benefits that project will leave behind. In addition, Budikadidi hopes that there will be some continuation of LM activities beyond the life of the project. Senior-level management and technical staff see the sustainability of LM work as built on four pillars: 1) capacity, 2) resources, 3) motivation, and 4) linkages. For capacity, Budikadidi is relying primarily on building the capacity of LMs to continue sharing knowledge and skills beyond the life of the project and providing them with the resources (such as boites a images) they need. The intentional approach of not providing financial incentives to LMs is part of this strategy to emphasize intrinsic motivation. LMs and Promoters serve as volunteers and cited several sources of motivation for carrying out and continue their work, including benefits, gained based on their increased knowledge on the health of their families and their neighbors, status associated with being perceived as a leader/role model. The Budikadidi team is working to strengthen linkages between LMs and their own communities – particularly CACs – for ongoing support and encouragement. This approach is showing early signs of success in some areas, with LMs and promoters citing CACs as a source of support and oversight for their work. However, in other communities, CACs have not yet been as involved in leadership on health and nutrition issues as other community activities. LM Linkages with RECOs and linkages between RECOs and health facilities also offer the potential for continued transfer of new information and updates through LMs, though experience in DRC suggests this is unlikely to occur beyond a few isolated cases, without additional external impetus. Management and technical staff recognize that not all LMs are likely to continue their work, but point out that even continuity of 30% would be an impressive asset to communities. Budikadidi seems to Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 35 have reasonable expectations, given that it will be unrealistic to assume the health system will continue training or supervision at current levels, or be able to provide new materials to LMs. Some communities are already taking steps to sustain and expand CG activities, for example, by identifying new LMs to replace those that have left and proactively identifying new households to enroll in program activities. The project team is in the process of assessing LM attrition rates and reasons. 4.1.6 Objective 6: Appropriateness and Quality of the R&I Approach The health and nutrition team commissioned three studies focusing as part of the Refine and Implement process: a local food study, a barrier analysis for key health and nutrition behaviors, and a study on the feasibility of fertility awareness methods to support healthy timing and spacing of pregnancy. These studies appear to have been highly participatory, with CRS project staff, partner staff, and government officials indicating that they were involved in one or more of the studies. The studies were used to help refine and finalize the project's theory of change. Across the board, the team indicates that the local food study was particularly useful and heavily influenced project activities, including recipe development and cooking demonstrations. The barrier analysis was used to inform the development of CG modules and other messaging, although front line staff seems less aware of these linkages. The project's approach to healthy timing and spacing pregnancy shifted from delivering fertility awareness counseling toward coordinating with IHP, who will be delivering a full package of family planning options. As a result, the study was less used by the project for direct delivery but was key in advocating with IHP for greater attention to fertility awareness methods in their package of family planning options. In addition to specific health and nutrition studies, the health and nutrition team was involved and very familiar with the findings from the Gender Analysis, and had used this to inform their project strategies. Some health and nutrition team members also participated in studies for other sectors, including the value chain assessment and resilience study. While the project team defines R&I as taking place during the first year of the project, the nutrition/health technical team leads nonetheless continue to embrace the spirit of “refine and implement” by routinely analyzing project data and feedback from the field, and revising strategies and approaches to obtain greater impact. They also continue to test new approaches and carry out learning. For example, they recently tested the idea of “Lead Fathers” and are critically evaluating its success. Also, as part of the project’s learning agenda, the project is currently advancing two additional learning studies with support of Tufts – one focused on social network mapping to better understand who within communities is benefitting from project interventions, and a second focused on making stunting more visible (through use of length mats or other tools). 4.1.7 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross￾Cutting Themes The Budikadidi Gender Analysis revealed a particularly challenging operating environment for gender transformative programming. The health and nutrition team worked closely with the gender team to digest the findings from the study and adapt their program strategies to be both gender-responsive and gender transformative. For example, given women's heavy time burden, and restrictions on their movements, the team opted to favor home visits rather than neighbor group meetings; home visits are seen as a potential pathway for including men in discussions, given their decision-making authority. The program team likewise intentionally began with themes – particularly hygiene – that experience suggested would be more likely to garner men's interest. Also, the project strove to integrate health and nutrition content in approaches designed to improve couples' communication and joint decision making IMPEL | Implementer-Led Evaluation & Learning 36 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced (e.g. the Faithful House). Recently, the project team experimented with streamlining FH and CG activities by trying to incorporate the spouses of LMs in CG activities as Lead Couples/Lead Fathers. While this refinement was intended to increase linkages between joint decision making and nutrition and expand reach by leveraging the CG model's impressive coverage, the project team reports struggling to garner much male interest largely because many men in the target areas are transient migrant workers in nearby mines and because health and nutrition are largely seen as women’s issues. Indeed, the few Lead Fathers that did attend the observed CG were not active participants, and it was unclear whether this approach would be successful. As of the midterm evaluation, most men in project communities do have some basic familiarity with the project's health and nutrition content (though much less than with WASH content), suggesting that some progress has been made. However, female participants and project staff continue to identify widespread challenges stemming from men's limited interest and leadership on health and nutrition. The project had not emphasized delivering male-focused content to men through channels that men already know and trust. Overall, male engagement in health and nutrition remains poor and there are opportunities to build on work already done while better tailoring content and delivery modes specifically to the needs and interests of men. While the project youth work housed under the "gender" domain, there were nonetheless good efforts to engage youth in health and nutrition activities. Youth specifically targeted for content about delaying marriage and pregnancy and establishing healthy gender norms. However, youth "listening club" themes also included core health and nutrition priorities. As a result, through CACs, LMs were also able to secure support from youth to provide labor for the construction of latrines, dish drying racks. In terms of governance, LMs are a standard constituency member on the CAC. Budikadidi has used the CAC platform to raise health and nutrition matters, but this approach has not always garnered the attention required to influence community decision making. This is perhaps because LMs are all women, and women in many communities have relatively little public voice and decision-making authority. They may not have felt empowered to raise issues and push priorities in CAC meetings. In the past 6 months, the project team has made intentional efforts to strengthen linkages between CACs and LMs, particularly to involve CACs in combating rumors negatively affecting LM work and motivation, and in seeking strategies to maximize sustainability of the LM strategy. 4.2 SP 1.4: Communities and HHs Maintain a Clean Environment Conducive to Good Health and Nutrition 4.2.1 Objective 1: Quality of Program Service Delivery The geophysical makeup of the soil strata in Mbuji Mayi province has made it difficult for drilling and finding water in some communities, particularly where the water table is very deep. The availability of advanced water infrastructure technology to pump water from more than 100 meters is expensive. Even if the mechanized submersible pump technology was available, the complex operation and maintenance (O&M) of this system would become financially burdensome for vulnerable communities to sustain without ongoing technical support. In all three health zones visited, the natural spring drinking water sources were located at the bottom of steep ravines, making fetching water arduous for women and girls. Natural springs within Mbuji Mayi province are the most common groundwater source, which communities customarily view as free even when they are captured and protected. As a result, households were not willing to pay for spring water at the beginning of the project, leaving no revolving Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 37 funds available for its maintenance. However, after the project improved water capture and provided water testing, water management committees were able to convince communities to contribute 50 – 100 CDF (.03-.06 USD) per household per month. All these factors have contributed to limited access to quality drinking water for a growing population at different stages of the project. Home-based water quality treatment is another challenge. Household water quality treatment was reported as difficult to follow due to the time and cost involved in boiling water and the cost of aquatabs (a tablet available to treat water). Although the project mentioned flocculation and solar methods as other household water quality treatment, no other water quality methods were referenced by either men or women during group discussions. Correspondingly, shopkeepers selling aquatabs stated that few households purchase the tablets. The project plans to complete 83 water systems by the end of the project period. As of August 2019, CRS has captured 35 protected springs and installed 4 boreholes fitted with foot pumps. Also, the project has completed its target of 48 hydrogeological studies. In the future, the project will expedite the remaining borehole construction using contractors and a privately-owned drilling rig. Newly constructed borehole-pump systems visited during the evaluation met the minimal sphere standards of 500-meter radius of the target communities or less than a 30-minute round trip for households. Water committees interviewed were gender-inclusive, with one having a female president. All water committees visited had already received basic O&M training and had established accountability and administrative structures. All committees visited were able to provide documentation demonstrating tracking of water tariff fees and water quality testing for boreholes and monthly contributions for springs. Each committee had a water technician trained in basic upkeep of protected springs or repair of installed foot-pumps. CRS is working with Vergnet Hydro, a French water pump & hydrology company based in Kinshasa, to provide in-depth training to water committees on complex breakdowns, which may require removal and replacement of the pump manual rod-transmission and parts. CRS is also working with Vergnet Hydro to develop a local supply chain for spare parts at the health zone level, so that water technicians can have quick access to spare parts when needed. While CRS should be commended for its efforts to improve drinking water access and sustainability in a context where environmental and political challenges are widespread, the ET identified several challenges that affect the quality and need to be addressed in the new fiscal year in coordination with the SNHR and DPS as follows: a) Population Burden: Large numbers of people utilizing a manual foot pump will lead to frequent breakdowns and may damage the foot pump so that it cannot reach its expected lifespan. CRS confirmed that foot pumps could support no more than 300 households. Two communities visited during the evaluation, for example, had more than 850 households in and around the area using one borehole-foot pump system. b) Sustainability of water repairs: At the time of the evaluation, water committees had received basic training in the manual O&M of the foot-pump and price setting for water points, but there was no training on complex breakdowns or water infrastructure lifecycle costing. Water committees had no plans for financing complex repairs. The existing plan is to inform the CAC to relay a message to Budikadidi that assistance with repairs is needed. While the WASH team lead reported that all water committees had the simple foot pump hardware tools and a first phase instruction manual, during the evaluation, the ET found that some water committees did not have the repair manual. c) Queueing time: The average number of people accessing water at each water point, combined with the low flow rate generated by manual foot pumps, resulted in an average queuing time IMPEL | Implementer-Led Evaluation & Learning 38 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced within one village of 3 minutes per 20-liter jerry can (9 seconds per liter). Due to the large demand for potable water in two villages where foot pumps were installed, the village chief stipulated that households can only take 20 liters of water per day from the borehole. . Households interviewed felt that they had no choice but to frequent unsafe sources to meet their household water needs. It is worth noting that if villagers were solely reliant on these two boreholes for domestic water needs, it would not meet the minimum standards for drinking water established by Sphere for humanitarian response.4 Figure 3: Jerry cans lined up at water source. Photo credit: Lauren Blum d) Construction quality: Trained masons in the communities completed well development (drainage apron, traditional fencing, soak-away pits, and gravel pathways) of all water points visited. All water points in the four communities visited were traditionally fenced in, and most displayed the USAID logo (in one instance, the logo was missing), water collection rules, and scheduled times to obtain water. Budikadidi should take note that one rehabilitated spring box was patched up with a temporary cement/clay soil mix near the eye of the spring, which will continue to leak if not properly repaired. e) Government and community preferences: In two locations where a foot-pump was installed, water committees had requested a water tap network but were told by project staff that the system is expensive and would require complex repairs in the long-run. The committees were informed that the foot pump was a less expensive option and required low maintenance. SNHR representatives agreed that foot pumps be installed to meet the high demand for water in the area. f) Accountability and Transparency: Water committees have established water tariff collection systems, which were informed by the training they received and discussions with CAC members and village chiefs, with water tariffs fixed at 50 CDF (.03 USD). There was no life cycle costing done with the water committees, and it is unclear how the tariffs were established. However, it is questionable whether the established tariff prices will be enough to support both immediate and long-term repairs. Village chiefs and CAC leaders shared water tariffs with community members during general assembly meetings. Some respondents expressed dismay that they were not involved in decisions related to tariff prices. In villages visited, the water committee claimed to be developing a system which would waive tariffs for the most vulnerable households. Water committees reported that these households had been identified. During 4 Sphere standards require at least 20 liters per person per day. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 39 household interviews in one village site where a water point was installed, the ET found that some families were still drinking river water because they were unable to afford the 50 CDF (.03 USD ) tariff fee. One community water committee though had already collected 132,000 CDF (79 USD) in their cash box after two months of water point operation. As funds kept in the village cash box accrue, the risk for theft or corruption increases. Figure 4: Water point schedule. Photo credit: Lauren Blum g) Water quality testing: The team plans to work with and through relevant government counterparts (chef de secteur/chefferie) to establish an accountability system to continue water quality testing. The system will require management of water quality testing materials and appropriate village-level tariffs to ensure testing continues. The WASH team aims to ensure that testing kits are available in all Budikadidi project zones, whether SHNR or the DPS maintain the kit. The project is hoping that the water committees will have enough funds to purchase chlorine solution if their borehole water quality testing fails. Budikadidi has not officially handed over the Wagtech kit to the health zones; the project is responsible for purchasing reagents for the water quality testing going into FY20 of the project. Plans regarding the purchase of water testing materials have been discussed with the DPS, which has established links with suppliers to purchase consumable products for the kit, including the reagent. The majority of Budikadidi participants have achieved ODF status, but there are environmental, economic, and social challenges that hinder these participants from continuing to move up the sanitation ladder. There were examples of the household's traditional wood frame platform being eaten away by termites, making the pit vulnerable to collapse. Social customs prohibit men from sharing latrines with their daughters-in-law. This may encourage families living in extended households to continue open defecation, despite having access to a basic latrine. Also, migration of males drawn to diamond mining, leads to economic instability, particularly when male household heads are absent from IMPEL | Implementer-Led Evaluation & Learning 40 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced the home. These factors have created major barriers to constructing an improved household latrine, thus increasing the probability of ODF relapse. At the commencement of the project, only 8% 5 of the target population had access to basic sanitation services as compared to the national average of 29%6 . Since the inception of Budikadidi, a total of 80%7 of the population have achieved open defecation free (ODF) status using the CLTS approach, but only 29%8 of ODF communities have improved latrines. Once households have achieved ODF status, natural leaders monitor latrines to assess whether they meet the criteria of a hygienic basic latrine (no presence of flies, odor, trace of urine, and a covered pit). Even with the uptake of the hygienic latrines, many earth floorings of the household latrines visited were not well compacted, and some latrines already seem to be getting filled with loose soil. After communities are certified by the village chief and the territorial administrator, households are encouraged to transition to improved latrines. The Budikadidi improved household (generation 2) latrine has a cement slab, brunt brick flooring and superstructure, a wood privacy door, and a thatch roof. The cost of the latrine slab alone is, on average, 6 USD per HH. During a focus group discussion with female participants, the ET learned that the WASH promoter did a community demonstration with the trained community mason on how to construct the latrine slab. Three demonstration latrines constructed were given for free to the president of the CAC, the chief of the village, and the mason. This caused resentment, with focus group discussion members contending that all community members should receive free slabs as well. Most households cited lack of affordability as the major reason preventing transition to an improved latrine, which cost 25,000 CDF (15 USD) altogether. During the evaluation, the ET found that some households visited for WASH interventions are not part of a SILC group. During interviews with some households located in both P1 and P2 communities, participants mentioned that the project has not spoken to them about joining SILC or sensitized them on how through SILC, they can work towards installing an improved latrine with a slab. However, one community visited was tapping into social cohesion networks to construct latrines jointly. In this village, the trained mason developed a concept called "dalle de SILC,” which involves encouraging households to jointly save their money to purchase latrine slabs at a reduced price (8,000 CDF, 4.82 USD), compared to the regular price of 10,000 CDF (6 USD). The difference between this methodology and SILC is that the household does not need to front money before joining. The need for seed money to join a SILC group is what has discouraged some from joining SILC. A WASH promoter in Kansansa is experimenting with the production of a cost￾effective and innovative 1x1 (ft) sandplat slab made of a composite of cement, termite mound mixture (clay, termite saliva, and dung), and water mixed with cassava residue. The slab will be burned into a solid composite and will cost less than three dollars USD, half the cost of the cement slab. The slab is in its early stage of trials and should be tested with water and human weight repeatedly for durability before decision to scale. Although local masons have been trained on how to construct and fix the price for improved latrine slabs, slow demand for latrine slabs has led to the demotivation of the trained masons. For instance, in one area, the WASH promoter stated that only three of seventeen trained masons were still active in the community. Masons expressed disappointment that the project did not provide seed money after the training nor training on how to market their service. The Budikadidi latrine training manual focused on hygiene and sanitation education, technical standards, and instructions for constructing improved 5 DFSA DRC baseline report, 2018 6 UNICEF in DRC Report 2018 7 CRS Budikadidi Presentation August 2019 8 ibid Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 41 household latrine and slab. The manual, however, does not include information on developing a business plan, product marketing, financing, or latrine options and costs. Handwashing at critical times and appropriate water quality practices, including using aquatab and boiling water, are key areas of difficulty households face. The cost of a tippy tap is 500 francs, but tippy taps are reported to be frequently stolen by children. This has discouraged families, who generally have limited incomes, from replacing them. Respondents in most of the villages visited stated that tippy taps leak after some time due to the direct exposure to sunlight. Households were generally using ash to prevent flies and odor in their latrines, but not many households made ash available near their tippy taps for handwashing. Although the Budikadidi team made a significant impact on water quality interventions during the cholera episodes in early 2018, many respondents interviewed during the evaluation were unwilling to boil or treat their water due to the work intensity and cost involved. Budikadidi water points that offer potable water are not enough to meet population demands. A large sector of the population still frequents unsafe water sources to avoid queues of two to three hours and to pay the 50 CDF (.03 USD) required to fill a 20-liter jerry can at the boreholes. The presence of WASH promoters living and working in and near communities has helped to spur on and sustain mobilization efforts of WASH interventions and practices. All villages visited confirmed that their WASH promoter frequents their community at least once per week. The WASH promoters meet with the Water Committees, LMs, and Natural leaders meet weekly to discuss and troubleshoot challenges. The WASH promoter also meets with the CAC to reinforce support to Natural Leaders and LMs. This strong coordination established between the WASH promoters, water committees, LMs, and Natural Leaders proved to be very beneficial during a Cholera outbreak that affected the project area in early 2018. During the outbreak, Budikaddi staff participated in the cholera response committee at the provincial and zonal levels. As part of this effort, Budikadidi provided health zone and provincial level maps identifying cholera hotspot target areas and interventions the project was carrying out. The project also worked with local health authorities to set up cholera treatment sites in different locations. Examples of specific Budikadidi interventions include provision of chlorine to health zones, mobilization of latrine construction, video, and radio communication on cholera prevention, community awareness￾raising on handwashing and ORS use, water point chlorination using trained LMs, and collaboration with health zones and other organizations on provision of water quality treatment. Using LMs and Natural Leaders to convey messages, CRS worked with Solidarite International on cholera control and prevention through hygiene education and awareness-raising on the importance of chlorination of household water. During the evaluation, male and female respondents seemed aware of the importance of WASH practices and the potential consequences of cholera if a household was non-compliant. Households in both P1 and P2 villages maintained clean yards, had tippy taps, used ash to mask odor and prevent flies in latrines, and demonstrated knowledge of handwashing with ash. Although WASH-related information disseminated at the household level, many critical and routine WASH practices still lag behind. For example, handwashing with soap is a practice that does not appear to be getting traction. During observations, the ET found that some tippy taps did not contain water, and soap was never available for handwashing. Soap vendors interviewed indicated that people primarily buy soap for washing dishes, clothes, and bathing. While ash was observed to be available next to latrines, ash does not appear used for handwashing. It is not clear whether ash is an acceptable alternative to soap in this context. The ET used the Aquagenx compartment bag testing kit to test both household and borehole water supply. Six households and two borehole water supplies tested for e-coli contamination. The two boreholes tested for e-coli passed the test. Four household water tests showed contamination; three IMPEL | Implementer-Led Evaluation & Learning 42 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced had obtained drinking water from a nearby river (stating that they had no option due to long lines, restrictions on borehole water quantity per household, and costs) and one from a borehole. The ET suspects that the household using sole sourcing from borehole contamination was caused by cross￾contamination of the recipient used, which is employed to collect both river and borehole water. The last two households where water collected from the borehole passed the e-coli test. 4.2.2 Objective 2: Evidence of Change The project has made significant strides towards improvements. As of August 2019, the project has reached 47% of its water supply targets and has achieved 80% ODF status. Budikadidi is working towards moving households up the sanitation ladder by encouraging members to become part of SILC groups. Most masons, LMs, Natural Leaders, and WASH committees interviewed were motivated and were active members of SILC groups. Youth were also active change agents and were assisting vulnerable households in obtaining a hygienic latrine and encouraging households to maintain clean yards. Budikadidi is working towards strengthening government ownership and coordination through collaboration with CNAEA and SNHR, which started at the inception of the project. Regarding water quality management, the project staff has coordinated efforts between SNHR, the zonal WASH leads, CACs, and water committees to maintain quarterly testing of target community water supplies. Budikadidi is working towards establishing a value chain for consumables needed for water quality testing and water supply spare parts from Kinshasa to the zonal health levels. Other changes that have occurred since the baseline include:  Percent of households practicing correct use of recommended household water treatment technologies has increased from 2% to 11%  Percent of households practicing open defecation reduced from 63% to 32%  Percent of households obtaining drinking water in less than 30 minutes from their homes increased from 43% to 62%.  Percent of male and female participants correctly citing critical times for handwashing increased from 50% to 91.7 %. 4.2.3 Objective 3: Quality, Relevance, and Efficacy of the Design The project approach to utilize the CAC, village chiefs, Natural Leaders, and LMs to mobilize change has been effective in increasing WASH interventions. The CLTS and Village Ecole Assainis (VEA)9 approach for P1 and P2 communities respectively, has made significant steps towards open defecation free communities. However, the improved latrine model presented as the generation 2 model seems to be more of a top-down solution rather than a human centered design approach. Only one model option is provided to the community as the choice of improved latrine. Communities should be allowed to discuss and design various options that can range in affordability, but still meet the criteria of an improved latrine. The tippy tap is also a top-down solution recommended for community adoption by project staff. During the evaluation, household members complained that tippy taps are frequently stolen, leak after sun exposure, and are generally not a sustainable option. Finally, the foot pump technology that is implemented was not selected by community members, nor were their suggestions regarding preferred pump technology considered. Due to the high need for potable water, communities have accepted the foot pump model. However, if a foot pump encounters major breakdowns--which are possible given 9 UNICEF, 2018, Programme National École et Village Assainis : https://www.unicef.org/drcongo/en/reports/atlas-2018 Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 43 the frequent overuse of the pumps--requiring greater funds than is available in the existing WASH revolving funds, there is a strong likelihood that the system will be abandoned. The project should look for ways to partner with government structures, UNICEF, IHP, and other USAID projects to transition the system into a phase 2 approach. Ideas to gradually transform the system into the standard local government and community water technology appropriate for the population size to increase the probability that both government and community will assume the long￾term maintenance of the system when Budikadidi ends. The project will also need to get a strong buy-in from SNHR to take full ownership of the existing water supply network once the project ends. 4.2.4 Objective 4: Coordination, Collaboration, and Convergence Budikadidi hosted a workshop, in conjunction with the Health and Water Provincial level Ministries in Mbuji Mayi to discuss WASH lessons learned, challenges, and best practices following government policies and government plans referred to in the Water Law and Policy document which describes ways to decentralize and strengthen water supplies. CRS should be commended for its efforts to ensure coordination and collaboration with other stakeholders working on WASH activities. Specifically, CRS has worked closely with SNHR from the conceptual phase of the water infrastructure activities. Budikadidi has also shared the results of the KAP survey to CNAEA and DPS as their rationale for promoting a modified VEA approach in P2 communities. CRS has been proactive in providing updates to the National Water, Sanitation, and Hygiene Action Committee (CNAEA) on Budikadidi water supply and sanitation improvements and challenges. At the provincial level, CRS has shared its hydrogeological studies with SNHR and has worked with SNHR water technicians to provide technical review oversight of contractors' work progress. CRS has trained staff from the DPS, health zones and SNHR, as well as members of the village CAC and water committees, on the Wagtech potakit (water quality testing kit) and has consistently conducted quarterly joint water quality testing of new and improved water systems. Budikadidi has presented the CLTS model to DPS and CNAEA representatives and has received approval to use this model in non VEA (clean village model) communities. CRS is providing updates related to sanitation issues, including the uptake of latrines using the CLTS approach, to DPS personnel every quarter. The project hopes to have the CLTS approach and its modified clean village model integrated into DPS plans and policies. At the community level, the project has been strategically coordinating with local structures to achieve water, sanitation, and hygiene access. Natural Leaders have been trained on WASH IEC tools to conduct house to house messaging. Natural Leaders frequently accompany trained LMs during house visits. LMs participate in community dialogue sessions led by Natural Leaders, and both work together, also in conjunction with listening clubs, to organize larger community events on WASH. An integrated module focusing on WASH activities has been developed and is being used by LMs during nutrition sessions with participant mothers. Also, project staff WASH promoters conduct coaching with Natural Leaders and LMs working on WASH activities once a month. Every quarter, there is a general meeting hosted by the CAC and village chief to review work plans, including plans related to WASH. Water committees meet monthly with the CAC and village chief to discuss water committee updates and any changes (e.g. water tariffs or rules at the water points) that may affect the general population. When change is considered, village chiefs assemble community members to get input before final decisions. At the time of the MTE data collection, the WASH team lead was in the process off meeting with USAID IHP to see how IHP could assist with improving the water supply and strengthen behavioral change approaches related to WASH. One approach under consideration was for IHP to remove the foot-pump fitting in communities where it has been installed and replace it with a submersible pump and a water distribution line/reservoir and distribution points. IMPEL | Implementer-Led Evaluation & Learning 44 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced At the time of report writing, USAID IHP had decided to prioritize construction of other boreholes in selected villages in Kasansa based on the mapping and hydrology studies conducted by Budikadidi. USAID IHP went through a bidding process and was developing a contract with the selected drilling company. However, it was still unclear whether USAID IHP will pursue feasibility studies to examine transforming established boreholes in Kasana to a mini-adduction system. Budikadidi has shared key technical information about specific boreholes in Kasana, as well as demographic data from villages surrounding the boreholes, to IHP. 4.2.5 Objective 5: Early Evidence of Sustainability Budikadidi has established a strong community-based structure for water. Both village chiefs and CAC members are involved in governance and accountability of the water supply. CAC members meet with the water committees monthly to discuss challenges and check their book of accounts. Water committees have been trained on basic pump mechanics, operations, and bookkeeping. CACs are responsible for monitoring the action plans of water committees. The project has been effective in equipping and transferring knowledge to Natural Leaders and LMs on WASH concepts and practices. The Natural Leaders and LMs can utilize the learning modules to articulate good WASH practices during household visits and community dialogue. The project has trained community masons who are known by community members to have the capacity to construct latrine slabs. The masons have also been integrated into well development projects to construct drainage aprons and soak away pits once the borehole construction is complete. Signs of social cohesion are evident in some communities where youth are being galvanized to construct latrines for vulnerable community members. The project has trained the water committee on operation and maintenance (O&M) and plans to strengthen the local water supply value chain so that water committees have access and can pay for point of use water quality consumables and water supply spare parts using funds collected through the committee. These efforts should work towards building trust with the community members and contribute to sustaining the water system. Budikadidi is working at the provincial, zonal, and community level to bring all relevant stakeholders together to manage water quality. CRS is providing updates to DPS and CNAEA representatives every quarter on water and sanitation issues, including the uptake of latrines using the CLTS approach. The project takes advantage of the influence of village chiefs and religious leaders to reinforce WASH practices, including latrine construction and cultural use. 4.2.6 Objective 6: Appropriateness and Quality of the R&I Approach The Budikadidi project used results from the baseline assessment and barrier analysis, including a KAP survey in P2 communities, to inform its behavior change strategy for WASH activities. The project planned and implemented CLTS in P1 communities and the Village Assaini approach in P2/UNICEF communities. The project will conduct a second KAP survey in FY19. This will reassess the 7 WASH norms10. This is associated with certification and developed by UNICEF, in conjunction with the DRC government, to determine what adaptations need to take place in the modified CRS Village Assaini model. 10 Active WASH committee structure, access to clean water, use of hygienic latrines, handwashing before meals and latrine use, understand fecal oral route, village is cleaned once per month. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced 45 Two positive changes that have occurred since the onset of the project related to the 7 WASH norms include the existence of community structure supporting WASH activities and widespread knowledge of fecal-oral route pathways. The team should further examine social barriers to latrine use and handwashing practices and consider exploring the acceptability of psychosocial drivers to latrine use and handwashing practices shown to be effective in development contexts11. Implementation plans addressed the behavior change strategy. Sessions related to handwashing, latrine use, and with the onset of cholera, water quality as part of the behavior change strategy, were developed and effectively implemented in the CG modules. There was low uptake of the Budikadidi home-based water quality treatment as most households were not willing to boil their water or purchase aquatabs. Solar and flocculation methods never reported as a practice in communities visited. The behavior change strategy does not discuss Natural Leaders, so this may have been added to bolster the approach and response to community needs. Budikadidi also plans to implement TIPS (Trials of Improved Practices) techniques to look at effective ways to increase uptake of WASH practices. Recommendation that TIPS examines social marketing for household sanitation and the existing handwashing strategy, including the hardware used. Water quality practices should be examined further as communities seem to be less willing to adopt these activities. The gender analysis report done in 2017 illustrated that collection of water is a chore reserved for women and girls in the Kasai context. Women and girls frequently have to ask male heads permission to go to water points, which can be more than 10 kilometers from households, highlighting the secondary role women and girls have in household decision making and general restrictions on female movement. Women and girls living far from water points have to leave the household as early as 4 am to fetch water; young girls reported fear of being kidnapped, robbed, beaten, or raped when fetching water. Young boys were reported to refuse to collect water or to accompany their sisters or mothers to fetch water out of concerns of being ridiculed. However, couples agreed to work together to acquire and maintain WASH infrastructure to reduce the fear and disproportionate burden placed on women. 4.2.7 Objective 7: Appropriateness and Effectiveness of Interventions Focusing on Cross-Cutting Themes The ET observations and interviews confirmed that women and girls were solely responsible for fetching water. Men were only involved in WASH-related activities considered masculine, such as building drying racks and latrines, with women respondents indicating that they had never seen a man carrying water. The evaluation results illustrate that gender roles related to obtaining water are firmly embedded in the cultural framework and will likely require major sociocultural and economic transformations before tenets are changed. Budikadidi has been strategic in addressing some of the 2017 gender findings by focusing on structural changes to address gender barriers and to lessen the risk of violence against women while obtaining water. Specifically, the project is installing water points within a 30-minute roundtrip radius from households. Correspondingly, the borehole water points visited were within 500 meters or less than a 30-minute round trip. Although this is a positive change, women and girls were still primarily fetching water. Some have designated women with leadership roles 11 Curtis VA, Danquah LO, Aunger R V. Planned, motivated and habitual hygiene behaviour: an eleven country review. Heal Educ Res. 2009;24(4):655–73. Aunger R, Schmidt W, Ranpura A, Coombes Y, Maina PM, Matiko CN, et al. Three kinds of psychological determinants for hand￾washing behaviour in Kenya. Soc Sci Med. 2010;70(3):383–91. Curtis V, Kanki B, Cousens S, Diallo I, Kpozehouen A, Sangare M, et al. Evidence of behaviour change following a hygiene promotion programme in Burkina Faso. Bull World Heal Organ. 2001;79(6):518–27. IMPEL | Implementer-Led Evaluation & Learning 46 Findings – Purpose 1: Chronic Malnutrition in Children Under Five Sustainably Reduced as members of water committees, typically as treasurers. Budikadidi staff reported that across project villages, four women served as president of the water committee. In one village, the ET observed two teenage girls, who were members of the water committee, in charge of supervising the water collection point and pumping water into the jerry cans. The president of this water committee reported that they have both men and women overseeing the pumping station. Regarding the sanitation infrastructures, some couples were jointly involved in constructing basic latrines. In many instances, women interviewed said they relied on a son or paid someone to dig the latrine and provide the superstructure materials. To increase latrine access for the most vulnerable populations, Budikadidi has involved youth as change agents. Members of listening clubs, for example, are helping the most vulnerable community members to construct basic latrines, encouraging households to maintain clean land parcels, and constructing dish racks. Concerning governance, water committees have had training on bookkeeping, documentation, and water tariff collection. There are always two committee members in charge of the cash box, and the water committee is required to report water updates, including the total water funds, to the CAC, village chief, and the community during general assembly meetings. As funds increase, Budikadidi may consider depositing funds in a bank to ensure greater accountability. The water committee is integrated into the CAC structure, and the CAC provides general oversight to the committee and WASH systems. “Men will never carry water, for other workmen can help, but not water.” - Lead Mother Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved 47 5. FINDINGS – PURPOSE 2: HOUSEHOLD INCLUSIVE SOCIAL AND ECONOMIC WELL-BEING IMPROVED This section presents the findings related to all SPs under purpose 2. These are:  SP 2.1 Men and women share household responsibilities and decision making equitably  SP 2.2 Household income increased  SP 2.3 Households have access to diverse, appropriate foods for all members at all times. 5.1 SP 2.1, 2.2, 2.3 Inclusive Social and Economic Well-Being 5.1.1 Objective 1: Quality of Program Service Delivery The permagardens (PG) method combines components of permaculture, which aims to utilize natural systems for production, and bio-intensive agriculture designed to increase biodiversity and maximize production using a small amount of land. This approach, which intends to ensure year-round production of vegetables by and for households, has been widely adopted by communities. However, CRS recent reports indicate that at least 15,600 permagardens have been implemented, which falls quite short of the 38,332 target. Gardens contribute to improved availability of vegetables and diversification of household food consumption. Commonly produced vegetables include amaranth, okra, orange-fleshed sweet potato, and moringa. PGs also contributed to increased income for some households through the sale of surplus production, according to information collected from participants. PG is hampered by some constraints such as such inaccessibility of water during the dry season in some environments, although communities located short distances from water points can produce vegetables year-round. Infiltration of animals during the dry season, when domestic animals are untethered and allowed to stray, also poses a challenge, but good fencing would solve this problem. The ET found that critical techniques essential for garden plot preparation were not always followed. For example, essential recommendations related to the depth (typically, PGs visited were only 20 cm deep rather than the recommended 40 cm) and soil preparation (combining soil with appropriate waste materials), were not followed, thus affecting crop production. The training (cascade) approach used by Budikadidi, which involves various layers of training of trainers, has likely diminished the transfer of technical capacities to field agents and affected the quality of gardening activities. In general, field agents demonstrated deficiencies in their understanding of the approach, with some having inadequate capacity to supervise activities effectively, convey technically sound messages, and address false rumors, causing some communities to abandon home gardens. For instance, one polyvalent interviewed, who had an educational background in civil engineering, admitted to having a limited understanding of agriculture. He reported that, while “Despite all of the efforts I made to train community members to set up home gardens at each household in our village, I was never given the 50 USD bonus promised by the project. My male colleagues from other villages have received payment. I have complained several times to the facilitator without getting a response. Why am I the victim of this discriminatory treatment? Is it because I am a woman?” - Local permagarden expert IMPEL | Implementer-Led Evaluation & Learning 48 Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved the project was providing agriculture training for participants, he had not been adequately trained, and thus could not provide oversight and respond to participant questions and needs. Incentive payments to local experts who were overseeing PG implementation created tension due to unequal pay by field agents to some experts and non-payment to others. The use of lowlands for agricultural activities has existed for a long time in the region. Traditionally, communities reserved lowlands for maize cultivation, which has a long crop cycle (four months) and requires a large area for good yield. The contribution of Budikadidi entails introduction of vegetable production in the lowlands. During the evaluation, approximately 250 farmer organizations with an average of 25-30 members were participating in market gardening, farming an area estimated at 162 hectares, with some associations already having harvested their first crop of vegetables and sharing revenue across members. Budikadidi has supported this activity by supplying 84 kg of vegetable seeds consisting of Chinese cabbage, cabbage, celery, okra, onion, eggplant, and pepper. The ET found that seeds were provided to about 20% of participants and mostly given to members of communities in the Kasansa health zone. This raised tensions within the project and across communities, causing community resentment and opposition towards the project in villages where seeds were not received. A positive aspect of seed conservation is that those participants who received seeds were keeping them in Purdue Improved Crop Storage (PICS) bags or special storage sacs offered by the project and will be able to multiply them in their fields in the next agricultural season. This was the case of one OP visited, which had produced and stored 30 kg of cowpea H204 and 270 kg of MUS I corn. The ET uncovered poor respect of the agricultural cropping system calendar, such as late promotion and distribution of improved seeds and late planting. This can affect crop production, which, when harvested, may not be ready for storage, and increase pest infestation, causing a waste of valuable resources and the need to increase additional inputs, all of which can harm the environment. Seeds could not be multiplied for widespread distribution because the SENASEM seed certification and approval processes were not followed. In general, multiplication and distribution of improved quality seeds had gotten off to a slow start, in part due to delays in completing environmental compliance requirements of USAID, resulting in inadequate distribution. Those who had not received seeds were discouraged because they had prepared the land several times without receiving seeds and did not know if they would benefit from improved seeds for the next agricultural season. Lack of availability of seeds in some areas made it difficult to assess the impact of the market gardens. Implementation of market gardens have encountered other problems, some of which link to lack of adequate planning for development of lowlands and insufficient technical monitoring of activities. Budikadidi failed to develop a management plan to address key technical issues to consider when implementing lowland agriculture, such as a strategy to address water management. This has led to major challenges, such as the flooding of lowlands during the long rainy season (nine months), which at times can bring torrential rains, destroying elevated beds and excessive moisture causing disease in plants. The total absence of water during the dry season (three months) has caused plants to wither and led to the propagation of insects responsible for different plant diseases. During the evaluation, which was carried out during the dry season, ET members visited market gardens suffering from significant “We are tired of the promises made by Budikadidi related to the distribution of maize and cowpea seeds that they would give us. We missed two agricultural seasons due to these broken promises. We wasted our energy preparing fields, 2 hectares per season. How much time have we lost?” - OP President Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved 49 water deficit and experiencing these consequences. Without rational management of water, this cycle of water related barriers will continue. There were many signs that the technical supervision of market gardens was inadequate. For instance, the ET detected non-compliance of technical standards in establishing elevated beds and structures set up to provide shade for nurseries and irregular watering of nurseries, resulting in the death of seedlings, attacks of pests, and delay in plant growth. Some participants were transplanting seedlings to lowland fields beyond the recommended number of days, thus requiring transplanted plants a protracted time to recover, reducing yields, and prolonging the crop cycle. Promises made by the project to provide seeds and materials were often not followed. In several cases, the ET found that an agreement between the farmer's association and landowner for the use of lowlands for market gardens was not signed. This situation led to the eviction of farmer associations when crops were mature, and in two villages the team visited, appropriation of the crops by landowners. There were also instances where the contract drawn up between the landowner and the association ended before the project cycle was scheduled to conclude. Budikadidi has introduced several innovations to improve agricultural techniques. This coincides with the project target that 70% of farmers have applied at least 2 sustainable crop practices or technologies in the past 12 months. However, the ET found that these techniques were not introduced consistently in all communities visited across the three health zones. In theory, the ET learned that the project was encouraging minimum tillage, crop rotation, and promotion of improved seeds of resilient varieties (e.g., Mus I for maize, H204, and Diamond for cowpea). Other important technical approaches have not been introduced, namely composting of organic materials and integrated pest management. The ET also uncovered poor respect of the agricultural cropping system calendar, such as late promotion and distribution of improved seeds and late planting. This can affect crop production, which, when harvested, may not be ready for storage, and increase pest infestation, causing a waste of valuable resources and the need to increase additional inputs, all of which can harm the environment. Deforestation is a significant problem in the project health zones, where charcoal is used for cooking using highly polluting cooking stoves. The team noted a mixed application of approaches designed to protect the environment or mitigate environmental pollution. For instance, in villages visited in the health zones of Miabi and Cilundu, the ET found the persistence of slash and burn techniques to clear fields and the non-application of minimum tillage techniques. Regarding environmental conservation, an important achievement is that the project has introduced tree nurseries for community forestry and agroforestry throughout the DFSA area. However, the startup of nurseries was late, and the selection of tree seedling species occurred without consulting participants to assess their preferences. The project introduced fruit trees such as Citrus Gemilia to improve food security and nutrition and provide opportunities for additional income, as well as other trees such as Terminalia Superba and Cassia spectabilis Given the late start, the project may still explore the acceptability of introducing other tree options, such as trees where caterpillars or weevils thrive, insects consumed in Kasai, to improve dietary diversity and provide multiple economic opportunities. Many signs of climate change are evident in the project area, such as the late start of the rainy season and torrential rainfalls over prolonged periods causing water inundation, particularly in lowlands. It is critical that the project design and implement approaches that will allow participants to adapt to these ongoing environmental changes. Rehabilitation of rural roads in all three health zones has taken place on 98 km of the 171 km of planned road work. The areas where road work has occurred correspond with the project selection criterion, which aimed to open up agricultural production zones by linking them with central markets. IMPEL | Implementer-Led Evaluation & Learning 50 Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved The organization of the road work did not meet the standards followed by the Haute Intensité de Main-d’oeuvre (HIMO) approach, which involves intensive labor and is task-oriented. Also, the ET found that the work and pay conditions promised by the project are not always followed. For instance, workers promised that labor would involve 21 days at three USD per day, but in some cases, the work took far longer (as long as 38 days). Problems with payments were also identified; for instance, in one site, payment was given in two installments, and the second payment made in Congolais francs (CDF), which is less preferable, and workers were paid for 21 rather than the 38 days worked. In efforts to strengthen value chains, Budikadidi identified certain activities that should link directly with key stakeholders, including market gardening, seed production of different crops, and processing of agricultural products. Regarding market garden activities, some OPs were able to produce large quantities of vegetables and market the crops produced, particularly in the Kasansa health zone. Thus far, Budikadidi has only given the contact information of stakeholders, such as owners of restaurants and hotels, wholesalers and transporters, to producer associations, providing nothing concrete to market garden participants who continue to sell their produce at low prices in local markets. Since farmer associations received market garden inputs simultaneously, the same crops flooded markets at similar times, creating an abundance of vegetable crops higher than the consumer demand and causing vegetables sold at discount prices, and in some cases, not sold at all. Even though seed and production activities are still getting underway, it is important to develop concrete plans and formalize linkages for value chain activities. Regarding processing of agricultural products, Budikadidi helped high functioning SILC groups acquire grain mills and establish contacts with potential clients, as well as transporters. SILC groups are widespread, with thousands of SILC groups established across the three health zones. CRS reports approximately 40,000 of the project target of 60,000 participants engaged in SILC activities. During the evaluation, four SILC groups benefited from the receipt of grain mills, including one comprised of women. Budikadidi also supported SILC members with the provision of goats to strengthen social cohesion and resilience against shock. At the start of this activity, the project relied on the expertise of a team of veterinary doctors who had previously trained 45 auxiliary veterinarians who are involved in administering animal care in their respective health zones. It should be noted that the trained auxiliary veterinarians are not yet providing services because they are waiting for starter kits promised by the project. The first batch of 280 goats were bought by Budikadidi and introduced in villages in all three health zones. Due to lack of local suppliers, some goats were purchased in neighboring Kasai Central; project staff reported that veterinarians and veterinarian assistants vaccinated these goats before being introduced into communities. Unfortunately, the introduction of goats coincided with a local epidemic, which led to high mortality of the goats, with about 50% of the goats introduced by Budikadidi perishing. Participants were not involved in the choice of goats for breeding, and as a result, generally did not feel ownership in this activity. The selection of goats was risky, particularly during July – October, when goat disease is prevalent. The reproduction cycle of goats takes about 1.5 years to produce one or two baby goats, and there is no guarantee that female goats will be produced. Therefore, it would take an extensive amount of time to provide goats to all SILC members, which generally include 15-25 per group. Evaluation participant respondents indicated that they would prefer rabbits, which are less risky and have a short reproductive cycle (30 days), with an average litter of four per cycle. “Due to the high mortality rate of the goats introduced to communities, members are no longer interested in raising goats as a support activity for the SILC groups. They experienced high losses of goats in this area.” - Project supervisor Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved 51 5.1.2 Objective 2: Evidence of Change Households successfully implementing PG have regular access to a diverse array of vegetables near their homes. Those participants in proximity to water sources can produce vegetables regularly year￾round to meet household needs, and in particular, to improve the nutritional intake of young children. Market gardens that are underway are contributing to both the food consumption and livelihoods of participants. There is evidence that the market gardening activity, (unlike maize cultivation, which occurs once a year), is improving the income of participants where gardens are successfully implemented. For example, an area of 1 are (10 x 10 meters) can produce about 15 kg of corn sold at 7500 CDF (less than five USD) after four months of cultivation, while market gardens can produce greater quantities of vegetables during the same period, which can be sold at a much higher price. Vegetables are not only destined for the market but also used for food intake in households where generally only cassava leaves are consumed during the dry season as a vegetable. Although the roads selected for road rehabilitation are not complete, roads that have repairs facilitate linkages of some villages with major centers by vehicle transport, which was impossible in the very recent past. The ET learned that money from savings and loans is being used for the schooling of children, medical care of household members, and other social needs. Some respondents indicated that SILC groups have improved social cohesion and reduced local theft. However, there has been less success in creating small business activities. Figure 5: Setting up a permaculture home garden. Photo credit: Felly Muambayi An unexpected negative effect is that in some areas, vegetables produced in market gardens were delivered for sale at the same time, causing an overabundance of vegetables, markets to flood and prices to drop, which was the case of a village visited in Kasansa. This is a result of poor planning by the Budikadidi technical team. Although some communities planting market gardens have selected seeds based on results from the demonstration fields, these seeds have not been certified and approved by SENASEM and therefore have no future for large scale-up. These problems associated with the market gardens are causing participants to drop out. There have been unmet promises to provide materials such as seeds, watering cans, and compensation for days of road work is affecting participant morale. The ET observed additional barriers to change affecting different components of purpose 2. The ET learned that some SILC members had made sudden and unexpected requests for credit, and once obtained, disappeared to remote cities such as Lubumbashi. The disease epidemic that led to high mortality of goats was unexpected and very unfortunate. In the context of the conservation and protection of the environment, a late start-up means that activities are still at the level of setting up nurseries. Deforestation is occurring at tremendous speed due to the need for charcoal, which is the IMPEL | Implementer-Led Evaluation & Learning 52 Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved primary source of energy for cooking in Mbuji Mayi where a 100 kg bag is trading at over 15 USD. Reports that the seeds provided for tree nurseries did not germinate, led participants to believe they had expired and raised concerns about the viability of the approach. Failure to meet promised payment for road work led to protests by participants, who even attempted to burn the Budikadidi coordination offices in the zone of Cilundu, and caused the suspension of project activities for some time. Some field agents complain that their responsibilities are two demanding, forcing them to cover large remote areas up to 70 kilometers from the field agent residence, decreasing their motivation. In general, monitoring of agricultural activities is not frequent enough, leading to inconsistencies in the application of effective and appropriate practices. 5.1.3 Objective 3: Quality, Relevance, and Efficacy of the Design The design of the agricultural sector approach is generally appropriate, entailing a package of complementary activities that are relevant and correspond to local needs. Some approaches have been revised or discontinued in order to focus on those that are more appropriate to the local context and needed to reach project goals. A new, dynamic agricultural team lead has recently been hired to direct and vitalize the agricultural sector. Market gardens were not included in the original plan but added as an activity with potential not only to provide household vegetables but to generate income year-round. The initial proposal to distribute cash to SILC groups to support small business and micro grants during the dry season changed. The project decided that activities focused on animal husbandry would be more appropriate to strengthen social cohesion. While SILC groups have been popular and successful in providing small loans, participants need more sizeable capital to start profitable income￾generating projects. Budikadidi initially did not plan to distribute seeds to farmers' organizations, but the project realized that seed distribution was essential to accelerate agricultural activities. Unfortunately, seed distribution has thus far been slow and insufficient, therefore having no major effect on P2 related to food security and livelihoods. Now in its fourth year, the project has been generally slow in responding to the fundamental agricultural and livelihoods needs of participants, which involves the availability of improved quality seeds, an essential input to achieving food security. 5.1.4 Objective 4: Coordination, Collaboration, and Convergence The agricultural sector initially involved key partners in events such as the launching ceremony and trainings, but did not adequately involve them in activity implementation. More recently, collaborations have been established with different state structures engaged in the seed sector, including INERA, SENASEM, IPAPEL, and Conseil Agricole Rural de Gestion (CARG). At the end of August, a planning meeting to examine activities for the 2019/2020 agricultural season was organized with key collaborators under the project initiative. Around the same time, a meeting of the project steering committee involving partners representing all project activities was also organized. During this meeting, key recommendations concerning agricultural activities for Season A were elaborated. The agricultural team also participates in the recently established multisectoral committee. Although other development organizations focusing on food security, such as ENABEL, implemented by the Belgian Development Agency, operate in the same project area, there has not been an adequate attempt to synergize coordination or to ensure complementarity field activities. The Provincial Seed Council (COPROSEM), is a key platform designed to address problems related to seed. This platform comprised of researchers (INERAL), the government sector mandated to certify seeds (SENASAM), representatives of seed producers (APSKO), technical and financial partners intervening in food security, and ITAPEL, and is functioning at the provincial level. Budikadidi has not been participating in this mega platform, where some of the challenges the project faces could be addressed. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved 53 Also, no efforts have been made to coordinate with humanitarian assistance agencies involved in the distribution of food and cash. The ET noted that some participants are taking leadership roles in multiple agricultural activities, causing them to be overextended and unable to fill their roles and fomenting confusion in activity structures. Budikadidi structures and organizations (OPs, SILC, listening clubs, youth organizations, etc.), may be recognized by the CAC and village leaders; however, they are not officially recognized by the government or functioning in accordance with the legal system, thus making them vulnerable to suspension if problems arise after the project completion. 5.1.5 Objective 5: Early Evidence of Sustainability Because activities are either in their initial phases (seed production, tree nurseries, CLER (comité locale d’entretien routier de club d’ecoute) or being implemented but not yet well organized or highly functional (market gardens), it is too early to predict the sustainability of interventions aimed to improve food security and livelihoods. At present, low crop production and high turnover of participants undermine sustainability of market gardens. Efforts to strengthen value chains have started but depend on agricultural production of OPs, which are only being scaled up this planting season. Listening club members are willing to take on the maintenance of rehabilitated roads, but lack the tools needed. Some activities are not designed to ensure the appropriation of participants, reducing the likelihood of sustainability. The fact that the existing agricultural groups and organizations, whatever their affiliation, are not recognized by the government and governed by legal frameworks, makes them fragile post-project implementation when Budikadidi will no longer support them. Only if approaches accelerate and crop outputs improve, will community members embrace and take ownership of activities. 5.1.6 Objective 7: Appropriateness and Effectiveness of Interventions Focused on Cross￾Cutting Themes Efforts have been made to integrate women in agricultural activities, although the ET uncovered signs that female participation is receiving some local criticism, which reflects local customs and norms in contexts where men perceive women as inferior. Most OPs visited during the evaluation had relatively equal representation of women, with some women occupying positions of responsibility, including vice￾president, or more often, treasurer. Work is set up with men in charge of clearing fields, evacuating water, and construction of elevated beds for the market gardens, and women are responsible for planting, watering, and weeding. Women also take care of the harvesting and marketing of crops. While this approach reflects traditional agriculture gender roles, it places more burden on women who are responsible for activities involving regular work, such as watering, while men participate in communal work generally scheduled three times a week. In some OPs, the ET found rules favorable to women, such as allowing pregnant female members a reprieve from work seven weeks before and seven weeks after delivery. During this period, women could still enjoy benefits recognized by other members. While efforts have been made to increase female involvement, more emphasis should be placed on encouraging joint communication and decision-making to ensure meaningful engagement, rather than the nominal participation of women. “The sustainability of Budikadidi activities will depend on community perceptions of the approaches and the involvement of local authorities and community leaders. Given that Budikadidi is a development project, community ownership of activities will be critical.” - Consortium partner IMPEL | Implementer-Led Evaluation & Learning 54 Findings – Purpose 2: Household-Inclusive Social and Economic Well-Being Improved While youth actively participate in Budikadidi, youth are oriented towards community-based activities such as road maintenance, environmental sanitation, and SILC, and less interested in agriculture. The ET found that youth do not generally integrate with adult groups, establishing their separate groups like for SILC or road maintenance that operate independently. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Monitoring, Evaluation, Accountability, and Learning (MEAL) 55 6. FINDINGS – MONITORING, EVALUATION, ACCOUNTABILITY, AND LEARNING (MEAL) The findings in this section are presented slightly differently than the previous sections on the project purposes and sub-purposes. This section focuses on M&E and learning (MEAL). These findings do not lend themselves to being parsed by the evaluation objectives. Instead, these findings are presented as contextual factors, strengths, and missed opportunities. As with the other findings sections, the recommendations are consolidated in the recommendation section at the end of the report. 6.1 Contextual Factors It is important to document relevant contextual factors when evaluating the quality of the MEAL systems in this DFSA. First, Budikadidi is a new activity. There have not been any other FFP-funded development activities in this region of DRC. While CRS has extensive institutional experience in DRC and with FFP-funded development activities in other locations, they are new to this area of the Kasai region. CRS started from scratch with the development of a new M&E system and had to hire and train new staff on the system. The region of Kasai where Budikadidi implemented has been affected by conflict, and it received a substantial influx of humanitarian assistance around the time this activity was starting up. Humanitarian assistance in the form of food distribution is still ongoing. Because of the influx of humanitarian assistance and the lack of community-development activities similar to Budikadidi in the region, it has been difficult for CRS to manage expectations from communities about their role and the kinds of assistance that will be provided. For example, the majority of the feedback received through the activities feedback mechanisms has been requests for distributions of in-kind assistance (tools, food, etc.). FFP introduced some new policies, procedures, and tools with this round of DFSAs. The most substantial change was the introduction of the R&I approach, although several new or revised M&E tools were also introduced. The M&E tools included revised indicator lists, additional guidance on the Log Frame template, and a target setting tool that allows partners to more accurately estimate achievable population-level targets by inputting key variables about the interventions and underlying population characteristics. Project staff expressed that they faced challenges understanding the new requirements surrounding the R&I approach and struggled to comprehend how to use new templates like the target setting tool effectively. This led to delays in submitting key documents to FFP as they sought clarification from FFP and worked to understand the new requirements. They said that overall, it has been a learning experience for them. 6.2 Strengths Budikadidi has a generally strong routine monitoring system that is an effective project management tool by staff at all levels of the organization. The system comprised of a data collection tool using CommCare, which is a commercial mobile data collection software built on ODK. Data collected on forms built-in CommCare. After data is collected, it is analyzed and linked to a Power BI dashboard, which is accessible to all staff, as well as some external stakeholders. The dashboard can be used to check progress against key targets. IMPEL | Implementer-Led Evaluation & Learning 56 Findings – Monitoring, Evaluation, Accountability, and Learning (MEAL) Each participant is issued a unique ID and a participant ID card. The participant ID card contains the participant’s name, location, age, gender, and unique ID number. Each ID card has a QR code that can be scanned by tablets using CommCare to register participants' attendance in different activities. For example, the participant's ID card would be scanned when the CG promoters trained new LMs or when a household established a new PG. This system allows Budikadidi staff to track which interventions each participant is engaged in. When the routine monitoring system was rolled out in May 2018, CRS held monthly training for project staff to get oriented to the system and the use of the tablets. All field agents have access to tablets, though staff expressed that there are not enough tablets (or staff) to collect some routine monitoring data that they need to gather from all 70,000 participant households. Staff consistently reported that the data collection system worked well, that they have confidence in the data presented on the dashboard, and that they regularly reference the dashboard to check their progress. This effective routine monitoring system is a significant accomplishment and demonstrates the hard work that the MEAL team and other project staff have committed to getting the system in place. That said, the system is expensive, and the high cost prevents them from purchasing additional tablets. Another strength of the project's MEAL system is the process for tracking and responding to feedback received through the Green Line and suggestion boxes. The Green Line is a toll-free hotline for participant complaints and feedback that is used by CRS countrywide and is the most frequently used feedback mechanism in Budikadidi. Figure 6: Suggestion box. Photo credit: Adam Trowbridge Feedback received on the Green Line that is relevant to Budikadidi shared with the A&L Officer. The A&L Officer then shares feedback with the relevant sector leads, who are responsible for responding. Information identified as sensitive, such as reports related to fraud, corruption, or violence, is shared with the CRS Representative in Kinshasa, who is in charge of putting together a team to investigate the situation. The mechanisms to provide feedback (Green Line and suggestion boxes) were selected based on preferences obtained during community consultations. This process allows the feedback to efficiently Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Monitoring, Evaluation, Accountability, and Learning (MEAL) 57 get back to the right sector lead to review and provide a response. The feedback has been used to adapt project implementation, including changing the approach for paying cash-for-work participants. 6.3 Missed Opportunities While Budikadidi’s routine monitoring system is highly effective at tracking outputs, there is a minimal focus on measuring and tracking implementation quality. Systems for tracking implementation quality are essential for an activity of this scale to promptly identify issues that may prevent the desired outcomes from being achieved. Because implementation quality issues not incorporated into the M&E system, some teams have begun developing their parallel systems. For example, the nutrition team developed a system for tracking that promoters have the appropriate knowledge. Budikadidi has established a strong foundation with its feedback mechanisms, but there are opportunities to make it more accessible and responsive. The ET twice attempted to use the Green Line. On the first call, there was no answer. On the second call, the ET was told that the line is only for emergency-related projects. In mid-July of 2019, the Green Line reduced its hours of operation from six days per week to just three days per week for budgetary reasons. Budikadidi pays for each call so that the line can be free to use for participants, but the volume has been high. While project staff reported that they have begun informing community members about the changes, it is not clear how this is conveyed to both field staff and community members. Given that many participants may not have cell phones to begin with, and now with reduced hours, there must be other mechanisms for providing feedback. The alternative mechanism for feedback is suggestion boxes placed in some communities. While there had initially been a plan to install suggestion boxes in each community, at the time of the midterm, there were just 60 boxes12 across a total of 474 targeted villages. The requirement for a participant to travel to another village to submit a complaint presents a significant barrier. While there appears to be interest in adding additional suggestion boxes, the specific plan is not clear. Unfortunately, during the evaluation, the ET was unable to obtain specific information about the volume and nature of complaints submitted through the Green Line and suggestion boxes. Additionally, literacy rates in the targeted communities are low, with significant disparities between males and females. The CRS proposal notes that 50% of women aged 15-24 are illiterate13 . This leaves illiterate participants who do not have access to a cell phone with only the option to provide verbal feedback to project staff. Verbal feedback may present protection issues, especially in cases where the complaint is related to project staff. At the same time, there are indications that participants are more comfortable providing verbal feedback and see it as less formal. Evaluation results from other projects in the DRC using telephone lines have found that community members view a complaint submitted by phone or in writing to be more “formal” and may lead to negative consequences (REFERENCE). While there is a well-developed system for tracking feedback received through the Green Line and suggestion boxes, the process is less structured for feedback given verbally. Verbal feedback is likely one of the most common for participants to provide feedback given lack of access to mobile phones and low literacy rates. It is likely culturally preferred and confirmed in focus groups with participants. Project staff acknowledged that there is not a transparent system for capturing and tracking verbal feedback other than informal discussions during staff meetings. Feedback may not make its way back to sector 12 According to interview with Jackson Chabuine, Accountability & Learning Officer 13 CRS revised proposal. Original source: UNICEF MICS 2010. IMPEL | Implementer-Led Evaluation & Learning 58 Findings – Monitoring, Evaluation, Accountability, and Learning (MEAL) leads in Mubji-Mayi, who are best able to respond. During the evaluation, the ET identified several instances where complaints or rumors were not rapidly or systematically reported and addressed, furthering discontent and distrust in some communities. Budikadidi exhibits a commitment to learning and adaptive management, but routine learning activities lack strategic focus and buy-in. The activity’s senior leadership has shown a strong commitment to integrating learning from the one-off studies and TOC reviews conducted through the R&I process, and cite examples of where approaches have been adapted as a result of what they have learned. At the same time, the activity lacks formalized processes for ensuring that ongoing learning activities are effective and inclusive. . The routine learning activities conducted by the project are ad hoc, and appear to be disconnected from the needs of the technical teams. While staff described plans for developing a learning agenda to identify learning priorities and formalize the process, it does not currently exist. The routine learning activities that take place are led independently by the A&L Officer. The A&L Officer identifies lessons learned through an analysis of data from the feedback mechanisms, or through identifying success stories or other lessons from conversations with staff in the field. This is reported in a table developed quarterly. The report is shared with technical leads and project leadership for feedback. According to interviews, the technical leads do not regularly provide feedback on learning products, and the A&L Officer is not involved in planning meetings, which could help him better understand the needs of the teams. The project has faced challenges with M&E tools and templates required by Food for Peace. For example, FFP’s approval of the Indicator Performance Tracking Table (IPTT) took longer than anticipated. Because the numbers in the IPTT were not officially approved, Budikadidi leadership was reluctant to share it with other project staff or external stakeholders, limiting its use as a management tool. Because the IPTT is primarily a reporting tool, the practical implications of the delays in finalizing the IPTT were limited. The COP stated that staff were generally very familiar with their targets. While Theory of Change (TOC) is used extensively by senior project staff, and is reviewed in detail each quarter, it was seen as complicated and thus not shared widely among many lower-level project staff. As a result, some staff, particularly at the field level, did not know exactly how their interventions contributed to the project's objectives. Some field-level staff reported not knowing the project's high￾level objectives or indicators. This likely represents a breakdown in communication between senior-level technical staff and field staff. While this varied to some extent by team, and it remained unclear how much this was driven by staff turnover, it represents a threat to coordinated and integrated interventions. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Refine and Implement (R&I) 59 7. FINDINGS – REFINE & IMPLEMENT (R&I) As with the MEAL findings section above, the findings in this section are presented slightly differently than the previous sections on the project purposes and sub-purposes. This section focuses on the R&I approach. These findings do not lend themselves to being parsed by the evaluation objectives. Instead, these findings are presented as contextual factors, strengths, and missed opportunities. As with the other findings sections, the recommendations are consolidated in the recommendation section at the end of the report. A total of eight studies conducted during the refinement year. These were designed to address 52 key questions that identified as knowledge gaps across eight sectors. The eight studies were: 1. Gender analysis 2. Seed Security Systems Assessment 3. Value Chain Analysis 4. Local Food Assessment 5. Initial Environmental Examination 6. Resilience Study (SenseMaker) 7. Fertility Awareness Method (FAM) study 8. Barrier Analysis 7.1 Contextual Factors The context in the implementation areas changed rapidly between when design activities took place and today. The conflict that began in 2016 forced the activity to change their targeted locations and had severely affected some of the communities where Budikadidi is implementing. The cholera outbreak of 2017 also forced the project to revise their activities to address critical needs. Some project staff expressed concern that the findings of the refinement year studies would become irrelevant due to the rapidly changing context. The conflict forced the project to adjust their targeted areas of implementation. Due to delays caused by the conflict and change in geographic targets, the project did not have a full refinement year. They were unable to begin their formative research until April 2017 and had to complete the majority of the research in time for a workshop FFP scheduled in September 2017. They were not able to meet that deadline for all studies, with some continuing through December 2017. The Refine & Implement approach was new to all stakeholders. CRS expressed that they faced challenges communicating the approach to the host-country government, who did not understand why the project did not begin direct implementation in the first year. 7.2 Strengths Several studies that were conducted during the refinement year were used to make substantial changes to the project's theory of change and implementation strategies during the refinement year. Some examples of changes to program design and implementation resulting from the studies include:  Literacy activities expanded because they identified illiteracy as affecting women's self￾confidence and a barrier to women's involvement in leadership. IMPEL | Implementer-Led Evaluation & Learning 60 Findings – Refine and Implement (R&I)  The CG model was expanded to include husbands (particularly young men who are more open to change) and mothers-in-law as a from the gender analysis that women require their husband's permission to participate in activities outside of the house, and identifying mothers￾in-law as very influential in household decision making, especially decisions related to child marriage.  Promoted recipes were adopted and varied by location based on the availability of local nutritious foods.  Prioritizing improved crop storage to reduce post-harvest losses.  Identifying village chiefs as important stakeholders and increasing the level of engagement with them.  Due to female workloads and restrictions on female movement identified through the gender analysis, household visits rather than group meetings considered a better way to reach women through the CG approach  The project's approach to healthy timing and spacing pregnancy shifting from delivering fertility awareness counseling toward coordinating with IHP, which will be delivering a full package of family planning options. The study was less used by the project for direct delivery but was key in advocating with IHP for the inclusion of fertility awareness methods in their package. Figure 7: Theory of Change, Foundations Purpose – before and after R&I CRS is planning to carry out multiple additional studies over the next year to further refine their strategies, continuing the applied research beyond the initial refinement year. These studies will look at social networks, approaches to introduce growth matts so that caregivers can monitor growth of their young children, additional barriers to behavior change, and the role of men in childcare. The Health and Nutrition team are especially committed to adaptive management and conducting research where necessary to inform their strategies. Budikadidi staff tended to view the R&I process positively. Staff reported appreciating that the process allowed them to develop a deeper understanding of the context they were operating in and to make adjustments to the project’s design to better meet the needs in that context. This was reported to be especially important given that Budikadidi was a new DFSA operating in a new area, so there was no previous contextual experience to build upon. In interviews, project staff noted that each of the sectoral teams was significantly involved in determining the research questions that they wanted to answer through their formative research and participated in aspects of the studies related to their own and other sectors. This helped to ensure the buy-in of each sector team in the process, increasing the odds that the findings would be used to improve the design. Government officials were also reported to be involved in the formative research Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Refine and Implement (R&I) 61 studies, particularly related to nutrition and health, which likely enhanced the government buy-in to sector activities. 7.3 Missed Opportunities There was a lack of prioritization in eight studies conducted during the refinement period. This number of studies proved to be a significant management burden for the project, particularly given the late start and short period. They struggled to identify qualified consultants for each study, and managing the logistics for the fieldwork for each consultant was burdensome. Because of this, the quality of the studies varied greatly, as did their usefulness. Issues with quality driven by the challenges faced identifying qualified consultants with experience in the DRC, and by the short time period, they had to complete the studies due to the delays at startup. Budikadidi staff reported that Food for Peace partially drove the number of studies conducted, who, during a workshop held in Rwanda in 2017, apparently wanted to ensure that their technical sectors were included in the R&I period. This led to lack of ownership by CRS staff for some studies, potentially limiting utilization and an increase in the overall workload. After completion, some studies have not been translated to French. Budikadidi presented results from several studies (seed security systems assessment, local food assessment, fertility awareness, resilience) in public forums in Kasai and Kinshasa. However, those studies that have not officially been approved by USAID have not yet been made available to the broader public to enhance overall learning. Most Budikadidi staff were hired towards the end of or after the refinement year. Project leadership believes that this limited their ability to learn from the refinement process. This observation through interviews with project staff at multiple levels, where there appeared to be a common misperception that adaptive management should only occur in the first year. While R&I activities do have two distinct periods, a refinement year and then standard implementation, adaptive management should be used throughout the full period of performance. This may point to the need for better messaging by FFP about R&I, and improved training of DFSA staff as they are onboard. Project leadership expressed that there were significant delays in FFP approval of refinement year studies. While they noted that these delays did not preclude them from using the findings to adapt their strategies, it did increase the management burden. IMPEL | Implementer-Led Evaluation & Learning 62 Findings – Other Focus Areas 8. FINDINGS – OTHER FOCUS AREAS This last section of findings presents brief findings on three other focus areas that the ET considered to be important to highlight, but which did not fall within the other findings sections. 8.1 Operations and Management The Budikadidi consortium consists of international and local partners with varying internal operations and management structures. Due to challenges in identifying a sufficient number of candidates from the region with adequate technical capacity and experience, many national technical staff are not from Kasai. The hiring of non-local staff initially strained staff relations, which have improved over time through team-building efforts. CRS's technical team members are very strong, which has likely garnered the respect of colleagues. Generally, the ET observed good collaboration and cohesion among the staff working at headquarter and zonal levels. CRS has repeatedly demonstrated an ability to identify challenges and make adaptations to address operational weaknesses. One major adaptation that occurred at the outset of the project involved stationing field agents in village sites to ensure consistent and timely oversight of activities. Another example involved transferring oversight of one of the primary technical sectors from one consortium partner to another with more appropriate expertise in order to to improve performance in that sector. Changes in technical leads have also been made to improve technical oversight and management. The evaluation findings highlighted several issues that may be affecting field staff performance and impacting on the effectiveness of activities. Many field agents, who are responsible for traveling long distances to supervise multiple villages and activities, appeared to be overextended in their work. Also, some field agents did not have the expertise or training required to supervise certain technical components. This was particularly evident in the agricultural sector, where an interviewed polyvalent did not maintain sufficient knowledge to impart critical information related to farming techniques. Another concern relates to variable salary structures across the consortium partners, which seems to be affecting the morale and motivation of lesser paid workers with the same responsibilities as their higher￾paid counterparts. There has been some turnover of key personnel; for example, the Resilience Lead left and was not replaced. Additionally, the project has lost a key health zone supervisor and experiences occasional turnover of field agents. Budikadidi leadership, including the administrator, Deputy COP, and COP, should be commended for the project solid field operations and strong management structures. Activity staff respect the COP and feel he has been effectively providing oversight. The ET was highly appreciative of his availability and transparency during the evaluation period. One disadvantage, however, is that the COP and the Health, Nutrition, and WASH specialist, two critical members of the DFSA, are splitting their time between Kinshasa and Mbuji Mayi. These key personnel both have highly relevant experience, strong technical backgrounds, and outstanding leadership skills, and are critical to the overall success of the project. Both were consistently described as highly professional and responsive to requests from partners and collaborators. While we respect the CRS decision to accommodate family considerations to retain highly competent staff, and understand that these staff are involved in critical work related to collaboration and advocacy in Kinshasa, there were indications that they were not always fully aware of the current status of all aspects of field activities, underscoring that periodic absences may impact the DFSA implementation. More importantly, senior technical staff are not spending sufficient time in the Mid-Term Evaluation of the Budikadidi DFSA in the DRC Findings – Other Focus Areas 63 Budikadidi health zones, which is needed to identify key strengths, weaknesses, and how these can best be addressed. Evaluation results highlighted that frequent back and forth communications between the Kinshasa mission and CRS may slow the progress of activities. FFP may consider evaluating ways to decrease the requests made and time involved in responding to ongoing demands. 8.2 Sustainability For each technical sector, technical leads explored issues related to sustainability, which are delineated in the sector write-ups. One general concern is that the project lacks a project-level, formal sustainability plan to guide the package of Budikadidi activities. Another concern relates to the Year 4 PREP plans to reduce field staff at a time when some pivotal activities occur. For example, market gardens lag behind, and other activities, such as nutrition and WASH, need additional oversight to gain essential traction to meet project outcomes and continue post project. Activities, such as those related to WASH, are going to be reduced significantly at a time when additional investments are needed. 8.3 Social Behavioral Change Budikadidi has a strategy and team dedicated to assisting technical sectors to improve approaches designed to motivate adoption of improved practices. Specific efforts have involved sharing success stories of participants who have adopted positive practices at the village level (e.g. Champion Faithful House couples). They have developed and disseminated videos promoting activities, sharing a newsletter that delineates success stories and achievements, and promoting theme-specific messages through local radio emissions. Radio emissions focus on specific themes developed in conjunction with technical sectors, which broadcast over a month-long period. Programs involve interactions with listeners, allowing participants to share experiences. A detailed schedule of radio emissions shared with the ET. Unfortunately, during each attempt to listen to the radio messages, they were not being broadcast as scheduled. Staff should monitor radio emissions to ensure that channels are broadcasting messages as planned. A new SBC initiative implemented in FY19 involves influential leaders to convey messages promoting aspects of sector activities. Videos developed by the SBC team are providing information on activities projected in communities. This will follow with community discussions led by religious and other village leaders who have been exposed to and trained on key components of the interventions. This initiative presents an opportunity to engage influential community in behavioral change and to reach broader sectors of the population, particularly men. IMPEL | Implementer-Led Evaluation & Learning 64 Recommendations 9. RECOMMENDATIONS In this section, the recommendations from each of the various SPs evaluated are consolidated, as well as recommendations related to the other focus areas of the evaluation. Cross-cutting recommendations are presented last, which are more overarching key recommendations that relate to multiple sectors/project activities. SP F.1: Environment favors multi-sectoral development toward food and nutrition security for all community members 1) Strengthen linkages with local government entities working in project health zones. Weaknesses in collaboration and information sharing with government entities at the zonal level identified during the evaluation could jeopardize longer-term sustainability. Evaluation respondents in charge of local government entities indicated that they were not adequately informed or involved in the CAC development activities. This is despite the fact that the community action planning process entails identifying and engaging local resources. CACs would be strengthened by better identifying local government resources, establishing closer collaborative efforts with local government entities, and including government representatives in community planning, implementation, and monitoring of activities. 2) Develop processes to transmit information on project activities to state agents. Information collected by CACs on village-level activities is entered in the Budikadidi M & E system but is not systematically transferred to government territorial structures. Budikadidi should assess what aspects of their ongoing activity monitoring would be appropriate to transfer to government sectors. An orientation should be carried out to inform the CACs of local government institutions and their data collection systems and to train CAC members on how to transmit their activity reports so that activities carried out can be included in local government statistics. Budikadidi supervision structures should monitor whether transfer of reports, particularly at the outset, is taking place. While CODESAs may be used to facilitate transfer of information to the government health sector, the process is less transparent for activities related to agriculture, rural development, farmers' organizations, and SILC. 3) Disseminate project information to local government structures so that it can be captured in national statistics. According to the project approach, information on the different thematic activities is centralized at the CAC level and entered into the existing monitoring system (CommCare and Power BI). This information appears in the CRS platform but not in the national statistics. While in some areas, the Budikadidi field agent transmitted reports to the health zone core team and involved the health team in meetings and activities, but this cannot be included in national statistics unless official procedures are in place. In the health sector, this involves transmitting data through monthly government reports through SNIS, compiled at the health center level. Efforts should be made to ensure that project data reach local government structures representing different technical sectors for inclusion in national statistics such as DHIS2. 4) Assess how best to improve government involvement. Local government entities claimed not to be involved in Budikadidi activities, even though they appeared to be invited to meetings and Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 65 participated in training and other activities. Budikadidi should examine the type of involvement government officials are anticipating and how best to get them effectively involved in the project. Based on the information, the project can refine collaborative strategies in ways that can enhance positive government engagement and benefit project activities. 5) Improve information sharing with local government entities and IPs. Budikadidi has done a commendable job of identifying and setting up collaboration and coordination with national and provincial level government bodies and implementing partners. In several instances, the project took the initiative to establish relevant oversight committees, such as the steering committee and multi-sectoral committees. However, the ET identified problematic gaps in information dissemination and coordination, such as instances where implementing partners were setting up new CACs or employing LMs in project activities. In general, zonal and more local level information sharing and partnerships need to strengthen. It is also imperative to strengthen efforts to disseminate information on community initiatives and lessons learned with government entities at all levels and with other implementing partners, UN agencies, and donors. 6) Understand why some CACs function better than others. The evaluation illustrated that some CACs are higher functioning, showing greater independence and autonomy. In these cases, CAC and community members exhibited more significant initiative in advancing activities rather than waiting for additional assistance and resources. It would be beneficial for the project to explore factors underlying higher functioning CACs and use the information to improve the capacity of lower functioning CACs. 7) Formalize and communicate a sustainability plan to staff working at all levels. While CRS has developed sustainability approaches specific to different technical interventions, Budikadidi does not have a well-documented, coherent, project-level sustainability plan. As the project moves into year 4, the management team needs to develop a more strategic, comprehensive approach designed to ensure that assumptions about the sustainability of practices and local service delivery are collectively well designed, relevant and feasible in the dynamic project context. The project should communicate the plan to staff members to ensure that they have a clear understanding and can appropriately influence programming at all levels. SP F.2: Reduce systemic gender barriers to food and nutrition security 8) Identify ways to ensure longer-term implementation beyond the DFSA mandate of activities that appear to be getting traction in reducing systemic gender barriers. Gender norms in Kasai are deeply entrenched and will take a long time to change. Budikadidi needs to explore opportunities to better ensure that those activities that appear to have the potential to make a change, such as literacy and CG, continue post implementation and thus have a broader and long-term effect. Establishing firmer links with government sectors, such as the health sector in the case of CG, or other implementing partners working on health or education in the Budikadidi health zones should be explored to see how their involvement could better ensure the continuation and longevity of these important activities. 9) Examine ways to improve economic opportunities for women. Women are primarily involved in physically strenuous agricultural activities, with little opportunity to engage in income- IMPEL | Implementer-Led Evaluation & Learning 66 Recommendations generating endeavors. SILC groups provide credit for income generating activities, but income gains to date have been limited for participants. In addition, the ET learned that loans are often obtained to meet immediate family needs. The project should consult with women participants to determine their preferences regarding income generation and perceived challenges in participating in IGAs. The information can be used to identify opportune and culturally appropriate ways for women to engage in the cash economy and generate sustainable sources of revenue, which over the long term, should lead to greater decision-making power related to household food security and nutrition. 10) Reassess the effectiveness of approaches designed to reduce early pregnancies and forced marriages. Pregnancy and forced marriage before girls reach adulthood are common occurrences in Kasai. Increasing female school enrollment, enhanced education, improving income-generating opportunities, and enforcing policies designed to delay marriage and reduce adolescent pregnancy have proven effective in reducing these practices in other contexts. However, CRS has opted to implement strategies that focus on short-term and less evidence￾based approaches, such as sharing information with and counselling grandmothers, fathers, community leaders (traditional and religious leaders) and youth, aimed at raising awareness of the legal age of marriage, the importance of consent to marriage, and the consequences of early, unintended pregnancy, to reduce early pregnancy and forced marriage. While USAID IHP was supposed to provide the full government family planning package in health centers, due to many unforeseen structural and politically-related obstacles, USAID IHP has had a very late start in implementing activities. CRS should reassess the effectiveness of the strategy aimed to reduce early pregnancy and forced marriage, practices both affecting the future and status of girls and women and the high prevalence of malnutrition in children under five years. . The project should consider consulting Budikadidi participants to determine why early pregnancy and forced marriage persist and identify culturally feasible and acceptable approaches to instill change. 11) Develop a gender strategy as part of a sustainability plan. The ET found that gender initiatives could benefit from a formal, strategic plan. Formulation of a plan could be guided by findings from the MTE, monitoring data, and additional qualitative inquiry. The project should consider other approaches that could bolster ongoing gender initiatives. Suggestions include: a. Leverage policy and programs aimed to reduce gender inequities. Research in other countries shows that developing and enforcing policies and programs is an effective way to decrease early pregnancy and marriage. In one of the evaluation villages, police enforcement of policy designed to protect female youths from early marriage appeared to be perceived locally as a positive measure. The project should explore ways to use legal codes and policies to enforce measures against early marriage and pregnancy in minors. b. Explore potential new strategies for reaching men. Male engagement remains weak, particularly among men who work away from home (such as miners) and those who have low overall engagement in their families. While the existing Faithful House approach is demonstrating some gains in reaching men participating, the evaluation found that many husbands are refusing to continue to attend sessions due to concerns that activities are shifting household power dynamics in favor of women. In general, the Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 67 number of couple of participants is low, and there is a gap in strategies to reach men who are not engaged. Content geared toward men would be useful, i.e. interests in agriculture and livelihood activities, household management including birth spacing and family planning, and maternal and child health and nutrition, developing specific content for men and focusing on how they can benefit from activities. Likewise, while continuing to leverage the existing FH entry point, the project should explore whether it would be adequate to reach men through male-centered channels (for example, at mines or through community meetings or discussions on the radio). While the SBC approach using videos to convey messages in village settings should serve to capture male attention, at least those living in their villages, CRS should not assume that these efforts will be enough to change firmly entrenched male attitudes and behaviors that exist in the Kasai context. A combination of multifaceted, innovative strategies that captivate men's interests and goals related to maternal wellbeing, and child raising will likely be needed. SP F.3: Communities are resilient to common shocks 12) Examine whether the initial shocks identified need to be revised and resilience strategies redefined. Several shocks initially identified continue to exist, but some may be less relevant. New shocks may have developed since the original assessment. It is important for Budikadidi to carry out a review of shocks affecting communities and to evaluate if solutions previously proposed are still appropriate and relevant. Ongoing monitoring data should be used to support this review. 13) Strengthen interventions aimed to diversify crop production and livelihoods. Thus far, Budikadidi has not been successful in implementing activities that increase crop diversity and diversify livelihoods which will allow communities to absorb shock. Budikadidi must both accelerate implementation of agriculture activities to make up for delays and better ensure that appropriate technical oversight is provided by field agents and followed by participants. Inputs critical for activities and promised by the project, such as seeds, need to be provided in a timely fashion to avoid further delays and to gain project credibility. Implementation of permagardens require technical corrections (deeper beds, key inputs for soil fertility, effective fencing) and additional training of both field agents and participants, who both exhibited poor understanding of the approach. Budikadidi should also assess the cash payments given to permagarden technical experts, which appear to encourage implementation of more permagardens, but reduces the quality. Regular supervision by the agriculture technical lead, combined by more routine monitoring by field agents, is critical to provide needed oversight and address activity weaknesses in a timely fashion. Animal husbandry initiatives must adhere to preferences of participants and employ technically sound approaches. Strengthening linkages across purposes—particularly between P1 and P2—is also critical to establish resilience. For example, vegetables produced in permagardens should be promoted through CG food diversity messaging and during cooking demonstration. SILC should concentrate on encouraging participants to initiate small business activities. The project should explore other opportunities for startup income generating projects which will diversify livelihoods. IMPEL | Implementer-Led Evaluation & Learning 68 Recommendations 14) Strengthen involvement and capacities of local government entities to ensure complementarity and sustainability of activities. Several activities carried out by Budikadidi, which aim to increase community resilience, such as interventions associated with early warning systems and agriculture, do not appear to have adequate participation and support of government partners, despite Budikadidi efforts to engage government officials. Budikadidi must continue to attempt to involve government entities at the provincial and zonal and more local levels that will be responsible for supervising activities in post project implementation. Much data collected as part of resilience-related activities are not transmitted to government entities. In efforts to ensure sustainability, it is important to identify mechanisms to ensure transmission of these data. The multi-sectoral committee should be involved in providing oversight to resilience activities. SP 1.2: Households practice optimal HNW behaviors during the 1000-day period 15) Reinforce recognition of chronic malnutrition among project staff, the health system, and communities. A recent SMART survey in Miabi found over 59% of children stunted; however, project staff, health facilities and communities believe the overall nutrition situation is improving and, in some cases, declared the problem solved. This is based on the perception in the project area that malnutrition is only (severe) acute malnutrition, so without children who have red MUACs they do not believe there are malnourished children. This may undermine uptake of improved nutrition practices as participants may believe there is no need for them to change practices if their child is already "green." Additional, detailed modules on the different types of malnutrition, emphasizing chronic malnutrition and its long-term consequences and how it addressed by uptake of improved practices in the house and community, should be developed in conjunction with the health system and rolled out. In order to support such a module, the project should conduct formative research around how best to communicate the problem of stunting at community level (e.g. by measuring height at community level as being explored in a planned study with Tufts, disseminating SMART survey findings, etc.). 16) Refine behavior change approaches to maximize adoption of high priority behaviors. The first annual survey carried out in September 2019 should provide key insights on which behaviors are changing, and which need additional attention. For behaviors where change is lagging, the project team should identify 2-3 of the highest priority behaviors -- i.e., those with high potential impact on stunting and low barriers to adoption -- for additional, highly targeted efforts across multiple platforms to reduce barriers and improve uptake. Human-centered design approaches – that is, those that include the participants themselves in generating and testing solutions such as Trials for Improved Practices (TIPS) would also be particularly useful in identifying how these practices can be widely adopted. From these inputs, hands-on activities and approaches (such as the dietary diversity game) can be rolled out through a multifaceted approach that includes CG and other project platforms targeting a variety of audiences until widespread change occurs. Channels that will also reach men and create community engagement around the issues will be critical complements to the CG approach (e.g., community videos, engagement of religious leaders, and further engagement of CACs). It will likely require several months of intensive effort per behavior, before shifting to the next. As a Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 69 parallel effort, the team should also give additional weight to reinforcing problem-solving skills among both volunteers and field agents involved in ongoing work and supervision structures. 17) Explore drivers of high vs. low engagement LMs and participating households to strengthen participation of low engagement LMs. Ultimately, the project's success in bringing about household and community level behavior change demands highly effective LMs. Understanding what is driving the differences across LMs in terms of their engagement and the interest of their neighboring households will be valuable in maximizing project impact. The project should deploy strategies to strengthen engagement where it is lagging, based on replicating conditions in communities where engagement is strong. As some communities seem to have a misunderstanding of LMs' benefits from the project, greater transparency and communication about their role and benefits may be useful (via radio, CACs, and or churches). The project should also give continued and increased attention to high quality and effective supervision of promoters, including considering unannounced visits to ensure they are adhering to their work plans. Likewise, the project team should engage communities in finding solutions for strengthening or transitioning out promoters and LMs that are not meeting expectations. While this will be delicate given their volunteer status and the fact the communities chose them, it will ultimately improve the effectiveness of the project. SP 1.4: Communities and HHs maintain a clean environment conducive to good health and nutrition 18) Increase access to potable water points. The high demand for potable water in the Budikadidi project area far exceeds the budgetary allocation for water system installations. The population need for water sources is placing extreme stress on the water systems. The project has invested in and recently installed water systems, but they are failing to meet SPHERE standards related to queueing time and the average liter per person per day. This raises concerns about the durability of the borehole foot pumps and whether the new pumps will reach their lifespans. Furthermore, newly installed water points are not meeting the public demand for water, forcing communities to continue to utilize unsafe drinking water. CRS must reassess the appropriateness of the foot pumps. One option is to consider investing in fewer, but higher quality water systems that can provide potable water to more significant numbers of people and have a record of low maintenance and high durability. In this regard, CRS and FFP should carry out a joint review to assess where priority needs remain in the project area and how the address needs. The joint team will need also to reconsider the foot pump technology being used to source water and ensure that both the community and SNHR are in agreement with the selected technology and its long-term maintenance implications after the project ends. 19) Strengthen sustainability of water repairs. Budikadidi needs to ensure that parts and technicians are available so that breakdowns in water systems can be rapidly repaired. The project is currently working with Vergnet Hydro to decentralize its repair services in the project health zones. However, communities located in remote areas will still have limited access to technical assistance and spare parts. The project should source spare part shops closer to even remote communities to reduce delays. The project is working with water committees to ensure that business plans include adequate budgeting for complex repairs and water quality treatment. Plans should also include life cycle costing, how to ensure that adequate funds are IMPEL | Implementer-Led Evaluation & Learning 70 Recommendations available for complex repairs, and how to invest in diverse income-generating activities to increase the availability of revolving funds. The operation and maintenance instruction manuals should be laminated to ensure they are adequately maintained and durable. 20) Improve linkages between water committees and CAC structures and general governance of water committees. Water committees, which are well-established structures, meet with CAC members and the village chief monthly to provide updates on water management issues. Community members are not always included in general assembly meetings and, therefore, do not have decision-making authority related to water and sanitation approaches, which affect them. For example, the decision to charge 50 francs to fill a jerry can with water obtained from boreholes was taken without considering community opinions, most importantly affecting vulnerable members of the population. When water committees are discussing the price for the borehole water supply, they should include non-committee members in the discussion (women, men, and the most vulnerable households before raising the final decision to the CAC and the village lead. Water committees have established a system to ensure accountability of funds (kept in a cash box) collected at water points, yet it would be more beneficial to keep most funds in a bank or Mpesa account where interest accrues, and the money is safe from corruption and theft. 21) Increase the uptake of improved latrines by SILC group participants. The project aims to have most households participate in SILC groups to ease costs involved in the transition to improved latrines. However, many participants are unable to contribute the required funds to participate. In both P1 and P2 communities visited, few households had transitioned to improved latrines, citing the price as the main barrier, with the average cost of an improved latrine including the slab reported to be 25,000 francs or fifteen dollars altogether. In addition to SILC groups, Budikadidi should encourage a social cohesion option for those who are unable to be a part of a SILC group. Households can come together correctly to save money and purchase their improved latrines as a group. Purchasing materials in bulk, together as a group, can also lower the cost for the households. One mason mentioned that if he can construct the slabs in bulk, he will be able to charge five dollars for a slab rather than six dollars. Budikadidi improved latrines include a thatched roof, brick superstructure, improved brick flooring, a wooden or zinc door, and a reinforced cement slab. Although these other components are important, the priority should be on the sanitary slab and the compacted earth flooring. Households should be able to choose from varying styles of superstructures, doors, and roofing. The recommendation is that the project team develop a catalog of improved latrines for households to choose from, as well as action plans for upgrading from the basic latrine model. The catalog can include a list of improved latrines available in Budikadidi communities that members can choose from. For those households that choose not to invest immediately in improved latrines, it is important to encourage that compacted earth flooring or some natural cohesive composite is installed in traditional latrines so that they do not fill up quickly. 22) Train masons on developing business plans and marketing. The project should integrate the development of a business plan and marketing of concrete slabs and improved latrines as part of its mason training. Information on marketing is essential to masons, who are typically involved in home construction, who could benefit from learning how to build demand, market Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 71 their product in their own/ and neighboring communities, establish financing, and mass fabricate latrines through cost-saving methods. Demonstration slabs developed during mason training should be reserved for the most vulnerable homes in the community who have demonstrated positive behaviors related to the adoption of improved WASH practices. 23) Improve the uptake of improved WASH practices. Uptake of improved WASH practices is challenging, especially in environments with limited previous exposure to WASH initiatives such as villages in the Budikadidi project area. Community members in both P1 and P2 villages appeared informed of the benefits of WASH practices. Most households visited able to report disease related to poor WASH practices (cholera, typhoid, stomach parasites, and common diarrhea); they were not applying critical practices. During the evaluation period, it was difficult to tell whether failure to adopt improved WASH practices is due mainly to access or motivation to change behaviors. It is encouraging to note that Budikadidi plans to re-assess the seven norms of Village Assaini and adapt lessons learned. The recommendation is that the project team work closely with PRO-WASH, a five-year program focused on providing technical assistance. Training and knowledge-sharing opportunities with USAID/Food for Peace partners to assist with a broader assessment of tools and methods needed to strengthen the seven norms. It would also be beneficial to integrate an assessment of WASH practices in P1 communities, focusing the planned TIPS research on learning and subsequent program adaptations. WASH messaging has focused on conveying health-related information to promote hygiene and sanitation practices. Budikadidi should consider identifying other non-health approaches to improve WASH practices. The project could work with village chiefs and CACs to present a few monthly model households who could have access to free 20-liter jerry can of water for one week (reward-based approach). Another suggestion is to promote economic incentives, rather than focusing on health-related benefits. The acceptability and effectiveness of emotional and psychosocial triggers proven to motivate adoption of improved WASH- related behaviors in other development contexts14, rather than only maintaining the present focus on health-related motivators, should also be explored. Using results-based incentives and employing models promoting social rather than individual change should be considered. Communities visited have trouble adopting the Budikadidi water quality home-based treatment. If the water treatment is an option, in-line chlorination at the borehole source should be considered to reduce the burden on women who need to boil water daily for drinking or help reduce the cost of purchasing aquatabs. Lastly, the project should pay careful attention to the hygienic upkeep of the borehole facilities. A water committee takes water from a random jerry can to refill a borehole to increase the hydraulic pressure of the foot pump. This is a hazardous behavior that can increase contamination of the entire water supply system. If this is necessary to increase the water flow, this should be done by no more than two water committee members at times when water collection is not scheduled. Only one cleaned jerry can should be 14 Curtis VA, Danquah LO, Aunger R V. Planned, motivated and habitual hygiene behaviour: an eleven country review. Heal Educ Res. 2009;24(4):655–73. Aunger R, Schmidt W, Ranpura A, Coombes Y, Maina PM, Matiko CN, et al. Three kinds of psychological determinants for hand￾washing behaviour in Kenya. Soc Sci Med. 2010;70(3):383–91. Curtis V, Kanki B, Cousens S, Diallo I, Kpozehouen A, Sangare M, et al. Evidence of behaviour change following a hygiene promotion programme in Burkina Faso. Bull World Heal Organ. 2001;79(6):518–27. IMPEL | Implementer-Led Evaluation & Learning 72 Recommendations designated for this purpose, and the water committee members in contact with the borehole should have his/her hands cleaned with soap before contact with the borehole. SP 2.1, 2.2, 2.3: Inclusive social and economic well-being 24) Identify and accelerate components of the agricultural approach that show promise for widespread uptake. Aspects of the agriculture approach, which is intended to contribute to both income and nutrition, are struggling to reach scale and viability. Some components, such as the market and PG approaches, appear still to be in the "refine" phase. It is critical that the technical staff and leadership assess the current agricultural strategy and set of approaches to assess whether they can be implemented on a large enough scale to deliver on the anticipated results as planned. Those components that do have potential for wide range uptake should be scaled up quickly, while others that do not show promise should be eliminated. 25) Encourage community members to adhere to permagarden techniques recommended by the project. Many community members are not adopting agricultural techniques critical to the success of PG. Villagers may be convinced to follow these techniques (e.g., ensuring the appropriate mix of inputs, digging 40 cm when preparing the soil) if they can see successful PG and talk directly to the villagers who have implemented these gardens. Efforts designed to encourage application of appropriate PG methods could involve the organization of training sessions and visits to villages where PG have been highly productive. Field agents in charge of providing oversight at times exhibited poor understanding of the approach and could also benefit from additional training. Systematic and quality supervision by Budikadidi staff is needed to ensure that permagarden activities adhere to appropriate techniques. 26) Tailor permagarden strategies to address the challenges faced during the dry season. Promotion of PG during the dry season should be made in villages with easy access to water. The project should encourage the use of durable fencing, using materials such as moringa oleifera coupled with bamboo palms and reeds, to prevent infiltration of stray domestic animals. 27) Ensure that legal documents authorizing the use of the lowlands are in place for market gardens. The ET identified several cases where landowners refused to allow farmers groups to continue to use lowlands at a time when crops were already mature and ready for harvesting. In other cases, agreements with landowners regarding land use did not last the duration of the project period. Farmer groups must establish formal agreements with landowners regarding the use of lowlands throughout the project period. 28) Provide more technical support to OPs, focusing on sustainable land and water management practices. While in theory, Budikadidi is promoting sustainable practices, such as simple irrigation and drainage, organic composting, and mulching, these critical approaches are not being adequately adopted by many farmer groups who are implementing market gardens in the lowlands. Investigations should be carried out to understand why these key practices are not followed. Certain field agents need additional training to ensure that they can provide appropriate technical advice and oversight. 29) Improve marketing of market garden produce. Poor planning led to the overabundance of produce in some markets located in proximity to communities visited. Improved planning using predictions of crop production and availability in markets and demand for produce at different Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 73 times of the year should avoid flooding of markets with the same produce, which leads to discounted sales and low profit. The project needs to speed up the process of strengthening value chain activities by establishing better linkages with critical stakeholders (transporters, buyers) in the private sector. Efforts designed to ensure that crop production is adequately linked and responsive to the agricultural value chain need to be strengthened. 30) Accelerate the process of obtaining and distributing improved seeds. To make up for critical delays in activities, Budikadidi must accelerate the process of obtaining and distributing improved quality seeds so that they are widely available to OPs. Only if adequate seeds are available can Budikadidi achieve objectives related to food production and improving food security. 31) Establish effective collaborations with the various partners involved in food security. Many government organizations and implementing partners are working on agriculture activities in the health zones where Budikadidi is operating. Coordination and information sharing with these organizations, including those involved in the seed sector (e.g., PTF, COPROSEM, APSKO, CARG), has been inadequate. Agricultural activities would benefit from increased collaboration with these organizations. The involvement of government technical partners in the oversight, monitoring, and supervision of participants at all levels where these partners are functioning (provincial, territorial/zonal, and local) is critical to sustainability over the long-term. 32) Ensure that improved agriculture and maintenance techniques promoted through activities are applied. Evaluation results showed that improved agriculture techniques were followed in some communities but not others. Further investigation is needed to understand why critical techniques such as preparation of organic manure and using minimum tillage to decrease workloads and improve crop production are not widely adopted. Application of maintenance techniques, such as planting in a line, adequate spacing of plants, mulching, turning over, and allowing air to circulate in the soil, was also variable. Budikadidi needs to adapt strategies aimed to increase adoption that these practices. In this effort, Budikadidi will need to ensure that field agents are properly trained and able to transmit technically sound messages appropriately. 33) Introduce different tree species in reforestation activities. The project should consider adapting the selection of tree species to include trees preferred by participants and which provide nutritional food that could supplement household intake. Fruit trees or trees where caterpillars or weevils—insects highly popular for food consumption--thrive should be considered. Any changes in tree species will need to be approved by the USG as stipulated in the environmental plans. 34) Strengthen composting techniques and implement integrated pest control. While organic composting is supposed to be in place, there was little sign that organic composting is being implemented. The project needs to understand why organic composting is not being applied and address barriers to improve local capacities.The Safer Use Action Plan (SUAP) has not yet been approved by the Bureau Environmental Officer (BEO), and without that approval the project cannot implement integrated pest management. Budikadidi and USAID need to take measures needed to ensure that these strategies designed to protect the environment are implemented as planned. IMPEL | Implementer-Led Evaluation & Learning 74 Recommendations 35) Establish and empower a local structure that can lead the rehabilitation of feeder routes. The project identified "feeder roads" that are key to enhancing linkages between productive agricultural and market systems. While progress has been made in rehabilitating and maintaining roads, a structure is needed to ensure ownership and the sustainability of road rehabilitation. As part of this effort, youth groups that have played an integral role in road rehabilitation and maintenance need to be provided adequate tools and equipment to continue road work. 36) Allow participants to select animals for animal raising projects. Animal husbandry is designed to enhance social cohesion and increase the availability of protein-rich foods in the household diet. The introduction of goats coincided with an epidemic and led to high mortality; goats were not the preferred choice of animals of SILC group members, and are a poor selection regarding project objectives. Animals preferred by participants, such as rabbits, which have a short reproductive cycle and can be managed by women, should be introduced as soon as possible to meet nutritional objectives. 37) Encourage SILC groups to implement income-generating activities. Loans from SILC groups are often used for immediate family needs such as payment for school and medical care. Groups that used credit to initiate income-generating activities were often located in areas closer to markets. As group savings increase, groups should be encouraged to invest in local initiatives that have proven to generate revenue. Also, small grants designed to encourage groups to start business activities have not been widely used and seem to offer limited livelihood options. The project could benefit by consulting with participants to identify other contextually viable and promising income generating activities. Promotion of locale activities that are acceptable, such as beekeeping, and could build on other Budikadidi interventions (forestation in this case), should be considered. 38) Encourage agricultural and other groups to formalize their organizations. Budikadidi structures are recognized by the CAC and village leaders, but do not have official government recognition or legal rights, making them vulnerable once the project ends. The project should consider assisting groups in the development of official operational guidelines and in obtaining documents that formalize their associations and give them legal recognition and protection. Monitoring, Evaluation, Accountability, and Learning (MEAL) 39) Integrate implementation quality monitoring systems into the existing M&E system, and ensure that inclusion across all technical teams. The project would benefit from having these parallel systems integrated into the existing system (CommCare and Power BI). The M&E team should work with all technical sectors to help them develop and use implementation quality monitoring tools that meet the management needs of each team. Training and other routine support such as supervision will also be needed. 40) Conduct an assessment of the accessibility and responsiveness of the accountability mechanisms from the communities’ perspective. It is important to ensure that the accountability mechanism is accessible to participants and responsive to their needs. CRS staff mentioned that an evaluation of the accountability system planned for October, 2019. This should be prioritized, and the results should be used to inform adjustments to the system. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 75 41) Add a recorded message to the Green Line when it is outside of operational hours. When a participant calls the Green Line outside of the operational hours, a recorded message could communicate when they can call back to speak with an operator. A message could also provide information on other available feedback mechanisms (e.g. suggestion boxes) or what to do in case of an emergency that requires an immediate response, such as sexual exploitation or violence. 42) Formalize a process to track feedback that is given verbally to ensure that participants' feedback is responded to adequately and promptly and that verbal feedback included in the analysis of trends. Budikadidi’s system for tracking feedback received through the Green Line and suggestion boxes is strong, but it omits feedback received verbally. Some data indicated that verbal feedback preferred by participants and maybe the only option for participants who do not have access to a cell phone and located in villages that do not have a suggestion box or those who are illiterate. 43) Formalize the process of learning, integrate it with the planning process, and ensure that it is designed to meet the needs of each technical sector. Project leadership should develop a structured learning process and ensure that learning activities are designed to be responsive to the needs of each sector. Refine & Implement (R&I) 44) Strategically prioritize which studies should be conducted, ensuring that they can realistically conducted during the refinement period. Better prioritization and limiting the number of the studies conducted for future R&I DFSAs would reduce the management burden. Allowing partners to spend more time identifying the right consultants, give the consultants more time in the field, and increase the likelihood that the findings of the studies are used to adapt the project design. While acknowledging that it may vary by context, Budikadidi leadership recommended that FFP conduct an ex-post review of which formative research studies were most influential in informing project design to help future R&I DFSAs with this prioritization. The evaluation team agrees that R&I, particularly at this early stage, could benefit by gathering information across DRC DFSAs to determine which formative studies elicited most relevant information regarding activity design and implementation and why. Honing in on a reduced list of key formative studies will enhance efficient use of funds and time during the initial R&I phase. 45) Ensure that a culture of adaptive management continues throughout implementation. The R&I approach should integrate adaptive management throughout implementation. The ET observed differences between the technical sectors, and between senior and junior-level staff within sectors, in their plans for and commitment to adaptive management. Budikadidi should ensure that staff trained properly on the R&I approach as they are onboard and ensure that they understand the role of adaptive management and project refinement beyond the refinement year. Reports should be made available in both French and English to ensure that all staff can review the R&I findings. Efforts should be made to make R&I reports available to the broader development community in the DRC. IMPEL | Implementer-Led Evaluation & Learning 76 Recommendations Cross-Cutting 46) Strengthen linkages with Purpose 2 (livelihoods) activities. The project's theory of change shows linkages between the agriculture/income pathways and the nutrition pathways. However, the health and nutrition team (especially below senior management level) does not have enough familiarity with the targeting, strategies, and timing of the livelihoods work to understand how it should be supporting health and nutrition at the household level. There are missed opportunities to create synergies – such as linking food production activities to cooking demonstrations, speaking to the role of agricultural activities on nutrition through CG, connecting improved hygiene and sanitation efforts to SILC groups, and referring households during home visits to specific agricultural activities. Focused efforts are needed to help strengthen linkages between the agriculture/livelihoods work (P2) and nutrition outcomes (P1), including:  Orientation of all health/nutrition staff on the strategies and targeting of all P2 activities;  Development of joint messages/goals among P2 and health/nutrition teams;  Leveraging CG to emphasize linkages between P2 activities and nutrition;  The timely availability of improved seeds that produce highly nutritious crops;  Improved coordination of cooking demonstrations and agriculture production to strengthen access to ingredients and ensure recipes perfectly timed with the availability of ingredients produced in communities;  Strengthened coordination between nutrition field agents and agricultural agents (polyvalents); and  Joint field visits/supervisions with health/nutrition and agricultural staff. 47) Increase communication and transparency around incentive structures. To date, it appears that each project sector/strategy is determining the incentive structure that makes the most sense for that activity. Research suggests that the size of incentives relative to others’ incentives can be more important to recipient satisfaction than absolute size or fairness of the incentive. On the receiving end, communities perceive disjointed and unfair systems. For example, LMs, who are not receiving any monetary compensation, feel that incentive structures devalue their contributions. Their discontent is enhanced as community members believe LMs are benefitting more than they are (since other similar community actors do periodically receive cash for various reasons such as transport or a fee-for-service plan). In the future, the project should improve transparency with communities regarding incentive structures. Communication about incentives available in the community needs to come from a high level and cover all project activities (i.e. not depend on the sector-specific project staff, as some communities believe it is the person in charge of their sector that is withholding benefits). Going forward, CRS should ensure all new incentives/motivation structures discussed with the full project team (not left to the discretion of sectors), and community representatives, so that potential perceptions of "unfairness" can be addressed before they create problems. 48) Maintain the same numbers of field agents so that the DFSA can gain traction. CRS should be commended for establishing a sound infrastructure related to the foundational purpose involving the revitalization or establishment of the CAC community governance structure and critical aspects of objective one related to health, nutrition, and WASH. The establishment of a Mid-Term Evaluation of the Budikadidi DFSA in the DRC Recommendations 77 solid, well-integrated governance and CG approach can be attributed, at least in part, to the technical assistance and ongoing oversight and monitoring provided by field staff living in DFSA communities who are trained in technical areas and program implementation. However, these recently established structures are still at the inception phases and are very fragile. Moreover, several of the WASH activities and agricultural initiatives, which are central to increasing food security, generating revenue, and providing essential household foods needed for the nutrition approach and critical to the overall success of the project, lag. In this regard, the Year 4 PREP must revisit the plan to reduce field staff so that the structures put into place can continue to gain the traction needed for the project to succeed, and those components that lag receive adequate attention. 49) Increase time in Mbuji Mayi and field by key personnel. The Chief of Party and the Health, Nutrition and WASH lead are splitting their time between Kinshasa and Mbuji Mayi. These key personnel both have highly relevant experience, strong technical backgrounds, and outstanding leadership skills, and are critical to the overall success of the project. However they were not always fully up to date on aspects of field activities, underscoring that periodic absences \ impact the quality and oversight of implementation. The Budikadidi staff and project activities, in general, would tremendously benefit if key personnel spent more time in Mbuji Mayi and the field sites, particularly those where underperformance has been observed thus far (e.g. in the Year 3 / FY2019 ARR). . In general, technical staff would gain a better understanding of activity weaknesses and strengths by spending more time in the field, observing the status of activities firsthand and interacting directly with field agents and DFSA participants. 50) Work with consortium partners to reduce the disparities in salary structures across different consortium field agents. The Budikadidi consortium includes an important mix of international and local implementing partners with a range of expertise and experience critical to the success of the project activities. These organizations have different organizational and salary structures. During the evaluation, it became apparent that salary disparities of field agents maintaining similar roles and responsibilities leads to demotivation and discontent among those with less compensation. CRS should work with partner management to discuss challenges associated with unequal salary structures and evaluate feasibility of adjusting salary structures to reduce the disparities within the project team (ensure equitable pay among field agents carrying out the same work). 51) Establish strategies to ensure sharing of information and lessons learned across DFSAs. While there is some information sharing across DFSAs, these sessions appear to be informal and sporadic and generally involve COPs. MTE results illustrate that DFSAs operating in the DRC face many similar challenges and deficits in approaches. Failure to share both challenges and successes is a missed opportunity. DFSAs could benefit from holding regular conference calls, involving technical leads and alternating the focus on sectors. Site visits may also provide valuable and insights information that could strengthen activities. CRS should engage in (or even initiate) these and other learning opportunities involving experience and document sharing. IMPEL | Implementer-Led Evaluation & Learning 78 Annex 1: Budikadidi Activity Descriptions ANNEX 1: BUDIKADIDI ACTIVITY DESCRIPTIONS Governance A first step in introducing Budikadidi activities was the establishment of “Cellule d’animation Communautaire” (CACs), which serve as the foundational structure for community development in all 474 communities. Positioning the CAC at the center of Budikadidi community activities coincides with a 2018 MOH national policy mandating that CACs play a pivotal role in development activities, thus securing government recognition and important linkages between community members and government officials. The CAC has a mandate of ensuring integral, multi-sectoral development within the targeted communities. Its multidimensional structure is comprised of influential leaders, including representatives from the nutrition, health, WASH, agriculture and education sectors, who provide oversight to ensure that local structures include appropriate members and are functional. From the outset, Budikadidi pushed for transparent elections of the CAC governing body, with clear mandates for each of the leadership positions. Following the elections, CAC members were trained in topics central to good governance, including self-assessment, action planning, administrative and financial management, and transparency, with the overall goal to strengthen understandings of principles related to democracy and accountability. Additional training related to establishment of internal rules and regulation, social mobilization and interpersonal communication techniques, supervision, and monitoring and evaluation were provided with the goal to ensure that CACs have the tools to support food security, livelihoods and improved nutrition activities, as well as engage and collaborate with state structures and community-based organizations on ongoing interventions. Once established, Budikadidi began to work through CACs as an entry point into communities. For instance, the identification of Care Group promoters and Lead Mothers, as well as the establishment of Care Groups, was done in conjunction with the CACs. CACs are expected to have a development plan for their villages involving implementation and monitoring of all Budikadidi activities. Gender Sociocultural and economic factors enforce the persistence of male dominance in Kasai. In rural areas, females often do not attend school, and even when they do, the majority drop out at a young age. Wage employment opportunities for women are limited. The combination of these factors causes families to put pressure on their daughters to marry at a young age, in most cases during adolescence or early adulthood. Husbands are generally far older, ensuring male dominance. As a result, few women living in rural areas are self-reliant or have the skills and resources needed to allow them to progress economically and socially. These factors underline the need for long-term strategies designed to modify gender discrimination and equities. Budikadidi is implementing a range of activities to strengthen the quality of male-female intra￾household relationships and to improve gender equity across intervention areas. Budikadidi aims to have 30% CAC governing bodies represented by women, giving women community authority and visibility. CAC members have been trained to consider gender when managing community issues. Gender activities include the Faithful House and Natural Family Planning approaches, which are designed to improve relationship skills through effective couple communication in areas related to household finances, gender roles, sexual intimacy and problem-solving. The approaches promote responsible and positive parenting, task sharing, joint decision making, prevention of gender-based violence, shared economic responsibilities and natural child spacing methods. The Real Father approach uses positive masculinity to increase awareness about gender inequity and strengthen male support to Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 1: Budikadidi Activity Descriptions 79 wives and partners. An overarching goal is to involve women in household decision-making, including decisions related to the use of resources, and to encourage males to play a more positive role in supporting the welfare, health, and nutrition of family members. Participation in SILC groups provide women with opportunities to invest in small income generating endeavors. Other activities focus on improving knowledge to empower women. For example, Care Groups are led by women who provide education on nutrition, water, hygiene, and sanitation to mothers. During Care Groups, women are also provided information on gender issues. Numeracy and literacy training impart basic reading and math skills. Youth activities such as listening clubs ensure male and female involvement in ongoing activities. These activities aim to expose youth to important gender-based topics, such as the consequences of early marriage and pregnancy, and encourage more equitable gender relations for future generations. Resilience Shocks identified during the R&I phase included inter-community conflicts, drought, torrential rains and associated phenomena, climate change, disease and epidemics that affect humans, animals and plants, and insufficient application of laws such as those related to the legal marrying age and migration. Activities have been put in place to improve the resilience of participating communities to these potential shocks. Some communities are organizing negotiation mechanisms for conflict management and mediation. Rain gauges have been set up to track rainfall and nutritional surveillance serve as early warning systems of participating communities. Other activities such as producer groups, market gardens and permagardens, and SILC groups, are all designed to diversity food production and contribute to resilience. The project is currently working to strengthen value chains that involve SILC groups investing in livestock, marketing agents participating in the purchase and sale of animal feed, and community actors offering veterinarian services, that will serve to improve the nutritional status and protect livestock against disease. Nutrition and Health Budikadidi aims to achieve sustainable reductions in chronic malnutrition. Budikadidi’s core strategy for achieving uptake of improved health and nutrition behaviors during the 1000-day period is the CG approach. Through this cascade approach, project field agents train part-time volunteer community promoters, who impart knowledge and skills to part-time volunteer LMs through structured learning modules, who in turn support households to adopt improved behaviors. The project theory of change was used to identify five priority nutrition behaviors:  Mothers of children 6 to 24 months provide their children each day meals containing at least four food groups (animal products, staple foods, legumes and fruits)  Mothers of children 6 to 24 months who raise domestic animals clean/sweep animal excrement outside of areas where they place children when preparing food  Mothers of children 0 to 6 months exclusively breastfeed (provide only breastmilk to) their infants  Pregnant women consume iron-rich foods at least three times per week  Women in the union who do not wish to have another pregnancy at the current time have access to a modern, natural family planning option Four priority behaviors were determined to be appropriate for CG activities. The final priority nutrition behavior related to the use of natural family planning methods was determined to be better delivered through the FH as a couples-based platform. CG were also employed to support WASH behaviors. Budikadidi carried out formative research to develop the learning modules and worked with local health system actors to develop a preliminary list of CG sessions and key content. In total, Budikadidi IMPEL | Implementer-Led Evaluation & Learning 80 Annex 1: Budikadidi Activity Descriptions developed 18 CG session plans, divided into three main modules (infant and child health and nutrition (8 sessions); health and nutrition during pregnancy and lactation (6 sessions); cholera prevention and response (4 sessions). Each session is accompanied by visual learning materials and a detailed lesson plan that promoters use to impart content to LMs. All materials were reviewed by Budikadidi’s gender team to incorporate gender-transformative themes as appropriate. Some lessons include participatory activities – such a dietary diversity game – to reinforce learning. All lesson content and images were reviewed by a technical team, including local health system staff, and pretested with community health workers (relais communataires), women, men, and elders in the community before being rolled out. Working with communities, the project identified "promoters" who were literate, had experience with health promotion work and were willing to work as part-time volunteers. Most communities identified existing RECOs to serve in this capacity. Each promoter has the responsibility of supporting and supervising about half a dozen CG, each consisting of about 12 LMs. In line with standard CG practice, the project team trains promoters every few months on groups of 4-6 related lessons, which are cascaded to LMs monthly. Households were grouped into sets of 15, from which each group was to select a LM to represent them in the CG (on the basis of a selection criteria including the woman is considered a role model, is willing to volunteer, and has some literacy). According to project plans, LMs organized into groups of about 12 called a "CG." LMs attend CG meetings lasting one to two hours twice per month. Following receipt of lessons, LMs share lessons to each of her 14 supported households through home visits or a group session. Due to women's high time burden for domestic activities and limited decision-making power – both identified during the project's gender analysis - Budikadidi encouraged LMs to prioritize household visits to limit the time women are away from home, and also to interface with both women and men in the family. Lessons and content delivered through CG are primarily focused on women as principal caregivers, though efforts are also made to reach men through home visits. LMs were also tasked with encouraging uptake of relevant health services (particularly antenatal care, post-natal care, and well-baby visits), conducting mid-upper arm circumference (MUAC) measurements to identify and refer children with acute malnutrition, and monitoring uptake of (visible) project promoted behaviors such as installation of latrines, tippy taps, dish drying racks and home gardens. In complement of the CG approach, Budikadidi also organizes cooking demonstrations using local foods, uses radio to disseminate recipes and health/nutrition content, and incorporates health and nutrition content in other project approaches including literacy activities and the FH model. WASH Budikadidi has been working closely with the national water and sanitation committee (CNAEA), the National Rural Water Service (SNHR), the provincial Department of Public Health (DPS), and Health Zone Offices (BCZ) to improve and sustain water and sanitation services. Budikadidi plans to coordinate with the Integrated Health Program (IHP) to provide WASH services to its target beneficiaries. UNICEF has assessed some communities reached by Budikadidi and is planning to provide water supply rehabilitation in some areas. At the community level, the Budikadidi team is working with the CAC and village chief to identify, train, and equip water committees to provide oversight and governance on water supply systems. Budikadidi has provided training and support to a water technician, the health zone water officers, CACs, and water committees to conduct quarterly water quality monitoring of all new and rehabilitated water infrastructures. Budikadidi is working through two distinct community structures to improve access to sanitation. Communities impacted or certified through the UNICEF Healthy Village (VEA) model, referred to as P2 communities, are receiving a modified VEA approach based on results from a Knowledge, Attitude, Practice (KAP) survey. The second community structure, P1 communities, have not received any VEA interventions. In these communities, Budikadidi is Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 1: Budikadidi Activity Descriptions 81 implementing the community-led total sanitation approach (CLTS), which mobilizes communities to lead and facilitate actions towards an open defecation free (ODF) status. The CAC and the village chief designated community WASH champions or natural leaders to play a key role in ensuring that households contribute to the village ODF status and transition to the utilization of improved sanitation latrines. The project also leverages the CG model to reinforce sanitation access and promote improved WASH practices for pregnant and lactating mothers and caregivers of children under two years of age. LMs are trained to accompany natural leaders at the household level to discuss positive WASH practices related to handwashing, latrine use, and water quality. Agriculture Budikadidi operates in a mining area where artisanal diamond mining constitutes a central economic activity. The push to encourage inhabitants to alter the economic focus from mining to agricultural and to improve agricultural production and diversity requires intensive sensitization and the application of a complementary set of activities designed to address key factors that impact on crop production and food security. In this effort, Budikadidi has established a variety of sub-structures directly and indirectly affecting agricultural activities, including OPs, SILC groups, Farmer Leader groups, CLE, CG, FH groups, and youth groups and organizations. These structures are working to transform agricultural practices, as well as the mentality of community members, who have demonstrated strong interest in participating. Budikadidi agricultural activities attempt to provide solutions to address community needs, as well as specific problems affecting food security, particularly related to land management, sustainable soil management, farming techniques, seed technology, climate change and its impact on the environment, and access to local credit. Specific activities designed to increase crop production and diversity and improve access to market systems include market gardens, permagardens, strengthening value chains, and road rehabilitation. Many activities have been initiated but do not necessarily cover the whole geographical project area. Critical delays in implementation related to USAID environmental compliance requirements, procurement challenges, processes associated with testing improved seed varieties, technical staffing changes, and late distribution of seeds, as well as design issues that failed to consider contextual factors that interfered with the proposed approaches, have led to entire farming seasons being missed and a generally slow start. M&E Budikadidi’s mobile data collection system uses the software CommCare and the Power BI dashboard. Field agents carry tablets to enter ongoing data related to DFSA activities. Activity feedback mechanisms include a free of charge hotline, suggestion boxes located in communities, and verbal feedback to field agents. R&I A total of eight studies were conducted during the refinement year, which were designed to address 52 key questions that identified as knowledge gaps across eight sectors. The eight studies were: 1. Gender analysis 2. Seed Security Systems Assessment 3. Value Chain Analysis 4. Local Food Assessment 5. Initial Environmental Examination 6. Resilience Study (SenseMaker) 7. Fertility Awareness Method (FAM) study 8. Barrier Analysis Findings from initial formative research informed activity implementation and modifications in the TOC. IMPEL | Implementer-Led Evaluation & Learning 82 Annex 2: Evaluation Objectives and Questions ANNEX 2: EVALUATION OBJECTIVES AND QUESTIONS 1. OBJECTIVE 1: Review the quality of program service delivery related to the different themes and systems in addressing chronic food insecurity and child malnutrition with the targeted clients, taking into account contextual changes that may have occurred since the inception of the activities. Illustrative questions under this objective may include: 1.1. What factors appear to enhance or detract from the quality? This should include factors that are within the manageable interest/control of the IP as well as those outside the control of IPs. 1.2. What systems are used to capture, document, and share lessons learned? How are these lessons learned then use to continue ensuring/improving program quality? How is stakeholder and/or participant feedback/perspectives included in this system? 1.3. How well have the interventions met planned schedules and outputs? What factors promoted or inhibited adherence to schedules and planned interventions? How were challenges managed? 1.4. What are the strengths of and challenges to the overall design, implementation, management, communication, and collaboration so far? What factors appear to promote or impede activity operations or effective collaboration and cooperation among the various stakeholders? Are DFSA assumptions still relevant? 1.5. In each technical sector, what are the strengths of and challenges to the efficiency of interventions’ implementation and acceptance in the community? How well do implementation processes adhere to underlying principles and activity protocols? 2. OBJECTIVE 2: Identify evidence of changes (positive and negative, intended and unintended) associated with program interventions. This will include identifying factors that appear to promote or hinder women and men’s, and young people’s, food security and safety. Illustrative questions under this objective may include: 2.1. What changes do community members and other stakeholders associate with the DFSA interventions? What factors appear to promote and deter the changes? How do the changes correspond to those hypothesized by the DFSA TOC? 2.2. To what extent are there observable gender discrepancies? Discrepancies between youth and adults? How has the activity design accounted for these discrepancies and “course corrected” to mitigate these discrepancies? 2.3. How could the DFSA be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the DFSA TOC be refined or modified? 3. OBJECTIVE 3: Assess the quality, relevance and efficacy of the DFSA design, taking into account whether activities are contextually appropriate, address critical needs, and maintain standards that can impact on positive change. Evaluate relevant programmatic principles such as whether approaches are human centered, evidence based, systems oriented, focused on the most vulnerable, designed for multiple interventions to target the same household, and focused on quality. Illustrative questions under this objective may include: Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 2: Evaluation Objectives and Questions 83 3.1. How do community members and field agents perceive the activities in relation to local relevance and priority needs? To what extent do local perceptions regarding relevance and quality of the activities affect implementation of activities by field agents and participation of community members? 3.2. How could the DFSA design be adjusted to better coincide with local contextual factors and priorities and at the same time meet desired objectives and longer term sustainability? 4. OBJECTIVE 4: Assess the degree and benefits of efficacious coordination, collaboration, and convergence with locally-based DFSA partners and external organizations that are critical to achieve DFSA goals and purposes. Illustrative questions under this objective may include: 4.1. What mechanisms are in place to engage with local organizations, government entities, private sector, and/or other stakeholders? How strong are these relationships? To what extent will these stakeholders be able to sustain or take over interventions currently being implemented by the IP? 4.2. To what extent do local stakeholders help or hinder the implementation of activities? What has the IP done to overcome these challenges? 4.3. How has the prime implementing partner worked with other USAID activities in DRC? What has worked well and what has not worked well? What factors are within the manageable interest of the IP and what are not? How has the IP managed the relationship/s with other USAID activity implementing partners? 4.4. How has information/data been shared across partners? Other stakeholders? 5. OBJECTIVE 5: Assess early evidence of sustainability produced by the DFSA activities thereby determining the extent to which outcomes, systems, and services are designed and being implemented to continue after the DFSA ends. Illustrative questions under this objective may include: 5.1. To what extent has the activity documented and implemented a sustainability strategy? ● How familiar with this plan are senior-level management and technical staff? ● How familiar with this plan are “frontline” staff? ● How familiar with this plan are subs and other partners? ● How familiar with this plan are other indirect partners (e.g. local government, local businesses)? ● How familiar with this plan are participants? ● What activities do participants perceive to be most relevant/critical in regard to longer term sustainability post implementation? 5.2. How will positive outcomes be sustained after the activity ends? Which local systems/structures/entities will sustain these outcomes? 6. OBJECTIVE 6: Assess the appropriateness and quality of the R&I approach and activities. How well did the R&I approach work in DRC across the 3 DFSAs. Illustrative questions under this objective may include: 6.1. How have R&I activities been used to inform the design and ongoing modification of activity implementation? How were findings from the studies reflected in the M&E documents (e.g. TOC, LogFrame, IPTT) and DIP? IMPEL | Implementer-Led Evaluation & Learning 84 Annex 2: Evaluation Objectives and Questions 6.2. To what extent has R&I been used beyond the initial refinement period and during the implementation phase? 6.3. Have the DFSAs used R&I for DFSA staff capacity building and community engagement and development? 6.4. What are some of the challenges faced in implementing R&I activities and what adjustments need to be made to address and minimize these factors in the future? 6.5. How realistic was the budgeting and timeline for R&I? Did the DFSA significantly over or under￾spend? 6.6. How can the R&I model be improved to better ensure that R&I activities are effectively used to strengthen design and implementation of DFSAs? [Key Recommendation]? 7. OBJECTIVE 7: Determine the appropriateness and effectiveness of interventions focused on cross￾cutting themes for the activities. This includes cross cutting interventions designed to improve gender and equity in decision making to achieve food security outcomes and targeting youth to improve their access to, participation in, and benefit from DFSA interventions. This also includes analysis of the effectiveness of activities designed to strengthen governance at the local level, in an overall effort to empower communities to sustain improvements in food security and nutrition, and environmental risk and mitigating the impacts of climate change. 8. OBJECTIVE 8: Recommend adjustments to program implementation or design and explain how these changes would improve program outcomes and sustained impact. Illustrative questions under this objective include: 8.1. Assess personnel capacity, management of human resources (including sub-partners), implementation systems, operations and logistics, and M&E systems 8.2. Assess scope and quality of DFSA activities in relation to overall strategy: What aspects of program design, mechanics related to implementation, R&I, and human resource capacity facilitate or interfere with implementation of activities? How can these factors be modified to strengthen implementation of activities. Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 3: Field Work and Analysis Calendar 85 ANNEX 3: FIELD WORK AND ANALYSIS CALENDAR Activity August September October 4 1 1 1 8 25 1 8 1 5 22 2 9 6 1 3 2 0 2 7 Design Planning In-country Planning & Training Data Collection CRS/MC Validation Workshop 28 Mission Debrief 2 7 Data Analysis Report Writing IMPEL | Implementer-Led Evaluation & Learning 86 Annex 4: Evaluation Team Members, Roles ANNEX 4: EVALUATION TEAM MEMBERS, ROLES NAME ROLE ORGANIZATION RESPONSIBILITIES Lauren Blum Team Lead Evaluation Design, Coordination Tulane University Evaluation Design; Fieldwork; Analysis; Technical Input and Coordination of Report Writing Eric Mafuta Governance, Gender, Youth, Resilience Tulane University (Kinshasa School of Public Health) Fieldwork, Analysis, Sector Report Writing Adam Trowbridge M&E, R&I, sustainability FFP Washington Fieldwork; Analysis; Sector Report Writing Precious Sancho WASH FFP Washington Fieldwork; Analysis; Sector Report Writing Dieudonné Mbuka Agriculture & Livelihoods FFP Kinshasa Fieldwork Meredith Stakem Nutrition & Health CRS Fieldwork; Analysis; Sector Report Writing Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 5: Evaluation sites and locations in order of the evaluation visit 87 ANNEX 5: EVALUATION SITES AND LOCATIONS IN ORDER OF THE EVALUATION VISIT Health zone Health area Village Approximate distance from health zone capital in kilometers Miabi Miketa Kasenga 39 Cilundu Cilundu Bakua Lukanda 2 6 Kasansa Katenga Lubuwa Ditalala 2 Miabi Katende Bena Mbiya 14 Cilundu Bashingala Bakua Mbiya 27 Kasansa Nsengansenga Kanyinda 24 IMPEL | Implementer-Led Evaluation & Learning 88 Annex 6: Data Collection Instruments ANNEX 6: DATA COLLECTION INSTRUMENTS GUIDE D’ENTRETIEN POUR LES INFORMATEURS CLES GOUVERNANCE Administration du consentement Identification du répondant Age : Sexe : Année d’expérience au poste : Lieu : Questions Instruction : ces questions sont indicatives, vous pouvez ajouter de fouilles ou sauter de questions en fonction de réponses du répondant 1. Depuis quand est-ce que les activités du projet [nom du projet] ont-elles commencé dans cette région/province ? 2. Depuis quand est-ce que vous êtes impliqué dans les activités du projet [nom du projet] ? 3. Quelles sont les activités mises en œuvre dans le cadre du projet [nom du projet] dans cette province ? 4. Comment ces activités contribuent-elles à la Théorie du Changement du projet ? 5. Dans lesquelles de ces activités êtes-vous personnellement impliquées ? 6. Quelles sont les formations que vous aviez reçues dans ce cadre de ces activités ? 7. Quelles sont les études qui ont été conduites dans le cadre de R&I ? Comment ont-elles été utilisées dans la conception et la mise en œuvre du projet ? Donnez quelques exemples ? Après la mise en œuvre, quelles ont été les autres études diligentées ? 8. Par rapport à la gouvernance, comment les activités dont vous aviez la responsabilité sont-elles gérées dans le cadre du projet à différents niveaux ? Quels sont les acteurs impliqués ? Quels sont les rôles joués par chaque acteur ? Comment tout ce montage est-il coordonné? Supervision ? communication ? 9. Par rapport à la gouvernance au niveau de la communauté, quels sont les organes mis en place dans le cadre du projet pour assurer la gouvernance des activités ? Qui sont les acteurs ? Comment ont-ils été choisis ? Comment ces organes fonctionnent-ils ? Comment cela est-il coordonné ? 10. Quelle est la place de la CAC dans cette organisation de la gouvernance ? Comment fonctionne-t￾elle ? Quelles sont ces activités ? Comment assure-t-elle la coordination des activités ? A qui rend-t￾elle compte ? Sous quelle forme ? Comment les vulnérables sont-ils pris en compte dans ce montage ? Quels sont les aspects genre que le projet a initié dans le cadre de gouvernance locale ? 11. Comment les activités de la CAC ou de l’organe de gouvernance mis en place arrivent aux entités gouvernementales concernées ? Quelles sont ces entités ? Comment sont-elles impliquées ? 12. Quelles sont les changements que vous aviez noté dans cette région que vous pouvez attribués aux activités du projet ? 13. Quels sont les défis ? Comment aviez-vous adressé ces défis ? 14. Quels sont les éléments du contexte qui peuvent avoir influencé les résultats des activités du projet ? Comment ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 89 15. Quels sont les stratégies que le projet [nom du projet] met en œuvre pour assurer que les activités dans cette région continueront après la fin du projet ? les participants ont-ils été informés de la durée du projet et de quand le projet prendra-t-il fin ? Quels sont les signes actuels qui montent que les activités du projet peuvent continuer même si les appuis du projet s’arrêtaient ? Remerciement IMPEL | Implementer-Led Evaluation & Learning 90 Annex 6: Data Collection Instruments GUIDE D’ENTRETIEN INDIVIDUEL POUR LES INFORMATEURS AU NIVEAU LOCAL Administration du consentement Identification du répondant Age : Sexe : Profession : Année d’expérience au poste : Lieu : Questions Instruction : ces questions sont indicatives, vous pouvez ajouter de fouilles ou sauter de questions en fonction de réponses du répondant 1. Depuis quand est-ce que les activités du projet [nom du projet] ont-elles commencé dans ce village ou dans cette communauté ? 2. Depuis quand est-ce que vous êtes impliqué dans les activités du projet [nom du projet] ? 3. Quelles sont les activités mises en œuvre dans le cadre du projet [nom du projet] dans cette province ? 4. Connaissez-vous la Théorie de changement ? Comment ces activités contribuent-elles à la Théorie du Changement du projet ? 5. Dans lesquelles de ces activités êtes-vous personnellement impliquées ? 6. Quelles sont les formations que vous aviez reçues dans ce cadre de ces activités ? 7. Par rapport à la gouvernance au niveau de la communauté, quels sont les organes mis en place dans le cadre du projet pour assurer la gouvernance des activités ? Qui sont les acteurs ? Comment ont-ils été choisis ? Comment sont-ils structurés ? la place de la femme dans cette organisation ? Comment ces organes fonctionnent-ils ? A-t-elle un plan d’action ? Comment cette planification s’est-elle faite ? Comment cela est-il coordonné ? A qui rend-t-elle compte ? 8. Quelle est la place de la CAC dans cette organisation de la gouvernance ? Comment fonctionne-t￾elle ? Quelles sont ces activités ? Comment assure-t-elle la coordination des activités ? A qui rend-t￾elle compte ? Sous quelle forme ? Comment les vulnérables sont-ils pris en compte dans ce montage ? Quels sont les aspects genre que le projet a initié dans le cadre de gouvernance locale ? 9. Comment les activités de la CAC ou de l’organe de gouvernance mis en place arrivent aux entités gouvernementales concernées ? Quelles sont ces entités ? Comment sont-elles impliquées ? Quelles sont les relations avec les relais communautaires ? le Comité de développement de l’aire de santé ? 10. Quelles sont les changements que vous aviez noté dans ce village/cette région que vous pouvez attribués aux activités du projet ? 11. Quels sont les défis ? Comment aviez-vous adressé ces défis ? 12. Quels sont les éléments du contexte qui peuvent avoir influencé les résultats des activités du projet ? Comment ? 13. Quels sont les stratégies que le projet [nom du projet] met en œuvre pour assurer que les activités dans cette région continueront après la fin du projet ? Les agents du projet vous ont-ils informé de la durée du projet et de quand le projet prendra-t-il fin ? Quels sont les signes actuels qui montent que les activités du projet peuvent continuer même si les appuis du projet s’arrêtaient ? Remerciement Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 91 GUIDE D’ENTRETIEN INDIVIDUEL POUR LES INFORMATEURS ENTITES GOUVERNEMENTALES Administration du consentement Identification du répondant Age : Sexe : Profession : Année d’expérience au poste : Lieu : Questions Instruction : ces questions sont indicatives, vous pouvez ajouter de fouilles ou sauter de questions en fonction de réponses du répondant 1. Connaissez-vous le projet [nom du projet] qui réalise des activités dans cette région/province/zone de santé/aire de santé ou dans ce village ? 2. Depuis quand est-ce que les activités du projet [nom du projet] ont-elles commencé dans ce village ou dans cette communauté ? 3. Quelles sont les activités mises en œuvre dans le cadre du projet [nom du projet] ici ? 4. Quels sont à votre avis les objectifs poursuivis par le projet et Comment ces activités contribuent￾elles à l’attente de ces objectifs ? 5. Dans lesquelles de ces activités êtes-vous personnellement impliquées ? comment et pourquoi ? (Si le répondant affirme ne pas être impliqué, demandez-lui comment pense-t-il être impliqué dans l’avenir ? Quels sont les éléments de cette implication ? 6. Quelles sont les formations que vous aviez reçues dans ce cadre de ces activités ? 7. Par rapport à la gouvernance au niveau de la communauté, quels sont les organes mis en place dans le cadre du projet pour assurer la gouvernance des activités ? Qui sont les acteurs ? Comment ont-ils été choisis ? Comment sont-ils structurés ? la place de la femme dans cette organisation ? Comment ces organes fonctionnent-ils ? A-t-elle un plan d’action ? Comment cette planification s’est-elle faite ? Comment cela est-il coordonné ? A qui rend-t-elle compte ? 8. Quelle est la place de la CAC dans cette organisation de la gouvernance ? Comment fonctionne-t￾elle ? Quelles sont ces activités ? Comment assure-t-elle la coordination des activités ? A qui rend-t￾elle compte ? Sous quelle forme ? Comment les vulnérables sont-ils pris en compte dans ce montage ? Quels sont les aspects genre que le projet a initié dans le cadre de gouvernance locale ? 9. Comment les informations sur les activités de la CAC ou de l’organe de gouvernance mis en place ou encore les activités du projet vous arrivent-elles ? Comment ces informations sont-elles exploitées ? Pourriez-vous nous montrer un rapport que vous aviez reçu ? Pourriez-vous nous montrer le document de rapportage que vous utilisez pour les données de votre entité ? (Enquêteur, vérifiez si les évidences existent sur l’utilisation de données venant du projet dans les statistiques de l’entité) 10. Quelles sont les changements que vous aviez noté dans ce village/cette région que vous pouvez attribués aux activités du projet ? 11. Quels sont les défis que rencontrent les activités du projet Comment ces défis sont-ils adressés ? ? 12. Quels sont les éléments du contexte qui peuvent avoir influencé les résultats des activités du projet ? Comment ? IMPEL | Implementer-Led Evaluation & Learning 92 Annex 6: Data Collection Instruments 13. Quels sont les stratégies que le projet [nom du projet] met en œuvre pour assurer que les activités dans cette région continueront après la fin du projet ? Les agents du projet vous ont-ils informé de la durée du projet et de quand le projet prendra-t-il fin ? Quels sont les signes actuels qui montent que les activités du projet peuvent continuer même si les appuis du projet s’arrêtaient ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 93 Guide d’entretien avec le leader de communautés/membre CAC/CODESA Introduction Texte sur le consentement éclairé. Données générales Date: Sexe: Âge: Rôle dans la communauté: Zone de Santé: Aire de la Santé: Ville ou village: 1. Êtes-vous au courant des activités du (Nom du projet) dans votre village? 2. Si oui, quelles activités du (Nom du projet) ont été mises en œuvre dans votre village? Quels sont les objectifs généraux des activités? o Depuis quand ces activités ont-elles commencé? o A votre avis, lesquelles des activités ont apporté les plus grands bénéfices ou les changements positifs dans votre village/communauté? o Est-ce qu’il y a des activités que vous pensez qu’elles ne sont pas réellement adaptées aux besoins de membres de votre village/communauté ? Si oui, lesquelles? o Est-ce qu’il y a des barrières majeures à la sécurité alimentaire ou à la nutrition dans ce village/communauté que les activités du projet n’ont pas actuellement résolu ? Quelles sont ces barrières et quelles sont les propositions que vous avez ? 3. Quels canaux de communication le personnel du projet du (nom du projet) utilisent-t-ils pour communiquer avec vous et vos pairs (autres leaders communautaires ou les membres de communauté) ? À quelle fréquence communiquent-ils / vous rencontrent-ils? Que pensez-vous de la communication (y compris pour votre feedback au projet) ? Aimeriez￾vous voir des changements dans la façon dont la communication est-elle réalisée (y compris pour votre feedback au projet) ? 4. Si leader communautaire : Comment êtes-vous impliqué dans les activités? Comment travaillez-vous ou communiquez-vous avec la CAC/ le CDC sur les activités en cours? o Si membre CAC/CDC : Comment êtes-vous impliqué dans les activités? Comment travaillez-vous ou communiquez-vous avec les leaders communautaires sur les activités en cours? o Comment travaillez-vous avec les autres secteurs dans le cadre du projet [nom du projet] ? Comment cela était avant le projet [nom du projet] ? 5. Pour les CAC dans les zones CRS: 5a. Parlez-moi un peu du CAC de votre village. o Depuis quand la CAC a-t-elle été mise en place ? o Comment cela a-t-il été fait dans la réalité ? (Comment cela a commencé ? comment elle a été mise en place ?...) o Quelle est la composition de votre CAC ? Combien y a-t-il de femmes ? o Quels rôles ces femmes ont-elles dans la CAC? IMPEL | Implementer-Led Evaluation & Learning 94 Annex 6: Data Collection Instruments o Quelles sont les relations entre la CAC et les relais communautaires ? o Quelles sont les relations entre la CAC de ce village et le Comité de développement de l’aire de santé ? 5b. Est-ce qu’il y avait des activités menées spécifiquement avec CAC ou avec son appui? Si oui, quelles sont ces activités? A. Quelles formations aviez-vous reçu du projet? i. Quel était le programme et quelle était la durée de ces formations? ii. Qui a assure la formation? Quelles méthodes ont été utilisées ? iii. Qui ont participé à cette formation ? iv. Est-ce qu’il y a eu des recyclages? B. Quel appui ou quelle supervision recevez-vous actuellement du projet ? i. Qui a fourni l’appui? ii. Quelle est la fréquence de visites d’appui? Aviez-vous des feedbacks sur la fréquence de visite d’appui? iii. Pouvez-vous décrire qu’est-ce qu’une visite d’appui typique inclut? C. Est-ce que vous avez un plan d’action communautaire? i. Si oui, parlez-moi un peu du processus de développement du plan : 1. Qui sont impliqué dans le développement du plan? 2. Comment les éléments sont sélectionnés et priorisés pour être inclus dans le plan ? ii. Si oui, parlez-moi un peu de la manière dont vous utilisez ce plan 1. Qui utilise ce plan? et comment? 2. A quelle fréquence est-ce que vous faites un suivi de ce plan ? 3. Quel est l’état actuel des activités du plan? 4. Vous est-il déjà arrivé d’ajouter ou d’enlever les activités du plan? Pourquoi ou pourquoi pas? 5. Aviez-vous le sentiment que ce plan est utile pour votre village/ communauté? 6. Est-ce qu’il existe des priorités qui ne sont pas incluses dans le plan ? Pourquoi ou pourquoi pas? iii. Si oui, à votre avis, quels sont les succès les plus importants et les avantages de ce plan? iv. Si oui, quels ont été les défis majeurs dans le processus de planification ou de l’utilisation de ce plan? D. Comment est-ce que la CAC est impliquée dans les autres activités menées par le projet Budikadidi dans ce village ou communauté? E. Est-ce que vous pensez que la CAC est quelque chose qui appartient au projet et qui va s’arrêter avec le projet ou quelque chose qui va continuer même au-delà de la fin du projet? Qu’est-ce que vous pensez va rester identique après que le projet soit terminé ? qu’est-ce qui va changer et pourquoi ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 95 6. Savez-vous comment les décisions ont été prises quant aux activités qui seraient mises en œuvre dans votre village? Si oui, comment est-ce fait? Que pensez-vous de cette approche? Quelles suggestions, le cas échéant, avez-vous pour améliorer la façon dont les décisions ont été prises? (Instruction : dans cette question, nous voulons savoir dans quelle mesure les dirigeants ont été consultés et dans quelle mesure les membres de la communauté ont participé aux décisions liées aux activités locales.) 7. Savez-vous comment les participants au projet ont été sélectionnés? Quels critères, le cas échéant, ont été utilisés pour sélectionner les participants? Comment les dirigeants de la communauté étaient-ils impliqués dans la sélection? Que pensez-vous de ce processus et comment pourrait-il être amélioré? 8. Les personnes les plus vulnérables du village ont-elles participé aux activités du projet? Si oui, quels critères ont été utilisés pour identifier les plus vulnérables? Que pensez-vous de la sélection des plus vulnérables? Que pensent les autres membres du village de ne pas participer? Cela provoque-t-il des conflits ou de la mauvaise volonté chez les participants et les non-participants du village? 9. Avez-vous une liste des participants au projet dans votre village? Si oui, puis-je voir cette liste? À quelle fréquence cette liste est-elle mise à jour? Les participants au programme ont-ils un identifiant indiquant qu'ils sont des participants? 10. Avez-vous mis en place un mécanisme pour que les gens puissent formuler des suggestions ou des plaintes concernant l'intervention du projet dans votre village? Si oui, comment cela fonctionne? Savez-vous si les gens utilisent ce mécanisme? Pourquoi ou pourquoi pas? Si elle a été utilisée, pouvez-vous me donner un exemple de la façon dont elle a été utilisée et quels en ont été les résultats? NB. Posez la question 10 en considérant le projet comme acteur pour voir s’il a été mis en place un mécanisme de redevabilité 11. À votre avis, quel est le principal succès du projet dans votre village? Pourquoi voyez-vous cela comme un grand succès? Pouvez-vous me donner des exemples précis? 12. Quels sont, selon vous, les principaux défis et problèmes du projet du (nom du projet) dans votre village? Pourquoi? Pouvez-vous me donner des exemples de la façon dont cela a causé des problèmes? 13. Quels sont les changements survenus dans votre village qui sont attribuables au projet? i. En eau, hygiène et assainissement ii. Santé et nutrition o santé maternelle et infantile o Nutrition o Santé de jeunes et adolescents iv. sécurité alimentaire iv. transformation/commercialisation agricole v. atténuation/résolution des conflits vi. l'alphabétisation? IMPEL | Implementer-Led Evaluation & Learning 96 Annex 6: Data Collection Instruments vii. genre 14. A votre avis, entre les hommes et les femmes, qui sont les personnes les plus impliquées dans les activités du projet ? Qu’est-ce que vous pensez de cela ? Depuis que le projet a commencé, quels changements percevez-vous en rapport avec le rôle de la femme ou le rôle de l’homme dans le ménage ? 15. Comment pensez-vous que les activités réalisées dans le cadre du projet peuvent continuer lorsque le projet [nom du projet] va s’arrêter ? 16. Avez-vous des suggestions que nous pourrions transmettre au projet afin d'en accroître les avantages pour les membres de votre communauté? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 97 GUIDES NUTRITION/SANTÉ Entretiens avec les informateurs clés Entretiens avec les informateurs clés à l’aide de questions NUTRITION/SANTÉ posées aux COPs, référents techniques 1. Quand avez-vous rejoint le projet ? Quel type de formation ou d’orientation avez-vous reçu depuis le début du projet ? 2. Comment décririez-vous les buts et objectifs généraux du projet (nom du projet) ? 3. Quels sont les principaux indicateurs de résultats liés à la nutrition et à la santé ? 4. Y a-t-il eu des changements (prévus ou imprévus) dans la mise en œuvre des activités de nutrition et de santé depuis le début du projet (nom du projet) ? Dans l’affirmative, pourquoi ces changements ont-ils été apportés ? Posez des questions approfondies sur les changements intervenus dans les domaines relatifs à : i. La programmation des activités ii. L’approvisionnement en matériel et fournitures iii. La formation iv. La supervision des travailleurs v. Aux changements dans le suivi et l’évaluation des services vi. La rémunération ou autres formes de motivation. Posez des questions approfondies sur les différents types de personnel, comme par exemple les : i. Superviseurs/animateurs ii. Promoteurs d’hygiène iii. Mama leaders vii. Le renouvellement du personnel [Est-ce que chaque membre du personnel a une description de poste claire ?] 5. Quelles activités de recherche formative ont-elles été menées avant la mise en œuvre des activités de nutrition et de santé ? Comment les décisions concernant les activités de recherche formative à mener ont-elles été prises ? Quelle est votre évaluation de la qualité et du contenu de la recherche formative ? (Posez des questions approfondies pour obtenir des informations sur chacune des études de recherche formative portant sur la nutrition et la santé). Comment la recherche formative a-t-elle été utilisée dans l’élaboration et la mise en œuvre des programmes ? (Posez des questions approfondies pour déterminer si le projet a élaboré une stratégie de SBC et dans quelle mesure cette stratégie peut ou ne peut pas être utilisée.) 6. Dans quelle mesure le Service de R&I a-t-il été utilisé au-delà de la période de perfectionnement initiale et pendant la phase de mise en œuvre ? Dans quelle mesure avez-vous utilisé l’approche du Service de R&I dans le cadre de la mobilisation et du développement communautaires ? Avez-vous observé des difficultés à mettre en œuvre l’approche R&I ? Citez-en quelques-unes. 7. Quelles activités de nutrition et de santé sont censées être offertes par l’intermédiaire du projet (nom du projet) ? En vous basant sur l’examen de la documentation et la séance d’information du partenaire d’exécution, examinez les éléments suivants : i. Groupes de donneurs de soins/ Groupes des pères IMPEL | Implementer-Led Evaluation & Learning 98 Annex 6: Data Collection Instruments ii. Démonstrations de pratiques appropriées de transformation et de préparation d’aliments nutritifs iii. Identification, référencement et suivi des enfants malnutris iv. Séances de conseil et d’information v. Jardinage vi. L’élevage vii. Soins prénatals viii. Planification familiale et des soins de santé et espacement des grossesses ix. CPS x. Traitement de la malnutrition aiguë sévère xi. Prise en charge rapide des cas de diarrhée, de respiration accélérée et de fièvre xii. Autres activités de la stratégie de SBC, p. ex. les mass media/radio, théâtre/spectacles, mobilisation communautaire xiii. Activités visant à encourager les pères à jouer un rôle de soutien dans la prise de décision du ménage en matière de nutrition et de santé xiv. Création/support de VSLAs xv. Renforcement des systèmes de santé, l’accent étant mis sur l’ICCM xvi. Mercy Corps : Channels of Hope et Positive Peer Couples xvii. Santé des jeunes et des adolescents/espaces d’accueil sûrs 8. Quelles sont les activités de nutrition et de santé de l’enfant qui sont réellement menées à bien ? Santé des jeunes et des adolescents/espaces d’accueil sûrs 9. Quelles sont les activités de nutrition et de santé de l’enfant qui sont réellement offertes par l’intermédiaire de (nom du projet) ? Sonder pour comprendre dans quelle mesure les activités censées être offertes sont effectivement mises en œuvre. Vérifier si le plan de mise en œuvre détaillé est pratique ou non. 10. Nous avons remarqué que tous les villages ne mettent pas en œuvre les mêmes activités. Veuillez nous faire savoir comment sont prises les décisions concernant les activités qui sont mises en œuvre et celles qui ne le sont pas. 11. Veuillez décrire la théorie du changement (TDC) qui est suivie pour l’ensemble du projet. Dans quelle mesure vous référez-vous à la TDC en tant qu’activités en cours ? À quoi ressemblait la TDC au début des activités du projet ? À quoi ressemble la TDC à l’heure actuelle et, si des changements ont été apportés, pourquoi ces changements ont-ils été apportés ? 12. Quels groupes et organisations partenaires participent aux activités de nutrition et de santé ? Comment les différents partenaires sont-ils impliqués ? Quelles organisations sont responsables de quelles activités ? 13. Qui sont les acteurs clés sur le terrain en charge de la mise en œuvre des activités de nutrition et de santé ? Comment le personnel a-t-il été recruté ? Posez des questions approfondies pour comprendre le degré d’implication d’acteurs de terrain tels que les : i. Mama leaders ii. Promoteurs d’hygiène iii. Superviseurs et facilitateurs iv. Membres du CODESA Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 99 v. Relais vi. Les fournisseurs de services basés sur les établissements vii. Autres 14. Quels types de motivations sont-elles offertes aux différentes catégories de personnel ? Quelles sont les structures de supervision mises en place pour assurer le suivi des activités ? Comment, concrètement, les différentes catégories de personnel sont-elles supervisées ? Comment le personnel est-il évalué de façon continue ? 15. Quelles ont été les premières étapes de la mise en œuvre des activités ? Quelles sont les étapes ultérieures qui ont été réalisées au cours du déroulement des activités ? Quand l’ensemble des activités ont-elles commencé ? 16. Nous savons que les DFSA incluent de nombreux thèmes transversaux et que FFP souhaite que les différents secteurs soient intégrés. Veuillez décrire la manière dont le projet aborde les thèmes transversaux, la manière dont les activités de santé et de nutrition sont intégrées aux autres objectifs, Dans quelle mesure le ciblage garantit/ne garantit pas leur intégration entre objectifs, comment les interventions sont évaluées, et vos évaluations de l’efficacité de ces activités. Il serait utile d’en apprendre davantage sur les réussites et les difficultés rencontrées dans la mise en œuvre de ces activités. Posez des questions approfondies pour obtenir des informations relatives à : i. Genre et équité ii. Jeunesse iii. Atténuation du changement climatique 17. Y a-t-il une personne responsable de la stratégie de SBC au sein du projet ? Si oui, posez les questions suivantes à la personne responsable : Combien de comportements le projet tente-t-il de changer ? Y a-t-il eu des discussions sur la priorisation des comportements ? Dans l’affirmative, comment cela a-t-il été fait ? Combien de membres du personnel connaissent la stratégie de SBC ? Dans quelle mesure le personnel a-t-il participé à l’élaboration de la stratégie de SBC ? Dans quelle mesure le projet parvient-il à bien cerner les obstacles et les facteurs facilitant ou catalyseurs des comportements considérés comme prioritaires ? La stratégie de SBC comprend-elle une combinaison d’approches autres que la communication interpersonnelle et les médias de masse ? [Posez des questions approfondies sur la mobilisation communautaire et le plaidoyer et d’autres approches non liées à la communication.] 18. Quelles sont les approches de changement social et comportemental qui sont mises en œuvre dans le cadre de la stratégie de la stratégie de SBC et/ou de la théorie du changement du projet ? Comment les approches de SBC ont-elles été élaborées ? Comment les activités de la stratégie de SBC sont-elles suivies ou supervisées ? Posez des questions approfondies sur la liste de contrôle ou d’autres outils. Quels changements, le cas échéant, ont été apportés aux activités de la stratégie de SBC depuis le début des activités du projet ? Si des changements ont été apportés, expliquez pourquoi. Comment le projet assure-t-il la qualité des activités de la stratégie de SBC, ainsi que la capacité du personnel et la capacité des mères leaders, etc. Comment les messages SBC sont-ils séquencés ? Comment les messages de la stratégie de SBC sont-ils adaptés aux différents groupes/publics cibles ? Vérifier si le matériel et les messages de la stratégie de SBC sont différents pour les pères, les hommes, les adolescents, les grands- IMPEL | Implementer-Led Evaluation & Learning 100 Annex 6: Data Collection Instruments parents, etc. Veuillez décrire le rythme des activités de la stratégie de SBC, par exemple, quelle est la fréquence 1) des rencontres/interactions avec les mères et 2) des événements radio et représentations théâtrales ? 19. Pour le projet Mercy Corps : Nous avons remarqué que l’une des principales activités de votre projet est la formation de divers personnels de santé, y compris la formation sur la planification familiale. Pouvez-vous décrire de quelle manière le projet assure la qualité de la mise en œuvre par le personnel de santé et les agents de santé communautaires, au-delà de la formation en salles de classe ? Quels obstacles le projet a-t-il rencontrés dans la mise en œuvre des activités de formation et des activités de SBC axées sur la planification familiale ainsi que sur la planification et l’espacement sains de la grossesse ? 20. Quels sont les systèmes de suivi et d’évaluation disponibles pour mesurer les activités en cours en matière de nutrition et de santé ? Comment ces systèmes sont-ils utilisés pour éclairer la mise en œuvre du programme ? 21. Y a-t-il eu des changements contextuels (p. ex. des questions liées à la sécurité, aux structures du gouvernement national et des collectivités locales, à l’environnement, à l’économie locale, etc.) depuis le début du projet qui ont pu influencer les activités du projet ? Dans l’affirmative, comment ont-elles affecté les activités du projet ? 22. Comment décririez-vous la coordination et la collaboration avec d’autres organismes partenaires, des institutions gouvernementales et des collaborateurs locaux ? Comment fonctionne la coordination et la planification : Posez des questions approfondies sur : i. La programmation des activités ii. Les réunions avec les organisations partenaires iii. Le suivi et évaluation des activités iv. Les systèmes mis en place pour apprendre/partager les activités en cours, les leçons apprises et les défis sur le terrain v. D’autres acteurs susceptibles de présenter un intérêt comprennent : le Projet PROSANI/IHP, l’UNICEF, le PAM, les autres activités dans le domaine de la santé et de la nutrition. 23. Nous savons que la durabilité est l’un des principaux objectifs des projets DSFA. Comment mesurez-vous la durabilité ? Quelles preuves, le cas échéant, de la durabilité produite par les activités du projet sont-elles disponibles à ce jour ? Selon vous, que faut-il faire pour accroître la probabilité de durabilité au terme de l’aide fournie par le projet DFSA ? 24. Dans quelle mesure le projet a-t-il atteint les objectifs liés aux indicateurs clés de nutrition et de santé ? Posez des questions approfondies sur : i. L’autonomisation des mères dans la prise de décision en matière de nutrition et de santé ii. L’identification et la prise en charge des enfants malnutris iii. La disponibilité et utilisation accrues d’aliments nutritifs pour l’alimentation des enfants iv. La réduction de la prévalence de la malnutrition 25. Si les objectifs du projet ne sont pas atteints, est-ce dû à des échecs sur le plan de la conception ou de la mise en œuvre du programme ? 26. Quels sont certains des obstacles qui entravent l’atteinte de ces indicateurs de résultats clés ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 101 27. Pouvez-vous décrire les types de ressources et d’assistance technique que le projet, les bureaux régionaux ou l’administration centrale vous ont fournies depuis que vous travaillez sur ce projet ? 28. Quelles sont certaines des faiblesses de l’approche du projet (nom du projet) ? Quels sont les principaux obstacles qui empêchent le projet (nom du projet) d’atteindre son plein potentiel ? Quelles sont les solutions pour surmonter ces obstacles ? 29. Quelles sont vos recommandations pour améliorer les activités futures en matière de nutrition et de santé au cours du projet ? Entretiens avec les informateurs clés Entretiens avec les informateurs clés à l’aide de question sur la nutrition et la santé posées aux Représentants des organisations partenaires, organismes publiques (au niveau national ou régional) 1. Quand votre organisation/agence a-t-elle commencé à travailler avec le projet ? Quel est le rôle général que votre organisation/agence assume dans le projet ? [Pour le gouvernement : quel sentiment d’appropriation ressentez-vous à l’égard de ce projet ?] 2. Comment décririez-vous les buts et objectifs généraux du projet (nom du projet) ? 3. Y a-t-il eu des changements (prévus ou imprévus) dans la mise en œuvre des activités de nutrition et de santé depuis le début du projet (nom du projet) ? Dans l’affirmative, pourquoi ces changements ont-ils été apportés ? Posez des questions approfondies sur les changements intervenus dans les domaines relatifs à : i. La programmation des activités ii. L’approvisionnement en matériel et fournitures iii. La formation iv. La supervision des travailleurs v. Les changements dans le suivi et l’évaluation des services vi. La rémunération ou autres formes de motivation. Posez des questions approfondies sur les différents types de personnel, comme par exemple les : i. Superviseurs/animateurs ii. Promoteurs d’hygiène iii. Mama leaders 4. À quelles activités de nutrition et de santé votre organisation/agence participe-t-elle ? En vous basant sur l’examen de la documentation et la séance d’information du partenaire d’exécution, examinez les éléments suivants : i. Groupes de donneurs de soins ii. Démonstrations de pratiques appropriées de transformation et de préparation d’aliments nutritifs iii. Identification des enfants malnutris iv. Séances de conseil et d’information v. Jardinage vi. L’élevage vii. Soins prénatals IMPEL | Implementer-Led Evaluation & Learning 102 Annex 6: Data Collection Instruments viii. Planification familiale ix. CPS x. Traitement de la malnutrition aiguë sévère xi. Prise en charge rapide des cas de diarrhée, de respiration accélérée et de fièvre xii. Autres activités de la stratégie de SBC xiii. Activités visant à encourager les pères à jouer un rôle de soutien dans la prise de décision du ménage en matière de nutrition et de santé xiv. Création/support de VSLAs xv. Renforcement des systèmes de santé, l’accent étant mis sur les ICCM 5. Quelles sont les activités de nutrition et de santé de l’enfant qui sont réellement offertes par l’intermédiaire du projet (nom du projet) ? Posez des questions approfondies pour comprendre dans quelle mesure les activités censées être offertes sont effectivement mises en œuvre. 6. Votre organisation/agence a-t-elle participé à l’une quelconque des études « Refine & Implement » au cours de la première année de la mise en œuvre ? (Posez des questions approfondies pour obtenir des informations sur chacune des études de recherche formative portant sur la nutrition et la santé). De quelle manière la recherche formative a-t-elle été utilisée pour l’élaboration et la mise en œuvre des programmes ? 7. (DESTINÉ AUX PARTENAIRES SEULEMENT) Veuillez décrire la théorie du changement (TDC) qui est suivie pour l’ensemble du projet. Dans quelle mesure vous référez-vous à la TDC au fur et à mesure que les activités se poursuivent ? À quoi ressemblait la TDC au début des activités du projet ? À quoi ressemble la TDC à l’heure actuelle et, si des changements ont été apportés, pourquoi ces changements ont-ils été apportés ? 8. Quels groupes et organisations partenaires participent aux activités de nutrition et de santé ? De quelle manière les différents partenaires sont-ils impliqués ? Quelles organisations sont responsables de quelles activités ? 9. (DESTINÉ AUX PARTENAIRES SEULEMENT) Quels sont les acteurs clés de votre organisation sur le terrain en charge de la mise en œuvre des activités de nutrition et de santé dans le cadre de ce projet ? Comment le personnel a-t-il été recruté ? Posez des questions approfondies pour en savoir davantage sur l’implication d’acteurs de terrain tels que i. Mama leaders ii. Promoteurs d’hygiène iii. Superviseurs et facilitateurs iv. Membres du CODESA v. Relais vi. Les fournisseurs de services basés sur les établissements vii. Autres 10. Quels types de motivations sont-elles offertes aux différentes catégories de personnel ? Quelles sont les structures de supervision mises en place pour assurer le suivi des activités ? Comment, concrètement, les différentes catégories de personnel sont-elles supervisées ? Comment le personnel est-il évalué de façon continue ? 11. (DESTINÉ AUX PARTENAIRES SEULEMENT) Veuillez décrire toute capacité technique qui a permis de renforcer les capacités du partenaire technique en chef/principal qui vous a été fourni Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 103 depuis le début du projet. [Posez des questions approfondies sur la stratégie de SBC] Quels autres domaines techniques aimeriez-vous connaître, s’il y a lieu ? 12. Quelles ont été les premières étapes de la mise en œuvre des activités ? Quelles sont les étapes ultérieures qui ont été réalisées au cours du déroulement des activités ? Quand l’ensemble des activités ont-elles commencé ? 13. Quelles sont les approches de changement social et comportemental qui sont mises en œuvre dans le cadre de l’approche ? Comment les approches de la stratégie de SBC ont-elles été élaborées ? [Question approfondie : comment avez-vous été impliqué ?] Comment les activités de l’approche SBC sont-elles suivies ou supervisées ? {Posez des questions approfondies sur la liste de contrôle ou d’autres outils.} Quels changements, le cas échéant, ont-ils été apportés aux activités de l’approche SBC depuis le début des activités du projet ? Si des changements ont été apportés, expliquez pourquoi. Comment le projet assure-t-il la qualité des activités de l’approche SBC, ainsi que la capacité du personnel et la capacité des mères leaders, etc. Comment les messages de l’approche SBC sont-ils séquencés ? Comment les messages de la stratégie de l’approche SBC sont-ils adaptés aux différents groupes/publics cibles ? [Vérifier si le matériel et les messages du SBC sont différents pour les pères, les hommes, les adolescents, les grands-parents, etc.] Veuillez décrire le rythme des activités de l’approche SBC, par exemple, quelle est la fréquence 1) des rencontres/interactions avec les mères et 2) des événements radio et représentations théâtrales ? 14. Pour le projet Mercy Corps : Nous avons remarqué que l’une des principales activités de votre projet est la formation de divers personnels de santé, y compris la formation sur la planification familiale. Pouvez-vous décrire de quelle manière le projet assure la qualité de la mise en œuvre par le personnel de santé et les agents de santé communautaires, au-delà de la formation en salles de classe ? [Posez des questions approfondies sur le mentorat, la supervision, l’accompagnement.] Quels obstacles le projet a-t-il rencontrés dans la mise en œuvre des activités de formation et des activités de l’approche SBC axées sur la planification familiale ainsi que sur la planification et l’espacement sains de la grossesse ? 15. Quels sont les systèmes de suivi et évaluation disponibles pour mesurer les activités en cours ? Comment ces systèmes sont-ils utilisés pour éclairer la mise en œuvre du programme ? 16. Y a-t-il eu des changements contextuels (p. ex. des questions liées à la sécurité, aux structures du gouvernement national et des collectivités locales, à l’environnement, à l’économie locale, etc.) depuis le début du projet qui ont pu avoir une incidence sur ses activités ? Dans l’affirmative, comment sont-elles affectées par les activités du projet ? 17. Comment décririez-vous la coordination et la collaboration avec d’autres organismes partenaires, des institutions gouvernementales et des collaborateurs locaux ? Comment fonctionne la coordination et la planification : Posez des questions approfondies sur : i. La programmation des activités ii. Les réunions avec les organisations partenaires iii. Le suivi et évaluation des activités iv. Les systèmes mis en place pour apprendre/partager les activités en cours, les leçons apprises et les défis sur le terrain IMPEL | Implementer-Led Evaluation & Learning 104 Annex 6: Data Collection Instruments v. D’autres acteurs potentiels susceptibles de présenter un intérêt comprennent : Projet PROSANI/PHI, UNICEF, PAM, autres activités sanitaires et nutritionnelles. 18. Nous savons que la durabilité est l’un des principaux objectifs des projets DSFA. Comment mesurez-vous la durabilité ? Quelles preuves, le cas échéant, de la durabilité produite par les activités du projet sont disponibles à ce jour ? Selon vous, que faut-il faire pour accroître la probabilité de durabilité lorsque l’aide du DFSA arrivera à terme ? 19. Dans quelle mesure le projet a-t-il atteint les objectifs liés aux indicateurs clés de nutrition et de santé ? Posez des questions approfondies sur : i. L’autonomisation des mères dans la prise de décision en matière de nutrition et de santé ii. L’identification et la prise en charge des enfants malnutris iii. La disponibilité et utilisation accrues d’aliments nutritifs pour l’alimentation des enfants iv. La réduction de la prévalence de la malnutrition 20. Si les objectifs du projet ne sont pas atteints, est-ce dû à des échecs liés à la conception ou à la mise en œuvre du programme ? 21. Quels sont certains des obstacles à l’atteinte de ces indicateurs de résultats clés ? Pouvez-vous décrire les types de ressources et d’assistance technique que le projet, les bureaux régionaux ou l’administration centrale vous ont fournis depuis que vous avez travaillé sur le projet ? 22. Quelles sont certaines des faiblesses de l’approche du projet (nom du projet) ? Quels sont les principaux obstacles qui empêchent le projet (nom du projet) d’atteindre son plein potentiel ? Quelles sont les solutions pour surmonter ces obstacles ? 23. Quelles sont vos recommandations pour améliorer les activités futures en matière de nutrition et de santé au cours du projet ? Entretiens approfondis avec les agents de santé/chargés de la nutrition 1. Quand les activités relatives à la santé maternelle et à la nutrition et à la santé infantile ont-elles été introduites dans votre région ? En quoi consistent les activités ? D’après l’examen de la documentation et la séance d’information du partenaire d’exécution, posez des questions approfondies pour avoir des informations sur les éléments suivants : i. Groupes de soins ii. Démonstrations des pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs iii. Identification des enfants malnutris iv. Séances de conseil et d’information v. Jardinage vi. Élevage vii. Soins prénatals viii. Planification familiale ix. CPS x. Traitement des cas plus graves de malnutrition xi. Prise en charge rapide des cas de diarrhée, de respiration accélérée et de fièvre xii. Activités de l’approche SBC Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 105 xiii. Activités visant à encourager les pères à jouer un rôle de soutien dans la prise de décision du ménage en matière de nutrition et de santé xiv. Création/support de VSLAs xv. Renforcement des systèmes de santé, l’accent étant mis sur l’ICCM xvi. Genre xvii. Jeunesse 2. Quelles sont vos responsabilités concernant les activités de nutrition et de santé du projet (nom du projet) ? Quels types de membres de la communauté participent régulièrement à ces activités ? i. Promoteurs de l’hygiène, Mama Leader, relais, CODESA, poser des questions approfondies sur les activités offertes au niveau communautaire, les participants de la communauté, y compris les caractéristiques et le nombre de personnes qui y participent, et les programmations continues liées aux éléments suivants : i. Visites à domicile ii. Séances de conseil et de groupe avec des groupes d’alimentation de l’enfant iii. Démonstrations de pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs iv. Jardinage v. Détection des enfants malnutris vi. Référencement d’enfants malnutris ou souffrant de diarrhée, de respiration rapide ou de fièvre vii. Collecte des données, remplissage des formulaires mensuels et suivi des services viii. Supervision (pour les promoteurs de santé) ix. Autres activités de l’approche SBC ii. Pour les travailleurs en centre de santé, vérifiez les services offerts dans le centre de santé. Posez des questions approfondies pour en savoir davantage sur : i. Le traitement des enfants malnutris dans l’établissement de santé ii. La distribution de RUTF/Plumpy Nut aux enfants souffrant de malnutrition plus sévère. iii. Activités de la CPS iv. Séances d’information ou de conseils sur les stratégies de SBC 3. Quelle formation avez-vous reçue dans le cadre des activités de santé et de nutrition ? i. Quelle a été la durée de la formation et où a-t-elle eu lieu ? ii. Qui étaient les formateurs ? Pensez-vous qu’ils étaient qualifiés pour dispenser la formation ? Pourquoi ou pourquoi pas ? iii. Quelle a été la nature des thématiques abordées au cours de la formation ? Posez des questions approfondies sur le contenu de la formation en mettant l’accent sur les sujets suivants : i. ANJE ii. Visites à domicile iii. Séances de conseil et de groupe avec des groupes d’alimentation de l’enfant IMPEL | Implementer-Led Evaluation & Learning 106 Annex 6: Data Collection Instruments iv. Démonstrations de bonnes pratiques de transformation des aliments et de préparation d’aliments nutritifs v. Jardinage vi. Dépistage des enfants malnutris vii. Soins prénatals viii. Planification familiale ix. CPS x. Référencements d’enfants malnutris ou souffrant de diarrhée, de respiration rapide ou de fièvre xi. Traitement des cas plus graves de malnutrition xii. Collecte des données, remplissage des formulaires mensuels et suivi des services xiii. Supervision (pour les promoteurs de santé) xiv. Approches axées sur le changement de comportement xv. Genre iv. Pensez-vous que la formation a été suffisamment longue et adéquate pour que vous puissiez mener à bien les activités de nutrition infantile comme prévu ? Dans la négative, quelles étaient certaines des lacunes de la formation ? Quels sujets manquaient et devraient être modifiés ou ajoutés à la formation ? v. Y a-t-il eu un suivi de la formation ou du recyclage ? Dans la négative, estimez-vous qu’une formation de suivi soit nécessaire ? vi. Êtes-vous satisfait de l’ensemble de la formation ? Pourquoi ou pourquoi pas ? 4. Quels matériels et fournitures avez-vous reçus en rapport avec les activités de nutrition et de santé depuis le début du projet (nom du projet) ? (Demander à l’informateur de partager la documentation). a. À quelle fréquence recevez-vous des fournitures, y compris du matériel didactique, des semences et des outils pour le jardinage ? b. Les matériaux et les fournitures sont-ils suffisants pour mener à bien les activités du projet ? Veuillez expliquer. c. Vous arrive-t-il de manquer d’outils et de fournitures ? Si oui, à quelle fréquence ? Que fait-on fait lorsque cela arrive ? d. Êtes-vous satisfait du matériel et des fournitures reçus ? Pourquoi ou pourquoi pas ? 5. Quel type de supervision recevez-vous dans le cadre des activités de nutrition infantile du projet (nom du projet) ? i. Qui supervise vos activités professionnelles ? ii. À quelle fréquence la supervision est-elle censée avoir lieu et à quelle fréquence a-t-elle réellement lieu ? iii. Quand les visites de supervision ont-elles lieu, combien de temps durent-elles ? Que se passe-t-il pendant les visites de supervision ? Dans quelle mesure interagissez-vous avec votre superviseur pendant ces visites ? iv. Êtes-vous satisfait de la supervision ? Pourquoi ou pourquoi pas ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 107 v. Quel autre soutien, au-delà de la supervision, le projet vous apporte-t-il ? Posez des questions approfondies sur le mentorat, le coaching, etc. Et s’ils ont reçu des outils en cours d’emploi pour les aider dans leur travail.] 6. Comment assurez-vous le suivi des activités en cours ? Posez des questions approfondies pour en savoir davantage sur : i. Les formulaires à remplir ii. La fréquence et le mode de transmission des formulaires aux bureaux des superviseurs/du partenaire d’exécution 7. Nous savons que le genre est inclus dans le projet (nom du projet) et que FFP est en faveur de l’intégration du genre dans les différents secteurs. Pouvez-vous nous dire quelles approches vous utilisez en matière de genre et comment elles sont intégrées dans les activités de santé et de nutrition ? Comment les activités liées au genre sont-elles intégrées aux autres objectifs, tels que les moyens de subsistance, la gouvernance et le WASH ? En quoi le ciblage assure-t-il ou n’assure-t-il pas l’intégration entre les objectifs ? Que pensez-vous de ces approches ? Comment ces interventions se mesurent-elles et quelles sont vos évaluations de l’efficacité de ces activités ? Comment ont-elles été accueillies par les membres de la communauté ? Pouvez￾vous nous faire part de quelques réussites et des difficultés rencontrées dans la mise en œuvre de ces activités ? A votre avis, comment les approches genre peuvent-elles être améliorées ? 8. Nous savons que la jeunesse est incluse dans le projet (nom du projet) et que FFP est en faveur de l’intégration des jeunes dans les différents secteurs. Pouvez-vous nous dire quelles sont les approches que vous utilisez à l’égard des jeunes et comment elles sont intégrées dans les activités de santé et de nutrition ? Comment les activités liées au genre sont-elles intégrées aux autres objectifs, tels que les moyens de subsistance, la gouvernance et le WASH ? En quoi le ciblage assure-t-il ou n’assure-t-il pas l’intégration entre les objectifs ? Que pensez-vous de ces approches ? Comment ces interventions se mesurent-elles et quelles sont vos évaluations de l’efficacité de ces activités ? Comment ont-elles été accueillies par les membres de la communauté ? Pouvez-vous nous faire part de quelques réussites et des difficultés rencontrées dans la mise en œuvre de ces activités ? A votre avis, comment les approches genre peuvent￾elles être améliorées ? 9. Quels sont les facteurs qui vous motivent à mener des activités liées à la nutrition et à la santé dans le cadre du projet ? (Posez des questions approfondies sur les indemnités journalières, l’avancement professionnel, les conditions de travail, la reconnaissance communautaire et les autres avantages non financiers). Que pensez-vous de la façon dont on vous motive à mener des activités ? Avez-vous des documents d’identification officiels qui indiquent que vous travaillez sur le projet de (nom du projet) ? Dans l’ensemble, êtes-vous satisfait de votre rôle ? Pourquoi ou pourquoi pas ? 10. Avec qui collaborez-vous dans le cadre des activités en cours dans le domaine de la nutrition et de la santé ? Comment fonctionne la coordination des activités sur le terrain ? Quels systèmes, le cas échéant, ont été mis en place pour partager les enseignements tirés ou les difficultés rencontrées sur le terrain ? En cas de difficultés, comment les surmonter ? Veuillez décrire toutes les réunions de coordination qui ont lieu actuellement. IMPEL | Implementer-Led Evaluation & Learning 108 Annex 6: Data Collection Instruments 11. Dans quelle mesure êtes-vous en mesure de mettre en œuvre les services de nutrition et de santé conçus dans le cadre de l’approche du projet ? Quels sont les défis à relever ? Y a-t-il des aspects de l’approche que vous n’êtes pas en mesure de mettre en œuvre ? Si oui, pourquoi ? Comment les surmonter ? 12. Par rapport aux stratégies précédentes auxquelles vous avez participé, comment l’approche a-t￾elle changé la façon dont les mères et les autres donneurs de soins s’occupent de leurs enfants ? Selon vous, l’approche donne-t-elle aux mères, aux autres donneurs de soins et aux membres de la communauté les moyens de s’occuper de leurs enfants ? 13. Quelle est votre évaluation globale des activités de nutrition et de santé dans le cadre de l’approche du projet (nom du projet) ? Êtes-vous satisfait des activités ? Pourquoi ou pourquoi pas ? Les gens de votre communauté sont-ils satisfaits des activités ? 14. Selon vous, quelle est la tendance générale de la malnutrition infantile de votre zone de couverture ? Est-elle à la hausse ou à la baisse ou n’y a-t-il aucun changement ? Qu’est-ce qui explique cette tendance ? 15. Que recommanderiez-vous pour améliorer la nutrition et la santé de l’enfant dans le cadre des activités de DFSA ? Discussions de groupe Discussions de groupe avec des donneuses de soins aux jeunes enfants 1. Connaissez-vous le projet (nom du projet) ? Que pouvez-vous me dire sur le projet ? Quelles sont les activités menées ? Quelles activités liées à la santé et à la nutrition sont-elles menées ? Activités communautaires 2. Quand les activités de nutrition et de santé ont-elles été introduites dans votre région ? En quoi consistent les activités ? Sur la base de l’examen de la documentation et de la séance d’information du partenaire d’exécution, interrogez les personnes concernées sur différentes activités, telles que : i. Réunions des groupes de donneuses de soins ii. Démonstrations de pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs. iii. Visites à domicile iv. Dépistage des enfants malnutris v. Séances de conseil et d’information vi. Jardinage vii. L’élevage des animaux viii. Soins prénatals ix. Planification familiale x. CPS xi. Traitement des cas plus graves de malnutrition xii. Prise en charge rapide des cas de diarrhée, de respiration rapide et de fièvre xiii. Activités du SBC xiv. Activités visant à encourager les pères à jouer un rôle de soutien dans la prise de décision du ménage en matière de nutrition et de santé Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 109 xv. VSLAs 3. Avez-vous participé à des activités sur la nutrition et la santé au cours des derniers mois du projet (nom du projet) ? Si oui, à quelles activités avez-vous participé ? Sur la base de l’approche projet, recherchez des activités liées à la nutrition et à la santé telles que : i. Réunions des groupes de donneuses de soins ii. Démonstrations de pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs. iii. Visites à domicile iv. Identification des enfants malnutris v. Séances de conseil et d’information vi. Jardinage vii. L’élevage des animaux viii. Soins prénatals ix. Planification familiale x. CPS xi. Traitement des cas plus graves de malnutrition xii. Activités de l’approche SBC xiii. VSLA ou autres programmes de crédit 4. Pour chaque activité mentionnée à la question 3, posez les questions suivantes : i. Qui a dirigé l’activité ii. Quelles informations ont été partagées pendant les sessions iii. A-t-on utilisé du matériel éducatif pour transmettre des messages ? i. Dans l’affirmative, le matériel éducatif a-t-il amélioré ou diminué votre compréhension des messages ? iv. Les séances ont-elles été utiles ? v. En quoi les séances n’ont-elles pas /ont-elles été utiles ? vi. En quoi ces séances ont-elles changé vos pratiques ? vii. L’une ou l’autre des pratiques modifiées cause-t-elle des problèmes ou entraîne-t-elle des conséquences inattendues ? viii. Spécifique aux groupes de donneuses de soins : i. Quand avez-vous reçu une visite à domicile pour la dernière fois ? ii. De quoi a-t-on discuté ? iii. Etait-ce pertinent pour vous ? iv. Quand la mère leader vient chez vous, que fait-elle ? Existe-t-il autre chose que la diffusion de messages ? (C.-à-d. observation, conseils sur des questions précises, etc.) v. Pourquoi recevez-vous des visites à domicile ? Pendant combien de temps recevrez-vous des visites à domicile ? 5. Avez-vous participé à d’autres réunions ou séances dans la communauté (recherche d’approches utilisées dans l’approche SBC) où des informations sur la nutrition et la santé ont été fournies ? Dans l’affirmative, quelles informations ont-elles été transmises ? Avez-vous trouvé ces séances instructives ? Pourquoi ou pourquoi pas ? IMPEL | Implementer-Led Evaluation & Learning 110 Annex 6: Data Collection Instruments 6. Avez-vous récemment écouté des informations à la radio sur la nutrition et la santé ? Dans l’affirmative, quelles informations ont été fournies ? Quels sont les messages radiophoniques qui vous ont motivé à faire quelque chose de différent ? Pourquoi ou pourquoi pas ? Avez-vous discuté des messages radio avec d’autres membres de la famille ou de la communauté ? 7. Les pratiques alimentaires de votre enfant ont-elles changé depuis que vous avez commencé à participer aux activités du projet ? Dans l’affirmative, quels changements spécifiques ont-ils été apportés à l’alimentation ? Pourquoi avez-vous opéré ces changements ? Selon vous, en quoi ces changements ont-ils affecté la santé de votre enfant ? Posez des questions approfondies sur les changements relatifs à : i. L’allaitement maternel ii. La transformation et la préparation des aliments iii. L’alimentation complémentaire, y compris i. Les aliments donnés au jeune enfant ii. Le nombre de repas quotidiens iii. Le lieu où on se procure les aliments complémentaires 8. Existe-t-il des sites de soins communautaires dans votre région ? Si oui, avez-vous eu recours à un site de soins communautaires ? Si oui, dans quelles circonstances ? Quels types de soins ont-ils été fournis ? Avez-vous été satisfait des soins prodigués ? Pourquoi ou pourquoi pas ? Mères leaders 9. Depuis combien de temps êtes-vous une mère leader ? 10. Quel est votre travail en tant que mère leader ? a. Réunions mensuelles du groupe de donneurs de soins : i. Quand vous êtes-vous rencontrés pour la dernière fois en tant que groupe de donneurs de soins ? Et avant cela ? Depuis le début du projet, vous rencontrez￾vous habituellement ___ (fréquence indiquée ci-dessus) ii. A part les mères leaders qui participent ? Qui est votre chef ? iii. Que faites-vous lorsque vous vous rencontrez pour le groupe de soins ? b. Visites à domicile : i. Veuillez-m’en dire plus sur les visites à domicile. ii. À quelle fréquence faites-vous des visites à domicile ? iii. Quand avez-vous fait une visite à domicile pour la dernière fois ? iv. Que faites-vous lors de la visite à domicile ? v. Comment décidez-vous du sujet à discuter ? vi. Outre le partage de messages lors de la visite à domicile, faites-vous autre chose pendant cette période (observation, discussion sur d’autres sujets, etc.) ? vii. Depuis le début des visites à domicile, quels sont certains des changements les plus importants que vous avez observés dans les ménages du quartier ? Pour certains des autres messages que vous transmettez et que les mères ne sont pas capables de mettre en pratique, quelles sont certaines des raisons pour lesquelles vous pensez qu’elles sont plus difficiles à mettre en pratique ? c. Démonstrations culinaires/réunions portant sur le ralentissement de la croissance Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 111 i. Pouvez-vous me dire comment le conseil en matière de démonstration culinaire et de ralentissement de la croissance est mis en œuvre (réaliser) ? ii. En tant que mère leader, participez-vous à cette activité ? Comment ? iii. Y a-t-il des femmes de votre quartier qui participent ? Constatez-vous que leurs enfants vont mieux ? iv. Comment les mères sont-elles ciblées pour participer à l’activité ? v. Quel type de discussion a lieu si une mère a de la difficulté à reproduire la recette à la maison ? Quels sont les autres services de conseils ou autres dialogues qui sont offerts ? 11. Comment avez-vous été choisie pour devenir une mère leader ? 12. Connaissez-vous d’autres membres de votre communauté qui aimeraient être des mères leaders mais qui n’ont pas été choisies ? 13. Pourquoi voulez-vous être une mère leader ? d. Si vous ne voulez pas être une mère leader, l’avez-vous fait savoir au projet ? Pourquoi ou pourquoi pas ? e. Si oui, voulez-vous continuer à être une mère leader ? Si oui, pensez-vous que vous continuerez d’être une mère leader même si _____ n’était pas dans la communauté ? Comment votre emploi changerait-il si _____ n’était pas là ? 14. Pouvez-vous me montrer le matériel de communication que vous avez reçu du programme qui vous aide à être une mère leader ? Parler à travers le matériel 15. Quelles sont les autres activités auxquelles vous participez ? f. Microjardinage sur la propriété familiale/Permagarden. i. Recevez-vous de l’aide pour avoir un jardin ? ii. C’est chez toi ? Quelque part dans la communauté ? iii. Depuis combien de temps l’avez-vous ? iv. Qui s’occupe du jardin ? Si vous, est-ce que quelqu’un d’autre vous aide, votre fils/fille/mari ? v. Qu’avez-vous planté ? Y a-t-il des choses que vous plantez que vous ne saviez pas avant le projet ? vi. En avez-vous déjà eu la production ? vii. Qu’en avez-vous fait (vendre/manger) ? viii. Pendant combien de temps a-t-elle fourni de la nourriture à votre maison ? ix. Combien de fois avez-vous replanté sans l’aide du projet ? x. Êtes-vous contente d’avoir ce jardin ? Activités menées dans un établissement de santé Soins prénatals 16. Avez-vous reçu des soins prénatals(ANC) avant votre dernier accouchement ? Pourquoi ou pourquoi pas ? 17. À quelle étape de votre grossesse avez-vous reçu des soins prénatals pour la première fois ? Pourquoi avez-vous commencé les soins prénatals au moment précis où vous l’avez fait ? IMPEL | Implementer-Led Evaluation & Learning 112 Annex 6: Data Collection Instruments 18. Combien de séances de séances de soins prénatals avez-vous suivies pendant votre dernière grossesse ? 19. Comment avez-vous décidé de vous présenter aux soins prénatals ? Qui a participé au processus décisionnel ? Quel rôle votre mari ou votre partenaire a-t-il joué dans la décision de suivre des soins prénatals ? 20. Comment décririez-vous les soins prénatals fournis ? 21. Qu’avez-vous appris pendant les soins prénatals ? Quel est, selon vous, le but des soins prénatals ? 22. Avez-vous été satisfait de l’ensemble des services fournis pendant les soins prénatals ? Pourquoi ou pourquoi pas ? Qu’est-ce qui vous a particulièrement plu ? Qu’est-ce que vous n’aviez pas apprécié, le cas échéant ? 23. Quelles recommandations, le cas échéant, feriez-vous pour améliorer les soins prénatals fournis ? 24. Après votre accouchement, avez-vous reçu des informations sur la planification familiale ? Dans l’affirmative, quels renseignements avez-vous reçus ? 25. Avez-vous reçu d’autres informations sur la planification familiale ? Si oui, où ? 26. Votre mari participe-t-il à des activités liées aux relations conjugales et à la prise de décisions au sein du ménage ? Visites de puériculture/Soins postnatals 27. Avez-vous bien assisté aux visites chez la puéricultrice avec votre jeune bébé ? Pourquoi ou pourquoi pas ? 28. Si les visites chez la puéricultrice ont été bien suivies, a. À quelle fréquence assistez-vous à ces séances chez la puéricultrice ? b. Quand avez-vous commencé à emmener votre enfant en visite ? c. Est-ce que vous continuez d’emmener votre enfant en visite chez la puéricultrice ? Si vous avez arrêté, pourquoi avez-vous cessé d’y assister ? 29. Pourquoi avez-vous décidé d’emmener votre enfant en visite chez la puéricultrice ? Qui a participé au processus de prise de décision ? Quel rôle votre mari ou votre partenaire a-t-il joué dans la décision de se présenter à la CPS ? 30. Comment les services sont-ils fournis pendant la visite à la CPS ? 31. Qu’avez-vous appris pendant les séances ? À votre avis, quel est le but des visites chez la puéricultrice ? 32. Avez-vous été satisfait de l’ensemble des services fournis pendant la visite chez la puéricultrice ? Pourquoi ou pourquoi pas ? Qu’est-ce qui vous a particulièrement plu ? Qu’est-ce que vous n’avez pas apprécié, le cas échéant ? 33. Quelles recommandations, le cas échéant, feriez-vous pour améliorer les visites des bébés chez la puéricultrice ? Autres questions 34. Quand votre jeune enfant a-t-il eu pour la dernière fois de la fièvre, de la toux ou de la diarrhée ? Où avez-vous emmené votre enfant pour le faire soigner ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 113 35. Pourquoi avez-vous emmené votre enfant chez ce prestataire (réponses à la question 1) ? Quels ont été les principaux facteurs qui ont guidé votre décision ? Posez des questions approfondies sur les points suivants : i. Disponibilité des services et de l’équipement médical ii. Coût des services iii. Établissement/infrastructure de santé iv. Relation avec le prestataire de soins de santé v. Qualité des soins de santé 36. Qui a participé au processus décisionnel ? Comment s’est déroulée la prise de décision concernant la recherche d’un traitement ? Quel rôle votre mari ou votre partenaire a-t-il joué dans la décision de consulter un professionnel de la santé ? 37. Quelle a été votre évaluation globale du traitement fourni ? Avez-vous été satisfait du traitement fourni ? Pourquoi ou pourquoi pas ? Qu’est-ce qui vous a particulièrement plu ? Qu’est-ce que vous n’avez pas apprécié, le cas échéant ? 38. Le cas échéant, quelles recommandations feriez-vous pour améliorer les services de soins fournis ? 39. Comment votre mari participe-t-il à la prise de décision concernant les décisions familiales, y compris celles liées à la santé et à la garde des enfants dans votre ménage ? Comment, le cas échéant, son rôle a-t-il changé depuis le début du projet (nom du projet) ? Que signifie pour vous le terme « malnutri » ? Posez les questions suivantes : vi. Existe-t-il un mot local pour désigner la malnutrition ? vii. Quels sont les signes et symptômes de la malnutrition chez les jeunes enfants ? viii. Qu’est-ce qui cause la malnutrition chez un enfant ? ix. Comment traite-t-on la malnutrition ? x. Comment prévenir la malnutrition ? xi. Comment essayez-vous de prévenir la malnutrition ? IMPEL | Implementer-Led Evaluation & Learning 114 Annex 6: Data Collection Instruments Guide d'entretien approfondi pour les agents communautaires Date: Informant: Location: 1. Quand les activités relatives à la santé maternelle et à la nutrition et à la santé infantile ont-elles été introduites dans votre région ? En quoi consistent les activités ? : i. Care Group ii. Démonstrations culinaires iii. Démonstrations des pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs iv. Identification des enfants malnutris/dépistage PB v. Activités visant à encourager les pères à jouer un rôle de soutien dans la prise de décision du ménage en matière de nutrition et de santé 2. Quelles sont vos responsabilités concernant les activités de nutrition et de santé du projet Budikadidi ? Quels types de membres de la communauté participent régulièrement à ces activités ? i. Promoteurs de l’hygiène, Mama Leader, relais, CODESA, poser des questions approfondies sur les activités offertes au niveau communautaire, les participants de la communauté, y compris les caractéristiques et le nombre de personnes qui y participent, et les programmations continues liées aux éléments suivants : i. Visites à domicile ii. Séances de conseil et de groupe avec des groupes d’alimentation de l’enfant iii. Démonstrations culinaire de pratiques appropriées de transformation des aliments et de préparation d’aliments nutritifs iv. Détection des enfants malnutris/Depistage PB v. Autres activités de l’approche SBC 3. Quelle formation avez-vous reçue dans le cadre des activités de santé et de nutrition ? i. Quelle a été la durée de la formation et où a-t-elle eu lieu ? ii. Qui étaient les formateurs ? Pensez-vous qu’ils étaient qualifiés pour dispenser la formation ? Pourquoi ou pourquoi pas ? iii. Quelle a été la nature des thématiques abordées au cours de la formation ? Posez des questions approfondies sur le contenu de la formation en mettant l’accent sur les sujets suivants : i. ANJE ii. Visites à domicile iii. Séances de conseil et de groupe avec des groupes d’alimentation de l’enfant iv. Démonstrations de bonnes pratiques de transformation des aliments et de préparation d’aliments nutritifs v. Jardinage vi. Dépistage des enfants malnutris vii. Soins prénatals Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 115 viii. Espacement des grossesses ix. Consultation prescolaire x. Collecte des données, remplissage des formulaires mensuels et suivi des services xi. Approches axées sur le changement de comportement xii. Genre iv. Pensez-vous que la formation a été suffisamment longue et adéquate pour que vous puissiez mener à bien les activités de nutrition infantile comme prévu ? Dans la négative, quelles étaient certaines des lacunes de la formation ? Quels sujets manquaient et devraient être modifiés ou ajoutés à la formation v. Y a-t-il eu un suivi de la formation ou du recyclage ? Dans la négative, estimez-vous qu’une formation de suivi soit nécessaire ? vi. Êtes-vous satisfait de l’ensemble de la formation ? Pourquoi ou pourquoi pas ? 4. Quels matériels et fournitures avez-vous reçus en rapport avec les activités de nutrition et de santé depuis le début du projet Budikadidi? (Demander à l’informateur de partager la documentation). i. À quelle fréquence recevez-vous des fournitures, y compris du matériel didactique, PB, etc.? ii. Les matériaux et les fournitures sont-ils suffisants pour mener à bien les activités du projet ? Veuillez expliquer. iii. Vous arrive-t-il de manquer d’outils et de fournitures ? Si oui, à quelle fréquence ? Que fait-on fait lorsque cela arrive ? iv. Êtes-vous satisfait du matériel et des fournitures reçus ? Pourquoi ou pourquoi pas ? v. 5. Quel type de supervision recevez-vous dans le cadre des activités de nutrition infantile du projet (nom du projet) ? i. Qui supervise vos activités professionnelles ? ii. À quelle fréquence la supervision est-elle censée avoir lieu et à quelle fréquence a-t-elle réellement lieu ? iii. Quand les visites de supervision ont-elles lieu, combien de temps durent-elles ? Que se passe-t-il pendant les visites de supervision ? Dans quelle mesure interagissez-vous avec votre superviseur pendant ces visites ? iv. Êtes-vous satisfait de la supervision ? Pourquoi ou pourquoi pas ? v. Quel autre soutien, au-delà de la supervision, le projet vous apporte-t-il ? Posez des questions approfondies sur le mentorat, le coaching, etc. Et s’ils ont reçu des outils en cours d’emploi pour les aider dans leur travail.] 6. Comment assurez-vous le suivi des activités en cours ? Posez des questions approfondies pour en savoir davantage sur : i. Les formulaires à remplir ii. La fréquence et le mode de transmission des formulaires aux bureaux des superviseurs/du partenaire d’exécution IMPEL | Implementer-Led Evaluation & Learning 116 Annex 6: Data Collection Instruments 7. Nous savons que le genre est inclus dans le projet Budikadi. Pouvez-vous nous dire quelles approches vous utilisez en matière de genre et comment elles sont intégrées dans les activités de santé et de nutrition ? Comment les activités liées au genre sont-elles intégrées aux autres objectifs, tels que les moyens de subsistance, la gouvernance et le WASH ? En quoi le ciblage assure-t-il ou n’assure-t-il pas l’intégration entre les objectifs ? Que pensez-vous de ces approches ? Comment ces interventions se mesurent-elles et quelles sont vos évaluations de l’efficacité de ces activités ? Comment ont-elles été accueillies par les membres de la communauté ? Pouvez-vous nous faire part de quelques réussites et des difficultés rencontrées dans la mise en œuvre de ces activités ? A votre avis, comment les approches genre peuvent￾elles être améliorées ? 8. Nous savons que la jeunesse est incluse dans le projet Budikadidi. Pouvez-vous nous dire quelles sont les approches que vous utilisez à l’égard des jeunes et comment elles sont intégrées dans les activités de santé et de nutrition ? Comment les activités liées au genre sont-elles intégrées aux autres objectifs, tels que les moyens de subsistance, la gouvernance et le WASH ? En quoi le ciblage assure-t-il ou n’assure-t-il pas l’intégration entre les objectifs ? Que pensez￾vous de ces approches ? Comment ces interventions se mesurent-elles et quelles sont vos évaluations de l’efficacité de ces activités ? Comment ont-elles été accueillies par les membres de la communauté? Pouvez-vous nous faire part de quelques réussites et des difficultés rencontrées dans la mise en œuvre de ces activités ? A votre avis, comment les approches genre peuvent-elles être améliorées? 9. Quels sont les facteurs qui vous motivent à mener des activités liées à la nutrition et à la santé dans le cadre du projet ? (Posez des questions approfondies sur les indemnités journalières, l’avancement professionnel, les conditions de travail, la reconnaissance communautaire et les autres avantages non financiers). Que pensez-vous de la façon dont on vous motive à mener des activités ? Avez-vous des documents d’identification officiels qui indiquent que vous travaillez sur le projet de (nom du projet) ? Dans l’ensemble, êtes-vous satisfait de votre rôle ? Pourquoi ou pourquoi pas ? 10. Avec qui collaborez-vous dans le cadre des activités en cours dans le domaine de la nutrition et de la santé ? Comment fonctionne la coordination des activités sur le terrain ? Quels systèmes, le cas échéant, ont été mis en place pour partager les enseignements tirés ou les difficultés rencontrées sur le terrain ? En cas de difficultés, comment les surmonter ? Veuillez décrire toutes les réunions de coordination qui ont lieu actuellement. 11. Dans quelle mesure êtes-vous en mesure de mettre en œuvre les services de nutrition et de santé conçus dans le cadre de l’approche du projet ? Quels sont les défis à relever ? Y a-t-il des aspects de l’approche que vous n’êtes pas en mesure de mettre en œuvre ? Si oui, pourquoi ? Comment les surmonter ? 12. Par rapport aux stratégies précédentes auxquelles vous avez participé, comment l’approche a-t￾elle changé la façon dont les mères et les autres donneurs de soins s’occupent de leurs enfants ? Selon vous, l’approche donne-t-elle aux mères, aux autres donneurs de soins et aux membres de la communauté les moyens de s’occuper de leurs enfants ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 117 13. Quelle est votre évaluation globale des activités de nutrition et de santé dans le cadre de l’approche du projet Budikadidi ? Êtes-vous satisfait des activités ? Pourquoi ou pourquoi pas ? Les gens de votre communauté sont-ils satisfaits des activités ? 14. Selon vous, quelle est la tendance générale de la malnutrition infantile de votre zone de couverture ? Est-elle à la hausse ou à la baisse ou n’y a-t-il aucun changement ? Qu’est-ce qui explique cette tendance ? 15. Que recommanderiez-vous pour améliorer la nutrition et la santé de l’enfant dans le cadre des activités de Budikadidi ? IMPEL | Implementer-Led Evaluation & Learning 118 Annex 6: Data Collection Instruments Focus group discussion (Group l’homme et femme a pars) 1. Les participants sont dans quelles group primaire et group secondaire de Budikadidi? 2. Liste des activites WASH dans votre village 3. Parmi les activites WASH que vous avez parle, quelle est difficile de faire, et quelle est facile? 4. Qui est en charge de WASH au niveau de Budikadid dans votre village et combiens de temps il vient pour faire le suivi dans le village 5. Comment votre village recu d’information WASH comme les messages? 6. Comment votre village recu de nouveau information WASH? 7. Est ce que vous avez un radio? Si oui, qui est en charge de radio dans votre famille? 8. Quelle message WASH dans la radio votre menages ecoute? Qu’est que le nom de cette emission? 9. Liste tous les point d’eaux dans votre village et at chaque, combine le distance aller-retour? 10. Dissez nous les membres de comite WASH dans votre village? Qui est homme et qui et femme? 11. Est ce que vous avez l’opportunity de participer dans la reunion assemblage de comite gestion d’eau? Si oui, quelle information vous avez recu dans ce reunion? 12. Combine de fois est ce reunion de comite gestion d’eau est qui participe normalement? 13. Vous connaissez combiens d’argent maintenant c’est dans la caisse de Comite gestion d’eau et il utilise l’argent pour quelle raison? 14. Est ce que les menages payer pour l’eau, si oui combien? 15. Est ce que ce village est une village de village assaini? Si oui, qu’est que les chose que le village a fait d’etre un village assaini? 16. Ou est ce que les enfants 0 a 2 ans fe Kaka? Comment mama s’occupe ave cette enfant? 17. Ou est ce que les enfants 2 a 5 ans fe kaka? Comment mama s’occupe avec cette enfant? 18. Quel type de latrine est dans la communautaire en majorite (traditionelle, amelioree avec dalle) et pourquoi? 19. Combine est le cout de latrine traditionelle et combine et le cout de latrine amelioree de dalle? 20. Qu’est que la difficulte d’avoir un latrine traditionelle? 21. Qu’est que la difficulte d’avoir un latrine amelioree avec dalle? 22. Est ce que vous connaissez le mason qui va construire ton latrine amelioree avec dalle? Qu’est que son nom? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 119 Mason In-depth Interview 1. Que lest votre role dans la communaute et Budikadidi in addition d’etre un mason? 2. Le formation qui’il a recu et qui a donnez cette information? (besoin de nom de cette personne) a. List le different sujet dans leur formations b. Est que vous avez recu un partie de votre formation sur comment fais le petit business? Si oui, expliquez 3. Que changeriez-vous dans la formation et pourquoi? 4. Qu'aimeriez-vous voir ajouté à la formation dans le futur? 5. Est ce-que vous pouvez montre le fiche de module ou bien fiche de suivi qu’il utilise apres la formation? 6. Est ce que vous connaissez que le village a ete certifie comme un village avec le latrine hygenique? Si oui, qui a certfie? 7. Êtes-vous en mesure d'accéder aux fournitures nécessaires pour construire des latrines conformément à ce que vous avez appris lors de la formation? sinon pourquoi? 8. Avez-vous eu des difficultés à appliquer ce que vous avez appris pendant la formation? Si oui, pourquoi 9. Combien des menages qui a deja construit leurs latrine en dalle? Est ce-que vous avez faits pour eux? Si oui, comment il connaissent de venir a toi pour le construit? 10. Qu’est que vous pensez c’est le defie dans le village pour le menage pour avoir le latrine en dalle? Et pourquoi les menages on ce defie? IMPEL | Implementer-Led Evaluation & Learning 120 Annex 6: Data Collection Instruments Water Technician (Reparateur d’eau) 1. Que lest votre role dans la communaute et Budikadidi in addition d’etre un mason? 2. Le formation qui’il a recu et qui a donnez cette information? (besoin de nom de cette personne) a. List le different sujet dans leur formations b. Est que vous avez recu un partie de votre formation sur comment fais le petit business? Si oui, expliquez 3. Que changeriez-vous dans la formation et pourquoi? 4. Qu'aimeriez-vous voir ajouté à la formation dans le futur? 5. Est ce-que vous pouvez montre le fiche de module ou bien fiche de suivi qu’il utilise apres la formation? 6. Quels endroits êtes-vous responsable de la réparation de l'eau? 7. Quels types de réparations d'eau avez-vous effectués jusqu'à présent et à quelle fréquence avez￾vous effectué des réparations? 8. Quelles sont les pannes courantes dans la communauté et combien de temps faut-il pour les réparer? 9. Si la réparation prend beaucoup de temps, pourquoi? 10. Êtes-vous payé à temps? Sinon pourquoi? 11. Quelle formation avez-vous reçue sur les tests de qualité de l'eau? Vous sentez-vous à l'aise avec l'outil d'analyse de la qualité de l'eau? Quel est le nom de l'outil d'analyse de la qualité de l'eau que vous utilisez et que testez-vous? 12. Où allez-vous trouver des pièces de rechange pour réparer l’eau? À quelle distance se trouve l'endroit où vous devez réparer le point d'eau? 13. Quelles sont les difficultés pour obtenir des pièces de rechange à temps pour les réparations? 14. Que comptez-vous faire et avec qui allez-vous continuer après la fin du projet? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 121 In Depth Interview with the Zonal WASH Supervisor 1. Comment le gouvernement s'est-il engagé dans les plans d'action communautaires, en particulier pour les plans d'action concernant l'eau,et l'assainissement? 2. Comment vous pensez que Budikadidi a fait un bonne coordination sur WASH avec la zone de sante? 3. À quels types d’ateliers de leçons tirées de Budikadidi avez-vous été impliqué et dans quelle mesure cet atelier at-il été utile? 4. Le gouvernement a-t-il adopté les meilleures pratiques ou approches du projet Budikadidi et pourquoi? Si oui, quelles meilleures pratiques ou approches? 5. Quels ont été selon vous certains des défis WASH du projet Budikadidid et quels ont été leurs succès? Quelles pourraient être les suggestions du gouvernement pour améliorer ces défis? 6. Dans quelle mesure le gouvernement participe-t-il à la certification ODF? Quel est le rôle du gouvernement? Prochaines étapes de la communauté pour gravir les échelons de l'assainissement (quels sont leurs plans pour adopter une latrine améliorée et quelles informations ont-elles reçues pour passer aux étapes suivantes) 7. Dans quelle mesure CRS doit-il impliquer le gouvernement local tout au long du processus de projet et dans les approches de durabilité? 8. Quel est le niveau de participation des gouvernements aux comités de l’eau dans l’établissement des règlements et la mise en place de mécanismes de rémunération des services? Quel est le processus d'approbation de ceci? 9. Quel est votre rôle dans l’analyse et le traitement de la qualité de l’eau? 10. Quelles formations le projet a-t-il fournies à votre équipe pour le traitement de la qualité de l'eau? 11. Sur quel type de système de traitement de la qualité de l'eau avez-vous été formé? Où pourriez￾vous aller si les consommables du kit de test sont épuisés? Qui va payer pour cela maintenant et qui va payer pour cela une fois le projet terminé? 12. Quel est le plan du gouvernement en matière de gestion de l'approvisionnement en eau et de traitement de la qualité de l'eau une fois le projet terminé? IMPEL | Implementer-Led Evaluation & Learning 122 Annex 6: Data Collection Instruments In Depth Interview: SNHR 1. Comment le gouvernement s'est-il engage dans les plans d'action communautaires, en particulier pour les plans d'action concernant l'eau? 2. Expliquez-vous les types de coordination vous avez avec Budikadidi? 3. À quels types d’ateliers de leçons tirées de Budikadidi avez-vous été impliqué et dans quelle mesure cet atelier at-il été utile? 4. Le gouvernement a-t-il adopté les meilleures pratiques ou approches du projet Budikadidi et pourquoi? Si oui, quelles meilleures pratiques ou approaches les government pensent qu’il va adopter? 5. Quels ont été selon vous certains des défis WASH du projet Budikadidid et quels ont été leurs succès? Quelles pourraient être les suggestions du gouvernement pour améliorer ces défis et des bonnes coordination durable? 6. Quel est le plan du gouvernement en matière de gestion de l'approvisionnement en eau et de traitement de la qualité de l'eau quand le projet Budikadidi terminé? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 123 Les guide pour demander les question WASH Promoter 1. Focal Point WASH ✔ Son Role dans le projet ✔ Les formations qui’il a recu pour lui aider sur les projet? ✔ Expliquez son structure de coordination/planification avec le reste de promoteur et les defies et success de cette structure ✔ Les niveau de soutien et de supervision il receive de son superviseur ✔ Expliquez le defie/Success sur le projet qui concerne L’eau et Assainissement ✔ Expliquez dans son avis les choses qui arriver durable sur WASH a ce periode la et le chose qui va continuer et besoin quelque temps pour la communite de prise en charge. Que’est que les choses qui ne seront pas fait de prise en charge a la fin du projet? 2. Agent WASH ✔ Son Rle dans le projet ✔ Les formation qui’l a recu pour lui aider sur le projet? ✔ Les niveau de soutien et supervision ils receive de son superviseurs ✔ Expliquez le defie/Success sur le projet qui concerne L’eau et Assainissement ✔ Expliquez dans son avis les choses qui arriver durable sur WASH a ce periode la et le chose qui va continuer et besoin quelque temps pour la communite de prise en charge. Que’est que les choses qui ne seront pas fait de prise en charge a la fin du projet? IMPEL | Implementer-Led Evaluation & Learning 124 Annex 6: Data Collection Instruments Water Committee 1. -Quel est votre role dans le projet Budikadidi? 2. -Est ce que vous avez des autres roles dans le projet Budikadidi 3. -Qu’est que le type de formation vous avez recu? 4. S'il vous plaît, montrez-moi des exemples de vos documents constitutifs (livre de comptes, règlements, procès-verbaux de réunions, liste de présence (recherchez la présence des membres de la communauté ainsi que des leaders naturels), rôles et responsabilités, etc.) 5. Pouvez-vous m'expliquer votre business plan pour assurer la quantite et qualite de l’eau dans votre communaute? Est-ce que le projet a aider vous de realiser votre business plan? 6. Combien de points d'eau y a-t-il dans votre communauté? Quel genre? Quelle est la distance moyenne de ces points d’eau à la communauté? 7. Comment tenez-vous la communauté au courant des finances existantes du comité, du calendrier des réparations et de l'entretien, des modifications du coût des services? 8. À quelle fréquence les redevances d’eau sont-elles perçues et qui les perçoit? 9. Qui approuve les paiements reçus et à quelle fréquence les examine-t-on et qui les examine? 10. Est-ce que tout le monde respecte le calendrier de paiement et le montant du paiement? Si non, pourquoi est-ce? 11. Quel est le temps moyen nécessaire pour réparer les robinets, les pompes cassées, etc.? Si cela prend plus d'une journée, quelle est la raison des non-réparations après une journée? 12. Où vont les ménages lorsque le point d'eau ne fonctionne pas? À quelle distance se trouve le point d'eau le plus proche et de quel type de point d'eau s'agit-il? Est-ce que ce point d'eau est potable? 13. Quelles sont d'autres sources dangereuses (vérification de l'utilisation des ravins) utilisées par la communauté? Si la communauté utilise des sources non salubres, quelles mesures ont été prises par les autorités locales, les CCC et le projet pour réduire les risques pour la santé et la sécurité liés à l'utilisation des sources d'eau libre? 14. Quelle est la fréquence de test du forage et quelle est sa fréquence de traitement par chloration? Qui fait la chloration de routine et est-ce conforme? Si ce n'est pas cohérent, pourquoi? 15. Quelles sont les pratiques courantes de traitement de l'eau que HH fait? 16. Qui leur a appris à traiter leur eau et à quelle fréquence traitent-ils leur eau? Si elles ne traitent pas leur eau, pourquoi est-ce? 17. Quelle a été la contribution de la communauté lors de la construction ou de la réhabilitation de l’eau? 18. Quel rôle les femmes jouent-elles dans les comités de l'eau et combien sont-elles dans le groupe 19. À quelle fréquence prennent-ils l'initiative de faciliter les réunions? Et s’ils ne le font pas, pourquoi cela? 20. Veuillez expliquer comment les administrations locales et les CAC participent-ils à la gestion et à la maintenance des systèmes d'approvisionnement en eau? quel rôle jouent-ils? Comment leur rôle a￾t-il été bénéfique et comment ne l'a-t-il pas benefice? 21. Quels problèmes ont été abordés et qui les a résolus (gouvernement local, CAC, leaders naturels) et comment il a resoulu? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 125 DI--Leader Naturel et Mama Lumiere (Faire Entretien A pars) 1. Quels sont les groupes dont vous êtes membre? 2. Dans quel groupe dirigez-vous et quel est votre rôle? 3. Combien de fois que tu fais le suivi au menages? 4. Quels sont les messages WASH clés dans votre fiche de module? Est que je peux me montre et explique le partie WASH? 5. Sur les messages de WASH qui concerne les pratiques. Quel practique WASH c’est facile et quell pratique et dificile pour les menages? 6. Quels sont les messages WASH qui sont plus faciles à pratiquer pour les ménages et quel est le moins à pratiquer et pourquoi? 7. Quel est le problème avec l'utilisation et la fonctionnalité des stations de lavage des mains? 8. Comment la communauté a-t-elle collaboré avec Budikadidi pour améliorer la technologie et l'utilisation du lavage des mains? 9. Quelles ont été vos observations concernant les mères enceintes et allaitantes qui ont adopté les pratiques WASH, especiallement pour les enfants 0 a 2 ans et 2 a 5 ans? 10. Que pouvez-vous expliquer sur le changement de comportement dans votre village? Quel est le defi et quel est le succes? IMPEL | Implementer-Led Evaluation & Learning 126 Annex 6: Data Collection Instruments Structured Observations of Household Latrines and Hand Washing Stations Work with the village chief or other knowledgeable person to identify which latrines were constructed by the family due to participation in (the project), and when each was constructed. Visit each latrine and complete the following observations. Travaillez avec le chef de village ou une autre personne bien informée pour identifier les latrines construites par la famille en raison de la participation à (le projet) et à quel moment chacune d'elles a été construite. Visitez chaque latrine et complétez les observations suivantes. Qualitative Questions Latrine and Hand Washing Station Latrine Name:________ 1. Was this latrine constructed by the HH during the project?/Latrine construit pendand le projet? Yes / No / Don’t Know 2. Take a photo of the latrine/ Prends un photo Yes / No / Don’t Know 3. Is latrine constructed on firm ground?/ La terre de latrine est forte? Yes / No / Don’t Know 4. Please ask or check if toiled it lined/ verifier la toilet fosse est renforce Yes / No / Don’t Know 5. Is the toilet 30 meters away from water source?/ la toilette est 30 metre de source d’eau? Yes / No / Don’t Know 6. HH received CLTS Triggering? HH a recu le CLTS Declanchement? Yes /No/Don’t Know 7. HH hired mason to construct latrine? Menage utilize un mason pour construit latrine? Yes/No Other:_______________ 8. Cost of Latrine (Coute de latrine) 1. Pit digging ____ 2. Lining______ 3. Slab 4. Superstructure_______ 5. Construction/Installation ____ 9. Latrine Model (modele latrine) 1 st Generation 2nd Generation Other:_________________ Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 127 10. Type of Latrine Lining used? Renforcement de latrine fosse? 11. When was this latrine constructed? ______ Date:___ Quand est que le latrine est contruit? Date:___ Year: _______ Don’t Know____ 12. Type of latrines: Type de latrine: a) VIP b) Pour flush c) Traditional, washable slab d) Traditional, non￾washable slab e) Other (describe): Water sealed 13. Type of Superstructure a) Bricks (burnt ___ Simple___) b) Traditional material c) zinc structure 13.1. Latrine is open for use (unlocked): Latrine pas ouvert pour utilisation? Yes / No 13.2. Is there clear evidence latrine is being used (note odor, contents of pit, observed use): Es latrine utilise? Yes / No Notes: _______________________ ________________________ _____ 13.3. Does latrine offer full privacy (walls and doors that can fully close):_ Latrine est prive pour la femme ( la porte est le mur) Yes / No 13.4. Door locks from inside? La porte peut fermer en dedans? Yes / No 13.5. Safe construction (secure slab, lack of crumbling walls or roof): Construction est secure (presence de dale, le mu rest forte, et il une toiture Yes / No Notes: _______________________ ________________________ ____ IMPEL | Implementer-Led Evaluation & Learning 128 Annex 6: Data Collection Instruments 13.6. Easily usable for individuals with physical disabilities (e.g. no stairs, rails or devices inside for support, seat): Est facile pour utilse pour la person handicape (presence de escalier, rail, est d’autre soutien) Yes / No 13.7. Individual with disability in the home? Person handicape dans la menage Yes/No 13.8. Where do small children (3-5 years) go to defecate and why? Ou est que le petite enfant (3-5 ans) fe kaka? Explain 13.9. Cleanliness: Acceptable cleanliness (not soiled with urine, feces, or used paper): Propete: pas de sol, urine, matiere fecales, etc) Yes / No 13.10. Latrine surrounding free of stagnant water; pas de l’eau stagnant Yes/No 13.11. Odor: Acceptable odor (no smell or mild, tolerable smell): Pas d’odeur intolerable Yes / No Notes: _______________________ ________________________ ____ 13.12. Flies: Are fewer than 3 flies present: moins de 3 mouches Yes / No 14. Are materials for anal cleansing (paper or water container) available in or near any stalls? Le materiaux pour netoyyer soi-meme est disponsible pres de latrine Yes / No 15. Presence of cleaning materials for latrine Presence de materiaux pour nettoyer le latrine? 16. Notes: Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 129 1. What kind of container is designated for handwashing? Quel type de recipient est pour lavez de mains? 2. Take photo of handwashing stations/Prends de photo 3. Where are the container located? (circle all that apply) Ou est que le recipient? 1.1. Near latrine 1.2. Near the house 1.2. Other (describe): __________ 4. Is there water for handwashing today in any of the containers? (if not, why) Is qu’il ya de l’eau dans tous les recipient? Yes / No 5. Is there soap for handwashing today at any of the containers? Est ce qu’il ya de savant pour lavez de mains? Yes / No 6. Locate where the soap is kept? Ou est qu’il guarde le savon? 7. Ask the name of the soap, type and where it was purchase? Qu’est ce que le nom de savon et ou est vous avez acheter le savon? Name:___________ Type:_______ Purchased where:_____________ 8. List all what the soap is used for without mentioning hand washing (list as spoken) Le savon est utilise pour qu’elle sort d’activite dans le menage? 9. Is there evidence that handwashing is happening today (e.g. ground or soap is wet)? Verifie si le los pres de recipient est humide Yes / No 10. Did you observe anyone washing their hands today? If yes, note gender/age Observation: vous voyez quelqu’un qui lavez main? Qui? (homme, enfants, femme, la jeunnesse) Yes / No Who? Notes: ________________________ 11. Did you observe anyone using the latrine and not washing their hands today? If yes, note gender/age Observation: vous voyez quelqu’un qui sort de latrine et ne lavez pas le main? Qui? (homme, enfants, femme, la jeunnesse) Yes / No Who? Notes: ________________________ IMPEL | Implementer-Led Evaluation & Learning 130 Annex 6: Data Collection Instruments Environmental Check Observation: Walk through the village and observe any issues with sanitation. This can be done while visiting households. Promenez-vous dans le village et observez tous les problèmes d'assainissement. Cela peut être fait en visitant les ménages. ✔ Community Free of debris Communauté sans debris matières fécales Yes_ No:__ Notes: ✔ No presence of fecal matter Pas de présence de matières fécales Yes_ No:__ ✔ Animals are separated from humans Les animaux sont séparés de l'etre humain Yes_ No:__ ✔ Personal hygiene of children under 5 is normal Hygiène personnelle des enfants de moins de 5 ans est normal Yes_ No:__ ✔ Presence of dish rack at HH Présence de grille à vaisselle chez HH Yes_ No:__ ✔ Children playing in the dirt Enfants jouant dans la terre Yes_ No:__ ✔ No presence of standing water Pas de présence d'eau stagnante Yes_ No:__ ✔ Household yard is well swept and free of debris and feces La cour de la maison est bien balayée et exempte de débris et de matières fécales Yes_ No:__ Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 131 Observations at Water Points Water Point Name and Location: ___________ Water point source: Borehole__________ Spring Catchment____________ 1. À quelle distance se trouve le point d'eau de votre maison? 2. Combien de temps faut-il pour arriver au point d'eau? 3. Combien de personnes attendent au point d'eau? 4. Quel est le débit de l'eau pendant la saison sèche? Fort ___ Lent____ Faible____ Pas de débit____ 5. S'il vous plaît expliquer ce qui se passe lorsque le point d'eau ne fonctionne pas? 6. Quelle source d’eau utilisez-vous lorsque votre point d’eau ne fonctionne pas? A quelle distance est-ce? 7. Qui répare le point d'eau et combien de temps leur faut-il pour le réparer? 8. Combien payez-vous pour l'eau? _ _________ À quelle fréquence? _________ 9. Pouvez-vous expliquer à quoi servent vos frais d’utilisation de l’eau? _________ 10. Savez-vous quel est le solde des fonds pour le comité de l'eau? Si oui, combien? _____________________ 11. Leur ont-ils expliqué quel est le rôle des comités de l'eau? 12. Qui sont les principaux dirigeants des comités de l'eau? _____________ les ont-ils rencontrés? __________ 13. Sont-ils au courant des réunions du comité de l'assemblée générale? Si oui, qui dans leur ménage participe? 14. Quel est le goût de l'eau? 15. Traitez-vous votre eau? Si oui, avec quoi traitez-vous votre eau? Où allez-vous acheter un traitement pour votre eau? 16. À quelle fréquence nettoyez-vous vos jerrycans? 17. Observation: Vérifiez si les jerricans sont nettoyés avant de les remplir? 18. S'il vous plaît expliquer ce que vous utilisez pour filtrer votre eau? 19. Demandez à quelques bénéficiaires au puits: Combien de temps faut-il pour aller chercher, faire la queue et rentrer chez eux? Combien de temps dure leur temps d'attente moyen? 20. Combien y a-t-il de conteneurs dans la file d'attente à remplir? (Identifiez le type de conteneur pour déterminer le volume) 21. Décrivez qui est rassemblé au point d’eau (par exemple, sexe, âge) 22. Y a-t-il des lavabos? 23. Si oui: existe-t-il des preuves de leur utilisation? 24. Y a-t-il des abreuvoirs? 1. Si oui: existe-t-il des preuves de leur utilisation? Et en bon état? 25. Le point d'eau distribue-t-il actuellement de l'eau? Oui Non 26. Si la pompe à main: Notez le nombre de coups qu'il faut pour que l'eau coule initialement: ___ 27. Remplissez un récipient de 20 litres et utilisez un chronomètre pour mesurer le temps nécessaire pour remplir le récipient avec de l'eau. S'il s'agit d'une pompe à main, comptez également le nombre de coups nécessaires pour la remplir. 2. 27.1. Nombre de secondes pour remplir 20 litres: ______ 3. 27.2. Nombre de coups à remplir pour 20 litres: ______ 28. Remarquez s'il y a une clôture entourant le point d'eau pour empêcher les animaux d'y accéder. Expliquez la qualité de l'escrime? 29. Remarquez s'il y a un abreuvoir pour animaux à proximité, ainsi que l'état et la qualité de l'abreuvoir. S'il vous plaît prendre une photo de l'auge et de la propreté autour de l'auge. 30. Notez s’il ya de l’eau stagnante à proximité, l’état du puisard (s’il existe), un système de drainage construit ou un petit jardin receveur pour les eaux de drainage. IMPEL | Implementer-Led Evaluation & Learning 132 Annex 6: Data Collection Instruments 31. Notez la gravité de toute fuite d'eau apparente 32. Notez tout besoin de réparation ou d’entretien apparent. Dans l'affirmative, depuis combien de temps le point d'eau n'a-t-il pas besoin de réparations apparentes ou d'entretien et pourquoi n'a-t-il pas encore été réparé? 33. Commenter les défis ou menaces globaux Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 133 Structured Observations at Spring Source Spring source Location: ____________ Water point Location: ______________ Location to the spring source is accessible for maintenance (cleared vegetation)/Location de source Yes/N O Who maintains the spring source and how often is it maintained?/ Qui maintain le source et comment? When was the last time the spring source was maintained? Quel est ce que le dernier fois le source etait maintaine? Who is responsible for the spring source and what is the process if the spring source stops functioning because of spare parts broken? Qui est responsible pour le source et quel est le procesus si le source ne function pas? How often does the spring dry up? Ques que le frequence et quand le source n’est pas d’eau? What is the community plans once the spring is dried and does not provide water? Quest que le plan quand le source ne function pas et ne pas provincionner de l’eau? Clear developed drainage area to avoid surface water infiltration/ la zone de drainage développée pour éviter l'infiltration d'eau de surface Yes/N O Reinforced concrete spring collection box/ Boite de collecte de source existe et reinforce. Yes/N O Spring Eye is covered and protected Spring Eye est couvert et protege Yes/N O Outlet, Drainage & Over flow system including all supply of Pipes, fittings & Control Valves Système de sortie, de drainage et de trop-plein, y compris toutes les fournitures de tuyaux, raccords et vannes de régulation Yes/N o IMPEL | Implementer-Led Evaluation & Learning 134 Annex 6: Data Collection Instruments Observe if the spring is free of any debris Le source est libre de tout debris Yes/N o Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 135 Structured Observations at Borehole Borehole source Location: ____________ Borehole Location: ______________ 1. Is there a slab/concrete floor around the pump? YES Issue: Placing a concrete, impermeable surface around a water point can help prevent excess water soaking into the ground during pumping, as well as prevent pollution of groundwater supply. Additionally, placing a slab can help limit the development of puddles or muddy conditions around the borehole, which can attract mosquitoes, flies, or other animals and potentially transmit disease. The slab/concrete floor should be at least 1 meter wide. Le fait de placer une surface imperméable en béton autour d’un point d’eau peut aider à prévenir l’immersion excessive d’eau dans le sol lors du pompage, ainsi que la pollution des nappes phréatiques. De plus, placer une dalle peut aider à limiter le développement de flaques d'eau ou de conditions boueuses autour du trou de forage, ce qui peut attirer les moustiques, les mouches ou d'autres animaux et éventuellement transmettre des maladies. Le sol en dalle / béton doit avoir au moins 1 mètre de large. NO 2. Is the borehole area fenced? YES Issue: Fencing can help protect boreholes from contamination by animal waste, limit un-metered water use, and prevent unnecessary tampering with the pump. Les clôtures peuvent aider à protéger les forages de la contamination par les déjections animales, à limiter l'utilisation d'eau non mesurée et à empêcher toute manipulation inutile de la pompe. NO 3. Is there a drainage/runoff channel below the pump outlet? YES Issue: Runoff channels also helps prevent muddy conditions developing around the borehole, and can also allow excess water to be used to fill livestock troughs for animal watering downstream of the pump. Les canaux d'écoulement aident également à prévenir les conditions boueuses autour du forage et peuvent également permettre à un excès d'eau d'être utilisé pour remplir les mangeoires pour le bétail afin d'abreuver les animaux en aval de la pompe. NO Other issues to consider/look for IMPEL | Implementer-Led Evaluation & Learning 136 Annex 6: Data Collection Instruments ● Is there a constant water supply during dry seasons? Have hydrological studies been conducted to determine depth and recharge rate of groundwater table? Successful development of water harvesting schemes should take advantage of existing, perennial or seasonal water supplies. Y a-t-il un approvisionnement en eau constant pendant les saisons sèches? Des études hydrologiques ont-elles été menées pour déterminer la profondeur et le taux de recharge de la nappe phréatique? ● Is the borehole located at least 30m from any potential contaminants (latrine, rubbish pit, animal enclosures)? Le forage est-il situé à au moins 30 m de tout contaminant potentiel (latrine, fosse à ordures, enclos pour animaux)? ● Is any water pooling around the hand pump? Pooling of water may be an indication that the joints and the hand-pump need to be repaired. Some parts of the pump may be broken or some screw nuts may need to be tightened or replaced. Y a-t-il de l'eau qui s'accumule autour de la pompe à main? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 137 Agriculture IDI avec les Agents de Terrain 1. Quand les activités agriculture et élevage ont-elles été introduites dans votre communauté ? Qu'est-ce que les activités impliquent ? Sur la base de l’examen de la documentation et de l’information IP, recherchez des informations sur i. Jardinage (permagarden et market garden) ii. Élevage iii. Activités SBC liées à l'agriculture iv. Création de SILC 2. Quelles sont vos responsabilités vis-à-vis des activités de (nom du projet) agriculture et élevage ? Quels types de membres de la communauté participent à ces activités de manière continue ? 3. Comment avez-vous été sélectionné / recruté pour ce poste ? Rechercher des connaissances préalables sur le sujet 4. Quelle formation avez-vous reçue du projet dans le cadre des activités Agriculture et moyens de subsistance ? i. Combien de temps a duré la formation et où a eu lieu la formation ? ii. Qui étaient les formateurs ? Pensez-vous qu'ils étaient qualifiés pour effectuer la formation ? Pourquoi ou pourquoi pas ? iii. Quels sujets ont été abordés pendant la formation ? Sondez pour le contenu de la formation liée à (énumérer les approches différentes) iv. Avez-vous pensé que la formation était suffisamment longue et adéquate pour que vous puissiez mener les activités Agriculture et Moyens de subsistance comme prévu ? Si non, quelles étaient certaines des lacunes de la formation ? Quels sujets manquaient et devraient être modifiés ou ajoutés à la formation ? v. Avez-vous participé à une formation de suivi / recyclage ? Si non, estimez-vous qu'une formation de suivi est nécessaire ? vi. Êtes-vous satisfait de la formation générale ? Pourquoi ou pourquoi pas ? 5. Quels matériaux et fournitures avez-vous reçu en rapport avec les activités Agriculture et Moyens de subsistance depuis le début du projet (nom du projet) ? a. À quelle fréquence recevez-vous des fournitures, notamment du matériel pédagogique, des semences et des outils pour le jardinage ? b. Le matériel et les fournitures sont-ils suffisants pour mener à bien les activités du projet ? S'il vous plaît, expliquez. c. Avez-vous déjà rencontré des pénuries d’outils et de fournitures ? Si oui, à quelle fréquence cela se produit-il? Que faites-vous quand cela se produit ? d. Êtes-vous satisfait du matériel et des fournitures reçus ? Pourquoi ou pourquoi pas ? 6. Quel type de supervision recevez-vous dans le cadre des activités de (nom du projet) Agriculture et moyens de subsistance ? i. Qui supervise vos activités de travail ? ii. À quelle fréquence la supervision est-elle censée avoir lieu et à quelle fréquence a-t￾elle réellement lieu ? iii. Quand les visites de supervision ont lieu, combien de temps durent-elles ? Que se passe-t-il lors des visites de supervision ? Dans quelle mesure interagissez-vous avec votre superviseur lors de ces visites ? iv. Êtes-vous satisfait de la supervision ? Pourquoi ou pourquoi pas ? 7. Comment surveillez-vous les activités en cours ? Sonde pour i. Formulaires à compléter IMPEL | Implementer-Led Evaluation & Learning 138 Annex 6: Data Collection Instruments ii. Comment et à quelle fréquence les formulaires sont-ils transmis au superviseur / aux offices de propriété intellectuelle ? iii. Comment il / compte les personnes formées ou visitées 8. Quels facteurs vous motivent à effectuer des travaux liés aux activités Agriculture et élevage dans le cadre du projet ? (Sondage sur les indemnités journalières, avancement professionnel, conditions de travail, reconnaissance de la communauté, autres avantages non financiers). Que pensez-vous de votre motivation à mener des activités ? Avez-vous une identification formelle qui indique que vous travaillez sur (nom du projet)? Globalement, êtes-vous satisfait de votre rôle ? Pourquoi ou pourquoi pas ? 9. Avec qui travaillez-vous sur les activités en cours de l’agriculture et des moyens de subsistance ? Comment fonctionne la coordination des activités sur le terrain ? Quels systèmes, le cas échéant, ont été mis en place pour partager les enseignements tirés ou les problèmes de mise en œuvre sur le terrain ? Si les défis sont confrontés, comment sont-ils abordés ? Veuillez décrire les réunions de coordination en cours. 10. Dans quelle mesure êtes-vous en mesure de mettre en œuvre les services agriculture et élevage tels que conçus dans le cadre de l'approche de projet ? Quels sont certains des défis rencontrés ? Y a-t-il des aspects de l'approche que vous ne pouvez pas mettre en œuvre ? Si oui, pourquoi ? Comment pourraient-ils être surmontés ? 11. CAHW, Comparé aux stratégies précédentes dans lesquelles vous avez été impliqué, en quoi l'approche a-t-elle changé la façon dont les membres de la communauté pratiquent l'élevage et assurent la santé des animaux ? i. i Agent VSLA, comparé aux stratégies précédentes dans lesquelles vous avez été impliqué, en quoi l'approche a-t-elle changé la manière dont les membres de la communauté ont accès aux VSLA? ii. Selon vous, quelle est la tendance générale en matière d’accès à l’élevage / aux armes à feu pour animaux domestiques / aux jardins permanents dans la zone que vous couvrez ? Augmente-t-il ou diminue-t-il ou n'y a-t-il pas de changement ? Qu'est-ce qui explique cette tendance ? 12. Que recommanderiez-vous pour améliorer les activités d’agriculture et de moyens de subsistance mises en œuvre dans le cadre des activités de la DFSA ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 139 GUIDE POUR LES DISCUSSIONS EN GROUPE POUR LE SECTEUR AGRICOLE I. ASPECTS AGRICOLES GENERAUX a. Généralités: - Depuis quand participez-vous aux activités agricoles dans votre région ? - A qui appartient la terre que vous exploitez ? - Quelles sont les précautions prises pour anticiper les enjeux fonciers et les actions collectives : ● Pourquoi vous avez choisi cette terre ? ● Quelles sont les modalités d’accès à la terre ? ● Etes-vous couvert par un acte d’engagement vous donnant accès à la terre ? ● Les femmes de votre communauté ont-elles les mêmes privilèges que les hommes pour l’accès à la terre ? ● Avez-vous d’autres contraintes ou exigences pour l’accès à la terre ? 1) PERMAGARDENS o Depuis quand pratiquez-vous cette activité ? o Avez-vous était formé pour la réalisation de cette activité ? Et de qui l’avez-vous réussi ? o Quand est-ce que vous avez commencé ? o Qui participe dans le permagarden ? o Tous les participants font le même travail ? o Quel type de culture vous planter ? o Qu’est-ce que vous faites avec les cultures ? o Cette activité vous rapporte-t-elle un certain changement dans votre ménage et quel type de changement ? o Quels sont les bénéfices majeurs que vous rencontrez pour la réalisation de cette activité ? o Quels sont les défis majeurs que vous rencontrez pour la réalisation de cette activité ? Et comment comptez-vous vous y prendre ? 2) MAREKETS GARDENS o Depuis quand pratiquez-vous cette approche au sein de votre communauté ? o Avez-vous étaient formés avant sa mise en œuvre ? o Comment êtes-vous organiser pour la réalisation de cette activité ? (Approche communautaire, Semi-communautaire ou individuelle ?) o Combien des personnes participent à la réalisation de cette activité ? Combien des femmes ? Est-ce ils ont les taches spécifiques ? o Avez-vous un calendrier de travaux ? o Quel est l’apport du projet (Non du Projet), sur la réalisation des travaux d’aménagement de votre bas-fond ? Quels types de semences avez-vous bénéficiers ou cultiver vous ? IMPEL | Implementer-Led Evaluation & Learning 140 Annex 6: Data Collection Instruments o Quel a été le niveau de votre participation aux travaux d’aménagement ? Avez-vous installé une pépinière pour ces activités ? Avec quels types des plantes ? D’où viens les semences ? Quels est votre finalité de cette activité ? o Avez-vous déjà procédé aux récoltes de vos produits de markets gardens, si oui comment celles-ci étaient organiser ? o Qu’est-ce que vous avez fait avec les produits de récolte ? o Quels sont les défis majeurs que vous rencontrez pour la réalisation de cette activité (contextuels, techniques, commercialisation…) ? 3) CULTURES VIVRIERES o Quelles sont les spéculations que vous utilisez pour vos champs des cultures vivrières ? o Avez-vous suivi les formations en rapport avec ses activités ? De qui l’avez-vous reçu ? o D’où provient les semences vous utilisez dans vos champs ? Est-ce les semences sont venues à temps ? Est-ce que ça répand au besoin formulée ? o Vos champs semenciers bénéficient-ils de l’encadrement technique de SENASEM pouvant aboutir à la certification de vos semences ? Comment vous stocker les semences ? Sont-elles traiter avec les produit phytosanitaires ? Est-ce que vous avez utilisé les sacs pics ? Quels est l’importances de sac pics ? o Qu’est-ce que vous avez fait avec les semences de récolte ? o Vos OP ont-elles des textes de base ? II. ASPECTS TRANSVERSAUX 1) SILCS o Depuis combien de temps cette activité est introduite dans votre communauté et par quel organisme ? o Votre association compte combien de membres ? Don’t combien de femme? o Quelles sont les responsabilités des femmes ? o Comment êtes-vous organiser/Quels sont les textes de base qui vous règlementent ? o Quel seuil minimal et maximal de vos dépôts/crédits exigez par l’organisation et à quelle condition ? o Avez-vous des activités communautaires pouvant renforcer la cohésion sociale ? Dans quels domaines ? o Au sein de votre association, existe-t-il un système de collecte pour les actions de solidarité ? o Etes-vous appuyer (assistance technique, financière, ou autre) dans cette activité ? Si oui par qui ? o Combien d’argent vous avez dans la caisse maintenant ? Combien de fois vous avez ouvrir la caisse ? Pourquoi vous avez ouvert la caisse? o Quelles sont les activités éligible chez vous ? 2) PROTECTION DE L’ENVIRONNEMENT Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 141 o Quels sont les activités que vous menez en rapport avec la conservation et la protection de l’environnement ? o Avez-vous installé une pépinière pour ces activités ? Avec quels types des arbres ? D’où viens les semences ? Quels est votre finalité de cette activité ? o Bénéficiez-vous d’un service/appui pour la réalisation de cette activité ? o Quels sont les défis majeurs que vous rencontrez dans le secteur environnemental ? 3) ACCESSIBILITE VOIE DE DESSERTE AGRICOLE o Votre zone de production est-elle connectée aux centres de commercialisation ? (observations passives) Comment vous était connectée ? Comment ça marche ? Quels sont les problèmes rencontrés pour la réalisation de cette activité ? o S’ils ne sont par connecter, avez-vous une planification pour vous les désenclaver ? o Le choix des axes routiers retenus pour réhabilitation, ont-ils fait objet d’un consentement avec les communautés ? o Quel a été le niveau de participation/implication à la réalisation de ces travaux ? 4) COMMUNICATION o Avez-vous un système de communication/information pour une bonne traçabilité de vos activités ? o Quels sont les mécanismes mises en place pour faire attendre vos préoccupations, attentes et au niveau du projet ? 5) SUIVI DES ACTIVITES SUR TERRAIN o Bénéficiez-vous d’un encadrement de proximité de la part du Projet, Acteurs Etatiques et autres de l’OSC ? A quelle fréquence ? o Est-ce que le système en place de supervision est efficace ? o A qui adressez-vous vos rapports d’activités ? IMPEL | Implementer-Led Evaluation & Learning 142 Annex 6: Data Collection Instruments GUIDE POUR LES DISCUSSION DE GROUPES AVEC LES FEMMES Introduction et salutation (suivant le formulaire de consentement) Question introductive 1. Que pouvez-vous nous dire au sujet des activités [nom du projet] dans votre village Quelles sont les activités que vous réalisez déjà au niveau de ce village Comment est-ce que les décisions ont été prises pour votre participation à ces activités ? Nutrition Nous allons parler maintenant de notre premier thème en rapport avec l’alimentation et la nutrition au niveau du village ● Avez-vous participé à des activités sur la nutrition et la santé au cours des derniers mois du projet (nom du projet) ? Si oui, à quelles activités avez-vous participé ? o Qui a dirigé l’activité ? o Quelles informations ont été partagées pendant les sessions ? o Quels matériels éducatifs ont été utilisés pour transmettre des messages ? o Quelles activités vous sont les plus utiles ? o Lesquelles parmi les activités vous a semblé moins utiles ? ● Les pratiques de soins et alimentaires de votre famille et votre enfant ont-elles changé depuis que vous avez commencé à participer aux activités du projet [non du projet]? Dans l’affirmative, quels changements spécifiques ont-ils été apportés? Pourquoi avez-vous opéré ces changements ? Selon vous, en quoi ces changements ont-ils affecté la santé de votre enfant ? ● Comment est-ce que votre mari est impliqué dans les activités de nutrition et santé? Comment est-ce que l’implication de votre mari dans les décisions concernant la consommation des aliments et les soins de santé a changé depuis le début des activités ? Comment est-ce que vos interactions avec votre mari dans le ménage ont changé en termes de décisions, discussions, appui? ● Quelles sont vos impressions en général sur les activités de nutrition ? ● Quelles sont vos recommandations pour améliorer les activités de nutrition et de santé dans votre village? WASH Nous allons parler maintenant de ce qui concerne l’eau, l’hygiène et l’assainissement au niveau du village 1. Qu’est-ce qui est en charge de puiser l’eau dans votre ménage? Où est-ce que cette personne va puiser l’eau? Combien de fois par jour cette personne va puiser l’eau ? a. SONDER: la source d’eau est à quelle distance de la maison? 2. Dans quoi transportez-vous l’eau à la maison? Est-ce qu’il y a un couvercle pour le récipient? 3. Une fois que l’eau est à la maison, est-ce que vous la traitez ? a. Si oui, avec quoi ? Depuis quand ? Avez-vous appris le traitement de l’eau avec le projet [nom du projet] ? 4. Est-ce que vous avez une latrine à la maison ? Depuis quand ? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 143 a. Est-ce que tous les membres de la maison utilisent la latrine facilement ? i. SONDER : difficultés pour les vieux, femmes pdt menstrues etc. b. Où est-ce que vous mettez les selles de vos enfants qui ont moins de 2 ans ? c. Est-ce qu’il y a eu un changement dans l’utilisation des latrines dans votre communauté depuis que le projet [non du projet] a commencé à travailler ici ? 5. Quand est-ce qu’on doit se laver les mains dans votre village? Quels sont les moments critiques ? a. SONDER : réponse est : après avoir déféqué, avant de cuisiner, après avoir changé les couches du bébé b. Est-ce facile de se laver les mains dans votre communauté ? C’est commun d’avoir des dispositifs de lavage des mains ? C’est commun d’avoir du savon et des cendres disponibles pour se laver les mains ? c. A votre avis, est-ce que les membres de la communauté se rappellent facilement de se laver les mains ? Si non, pourquoi pas ? 6. Comment est-ce que vous faites pour conserver de l’eau dans la maison ? a. Où est-ce que vous obtenez le récipient dont vous avez parlé? b. Quel est le plus proche endroit où vous pouvez acheter un nouveau récipient approprié? 7. Pouvez-vous décrire quels types de dispositifs de lavage de mains aviez-vous dans le village? a. Qu’est-ce que vous aimez ou n’aimez pas à propos de dispositifs de lavage de mains? b. Qu’est-ce qui devra changer dans les dispositifs de lavage de mains pour les rendre plus durables? 8. Qu’est-ce qui peut encourager les personnes dans votre village à utiliser convenablement les dispositifs de lavage de mains le plus souvent avec l’eau et le savon? Agriculture 1. Avez-vous participé à des activités sur la culture de maison (permagarden) or la culture maraîchère (market gardening) au cours des derniers mois du projet (nom du projet) ? Si oui, à quelles activités avez-vous participé ? a. Qui a dirigé l’activité b. Quelles sont les formations qui ont été données? c. Quelles informations ont été partagées pendant les sessions d. Quels matériels éducatifs ont été pour transmettre des messages ? e. Quelles activités vous sont les plus utiles ? f. Quelles activités vous ont semblé les moins utiles? 2. Depuis le début de votre participation dans les activités de l’agriculture, qu’est-ce que vous aviez déjà mis en application en relation avec les approches de culture de maison (permagarden) et la culture maraîchère (Market garden)? Qu’est-ce que vous pouvez nous dire à propos de ces activités ? Qu’est-ce qui à votre avis a été un succès ? Qu’est-ce qui n’a pas été un succès? Pouvez-vous expliquer? 3. Quelles sont vos impressions générales sur les activités d’agriculture dans votre village? 4. Quelles sont les recommandations (propositions) que vous formulez pour améliorer les activités d’agriculture dans votre communauté? Mécanisme de rétroaction et plaintes IMPEL | Implementer-Led Evaluation & Learning 144 Annex 6: Data Collection Instruments 1. Si vous aviez une remarque, une plainte ou quelques choses à dire sur le projet [nom du projet], comment est-ce que vous faites ou comment est-ce que vous devriez faire? a. Etes-vous au courant que de l’existence d’un numéro de téléphone que vous pouvez appeler pour faire vos remarques ? b. Est-ce qu’il existe des boîtes de suggestion dans votre village? c. Existe-il d’autres moyens que les membres de la communauté peuvent utiliser pour faire leurs remarques ou leur point de vue ? d. Est-ce que vous avez des soucis ou des appréhensions quant à l’utilisation de ces mécanismes? 2. Vous est-il déjà arrivé de faire des rétroactions, remarques ou plaintes sur le projet ? a. Quel type de remarques aviez-vous faites ? b. Quel mécanisme aviez-vous utilisé ? c. Est-ce que vous aviez obtenu une réponse en retour ? d. Si oui, étiez-vous satisfait de cette réponse ? e. Si vous n’aviez pas fait des rétroactions ou remarques, pourquoi ? 3. Est-ce que vous pensez que le projet est réactif (responsif) à vos remarques et plaintes ? Pourriez-vous expliquer pourquoi oui ou pourquoi non? Durabilité Nous allons parler maintenant de la durabilité 1. Est-ce que les personnes qui vous appuient dans les activités [non du projet] vous ont dit quand est-ce que les activités d’appui vous se terminez pour votre village ? Si non, quand est-ce que vous pensez que les activités d’appui [nom du projet] vont se terminer ? 2. Qu’est ce qui est actuellement fait pour que les activités dans votre village continuent après la fin de l’appui du projet [nom du projet] ? 3. Qu’est-ce que vous faites dans votre village pour vous assurer que les activités continuent même si l’appui du projet s’arrête? Remerciements et les informations de clôture Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 145 Mid-Term Evaluation Interview Tool: Management & Sustainability Background: This interview guide will be used by all technical leads on the MTE. These questions, related to sustainability, are “cross cutting” and should be adapted to the specific technical approach for each technical sector. These questions are intended to supplement the technical area specific tools that each technical lead will be using to carry out key informant interviews (KII) and focus groups (FG). The guide is broken into several sections: a section that will be used to interview the COP and other senior￾level technical leads; a section that will be used to interview field staff or other mid-level technical staff; a section that will be used to interview local stakeholders (e.g. local partners, government institutions); and a section that will be used to inform focus group discussions with participants. Evaluation Questions (What broader evaluation questions are we trying to answer using the tool?). [We are asking each technical MTE member to ask these questions to understand WHY the technical approaches are working or not working, and then make recommendations on how to improve the activity and the changes of seeing successful results going forward.] ● Assess the degree and benefits of efficacious coordination, collaboration, and convergence with locally-based project partners and external organizations that are critical to achieve project goals and purposes. (Evaluation Objective 4). Sub-questions under this objective include: o What mechanisms are in place to engage with local organizations, government entities, private sector, and/or other stakeholders? How strong are these relationships? To what extent will these stakeholders be able to sustain or take over interventions currently being implemented by the IP? o To what extent do local stakeholders help or hinder the implementation of activities? What has the IP done to overcome these challenges or build on successes? o To what extent is the consortium operating as a single team? Are the Prime and subs collaborating effectively? o How has the prime IP worked with other USAID activities in the implementation area (if any)? What has worked well and what has not worked well? What factors are within the manageable interest of the IP and what are not? How has the IP managed the relationship/s with other USAID activity implementing partners? ● Assess early evidence of sustainability produced by the project activities thereby determining the extent to which outcomes, systems, and services are designed and being implemented to continue after the project ends. (Evaluation Objective 5). Sub-questions under this objective include: o To what extent has the activity documented and implemented a sustainability strategy? (Note that partners should have a plan, but the goal is to see how the process is going to implement that plan and see if it is realistic. It will be important to know if there is convergence in opinions on sustainability across the senior IP leadership, subs, partners, local institutions, and even participants.) o How will positive outcomes be sustained after the activity ends? Which local systems/structures/entities will sustain these outcomes? o What mechanisms are in place (formal/informal) that will support these sustainable outcomes? (This should build on what was identified as part of Objective 4 above.) ● Assess the appropriateness and quality of the R&I approach/activities. How well did the R&I approach work? (Evaluation Objective 6). Sub-questions under this objective include: IMPEL | Implementer-Led Evaluation & Learning 146 Annex 6: Data Collection Instruments o How have R&I activities been used to inform the design and ongoing modification of activity implementation? o To what extent has R&I been used beyond the initial refinement period and during the implementation phase? o Do IPs have the capacity to use R&I activities as planned? What adjustments are needed to ensure that R&I activities can be used to their fullest? o What are some of the challenges faced in implementing R&I activities and what adjustments need to be made to address and minimize these factors in the future? o How can the R&I model be improved to better ensure that R&I activities are effectively used to strengthen design and implementation of DFSAs? [Identify internal recommendations for FFP, not partners] o Did the DFSAs meet standards to make them eligible for a cost extension based on the R&I model? What, if any, changes should be made in determining eligibility for a cost extension? [Identify internal recommendations for FFP, not partners] Questions for Partner Senior Staff Key Informant Interviews: 1. Does the activity have a sustainability plan? What are the key features of the plan? Can we get a copy of the plan? 2. How far along is the activity (broadly speaking) in implementing the plan? 3. What kinds of challenges or unexpected hurdles have you encountered in implementing the plan thus far? 4. What has been going well in implementing the plan thus far? 5. [Partnerships]: Who or which institutions are critical in realizing the sustainability plan? In other words, what local partners have you identified that will continue providing services and/or doing things to sustain positive results (and mitigate negative ones)? This may include local organizations who will continue providing services/etc. beyond the LOA; also local government and/or chiefdom/traditional leadership; local religious organizations/structures; and others. 6. [Partnerships]: Has the activity created formalized agreements with these institutions, such as MOUs? Can we review (or can you walk us through/describe) any other documents related to sustainability planning (e.g. MOU’s with partners, local government institutions, chiefdom leaders, care group plans, et cetera). 7. [Governance]: How is the activity [in X sector] working with the local government? What has been going well and what has not been going well thus far? How will partnership with local government work going forward? 8. [Governance]: How is the activity [in X sector] working with the local government? What has been going well and what has not been going well thus far? How will partnership with local government work going forward? 9. [Governance]: To what extent has poor public service delivery or public financial management [in X sector] been a hindrance or enabling factor to implementation? (While PFM may be beyond the manageable interest of the partners, it’s important to know if this is a big potential problem.) Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 147 10. [Partnerships]: How is the activity working with local NGOs [in X sector]? What has been going well and what has not been going well thus far? How will partnership with local NGOs work going forward? 11. [Partnerships]: How is the activity working with other local institutions [in X sector], e.g. civil society organizations, religious groups, et cetera? What has been going well and what has not been going well thus far? How will partnership with these entities work going forward? 12. [Partnerships]: How is the activity working with private sector entities (e.g. input supplies, training institutions, professional service providers)? What has been going well and what has not been going well thus far? How will partnership with these entities work going forward? Questions for Partner Field-Based (or non-senior leadership) Technical Staff Key Informant Interviews: 1. Does the activity have a sustainability plan? What are the key features of the plan? (In other words, how are activities currently being implemented by staff going to continue after the end of the activity?) [[The goal here is to see if the tech teams and senior leadership are in alignment about what the sustainability planning looks like.]] 2. Broadly speaking, how do you envision sustainability within your technical sector? (In other words, what would a sustainable [sector X] plan look like? This might include activities with mother care groups, business plans with private sector partners, trainings, linkages with other partners.) 3. Do you think the activity is currently doing everything needed to achieve this/your vision of what sustainability should look like? Why/why not? 4. What kinds of challenges or unexpected hurdles has your team encountered in implementing the plan thus far? 5. What has been going well in implementing the plan thus far? 6. [Governance]: How is the activity [in X sector] working with the local government? What has been going well and what has not been going well thus far? How will partnership with local government work going forward? 7. [Governance]: How is the activity [in X sector] working with the local government? What has been going well and what has not been going well thus far? How will partnership with local government work going forward? 8. [Governance]: To what extent has poor public service delivery or public financial management [in X sector] been a hindrance or enabling factor to implementation? (While PFM may be beyond the manageable interest of the partners, it’s important to know if this is a big potential problem.) 9. [Partnerships]: How is the activity working with local NGOs [in X sector]? What has been going well and what has not been going well thus far? How will partnership with local NGOs work going forward? 10. [Partnerships]: How is the activity working with other local institutions [in X sector], e.g. civil society organizations, religious groups, et cetera? What has been going well and what has not been going well thus far? How will partnership with these entities work going forward? IMPEL | Implementer-Led Evaluation & Learning 148 Annex 6: Data Collection Instruments 11. [Partnerships]: How is the activity working with private sector entities (e.g. input supplies, training institutions, professional service providers)? What has been going well and what has not been going well thus far? How will partnership with these entities work going forward? Questions for Partner Organizations (e.g. Subs, Local NGOs, Private Sector Entities, Local Government Institutions) Key Informant Interviews: 1. What is your organization/institution’s current role on the activity? 2. Do you know when the activity is ending? 3. What is your understanding of how the activity’s programming will continue beyond the end of the activity? 4. What is your organization/institution’s specific role in the continuation of [activities/service delivery/other intervention] after the activity has ended? 5. Does leadership in your institution support taking on these activities/service delivery/other in the future? 6. Does your institution have sufficient resources (e.g. funding, human resources, capacity, interest, systems, etc.) and interest to do this [i.e. take over activities]? Why or why not? Questions for Participants (Focus Groups and, if needed, KII’s): 1. Do you know when [DFSA activity name] will end? Do you know what will happen after the end of [activity name]? (In other words, do you know if another organization in your community will be able to provide X services that you are currently receiving through [DFSA activity name]?) 2. Do you see much value in adopting the practices/technologies being promoted by [DFSA activity name]? o WASH: use of latrines? Covered latrines? o WASH: handwashing at 5 key times o WASH/Nutrition: food preparation and food storage o Nutrition: diverse eating; cooking o Agriculture: IPM, GAP, etc. 3. Are you currently receiving services (e.g. veterinary, ag extension) through the program? How willing would you be to pay for this service in the future? How much would you be willing to pay? 4. [For community-based groups, e.g. water use, farmer groups]: o Does your community group charge user fees? o Is there a management committee? o Can you show us how you keep your books and/or manage the day to day actions required to manage the group finances? o Is there an annual planning process? How does that work and who is involved in that process? o Does the group have a budget? o How is the group managed in terms of leadership and planning? o Are the fees collected enough to sustain the activities/service provision in the long term (i.e. to cover costs of maintenance)? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 149 o What is the role of local government in providing this service, if any? 5. [Governance]: What do you feel is the role of the local government in providing key services? [This should be asked in a way that is specific to the specific service and the relevant level/s of government and/or chiefdom/local leadership] o Water access o Health services 6. [Governance]: Do you feel that the government is accountable to citizens? For example, if a health clinic is supposed to carry certain supplies but they sometimes do not have those supplies, do community members feel that anything can be done to change this? [We need to refine this one…] IMPEL | Implementer-Led Evaluation & Learning 150 Annex 6: Data Collection Instruments Mid-Term Evaluation Interview Tool: M&E Background: This interview guide will be used by the M&E technical advisors on the MTE. The guide is broken into several sections: a section that will be used to interview the COP and other senior-level technical leads; a section that will be used to interview field staff or other mid-level technical staff; a section that will be used to interview local stakeholders (e.g. local partners, government institutions); and a section that will be used to inform focus group discussions with participants. Evaluation Questions (What broader evaluation questions are we trying to answer using the tool?). [We are asking each technical MTE member to ask these questions to understand WHY the technical approaches are working or not working, and then make recommendations on how to improve the activity and the changes of seeing successful results going forward.] ● Review the quality of program service delivery related to the different themes and systems in addressing chronic food insecurity and child malnutrition with the targeted clients, taking into account contextual changes that may have occurred since the inception of the activities. (Objective 1) o What [M&E] systems are used to capture, document, and share lessons learned? How are these lessons learned then use to continue ensuring/improving program quality? How is stakeholder and/or participant feedback/perspectives included in this system? ● Identify evidence of changes (positive and negative, intended and unintended) associated with program interventions. This will include identifying factors that appear to promote or hinder women and men’s, and young people’s, food security and safety. (Objective 2) o What changes do community members and other stakeholders associate with the project’s interventions? What factors appear to promote and deter the changes? How do the changes correspond to those hypothesized by the project’s TOC? o To what extent are there observable gender discrepancies? Discrepancies between youth and adults? How has the activity design accounted for these discrepancies and “course corrected” to mitigate these discrepancies? (How is activity data being used to support these kinds of analyses?) o How could the project be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the project’s TOC be refined or modified? ● Assess the degree and benefits of efficacious coordination, collaboration, and convergence with locally-based project partners and external organizations that are critical to achieve project goals and purposes. (Evaluation Objective 4). Sub-questions under this objective include: o What mechanisms are in place to engage with local organizations, government entities, private sector, and/or other stakeholders? How strong are these relationships? To what extent will these stakeholders be able to sustain or take over interventions currently being implemented by the IP? o To what extent do local stakeholders help or hinder the implementation of activities? What has the IP done to overcome these challenges or build on successes? o How has the prime IP worked with other USAID activities in the implementation area (if any)? What has worked well and what has not worked well? What factors are within the Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 151 manageable interest of the IP and what are not? How has the IP managed the relationship/s with other USAID activity implementing partners? ● Assess early evidence of sustainability produced by the project activities thereby determining the extent to which outcomes, systems, and services are designed and being implemented to continue after the project ends. (Evaluation Objective 5). Sub-questions under this objective include: o To what extent has the activity documented and implemented a sustainability strategy? (Note that partners should have a plan, but the goal is to see how the process is going to implement that plan and see if it is realistic. It will be important to know if there is convergence in opinions on sustainability across the senior IP leadership, subs, partners, local institutions, and even participants.) o How will positive outcomes be sustained after the activity ends? Which local systems/structures/entities will sustain these outcomes? o What mechanisms are in place (formal/informal) that will support these sustainable outcomes? (This should build on what was identified as part of Objective 4 above.) ● Assess the appropriateness and quality of the R&I approach/activities. How well did the R&I approach work? (Evaluation Objective 6). Sub-questions under this objective include: o How have R&I activities been used to inform the design and ongoing modification of activity implementation? o To what extent has R&I been used beyond the initial refinement period and during the implementation phase? o Do IPs have the capacity to use R&I activities as planned? What adjustments are needed to ensure that R&I activities can be used to their fullest? o What are some of the challenges faced in implementing R&I activities and what adjustments need to be made to address and minimize these factors in the future? o How can the R&I model be improved to better ensure that R&I activities are effectively used to strengthen design and implementation of DFSAs? [Identify internal recommendations for FFP, not partners] o Did the DFSAs meet standards to make them eligible for a cost extension based on the R&I model? What, if any, changes should be made in determining eligibility for a cost extension? [Identify internal recommendations for FFP, not partners] Questions for Partner Senior-Level and Technical Staff: 1. What function/s does your M&E system serve (e.g. reporting, monitor performance, track progress, accountability)? 2. Does your team use the data collected though the official M&E system or do you use another data collection/monitoring system to complement or replace that official system? 3. How useful is the M&E Plan in program management? How useful is the existing data collected? 4. Do non-M&E staff generally feel that the M&E system is used for performance tracking and improving quality or for reporting? 5. [If applicable]: How are you conducting PDM? What information is collected and how is it used? IMPEL | Implementer-Led Evaluation & Learning 152 Annex 6: Data Collection Instruments 6. Does the project document lessons learned? How are these lessons documented? What changes have been brought due to these lessons learned? How are reports and other information shared across stakeholders? 7. [Preface this question with a note about how we want to be objective, and that if the interviewee would rather talk to the Eval Team Lead, that’s fine]: What’s been the team’s experience working with FFP M&E guidance (e.g. the P&G)? Have the Refine and Implement and Theory of Change 1. What was your experience with the R&I approach? What worked well and what didn’t work well? 2. How did your team use the findings from the R&I period to inform the technical design? 3. How did your team find the TOC as a design/management/other tool? To what extent did your team use the R&I process to refine the activity design using the TOC? Data Utilization 1. How is data being used? Who prepares reports or other products and who gets reports? Is information shared with others? Who? 2. Is data shared with the community? How? 3. What data do you find most useful and meaningful? 4. What challenges do you face to using data more effectively? 5. How has the project used results from assessments and formative research? 6. How is the project using data generated from routine monitoring? What decisions could be linked to the M&E system? 7. How does the M&E team see its role around data utilization? i. Coordination, Collaboration, and Convergence ii. Building on the questions about partnership we asked previously, Management and Budget 1. How do staff see the relationship between the prime and sub? [Ask this question differently based on who we’re talking to.] 2. To what extent do staff from different organizations operate as “one team”? Questions for Partner M&E Staff (Should include Prime and Subs) Monitoring system 1. How does the prime partner conduct routine monitoring? How do the subs conduct routine monitoring? (Review the system & walk through process) 2. How does data flow for different sectors? o Who collects? What are their capacities? What types of training they received related to data collection? Do they have any written guidance? 3. How often is data being collected? How is data reviewed by the prime? How is quality ensured? 4. What other responsibilities do the people responsible for M&E also have in the different sectors? 5. How is qualitative data recorded? What kinds of qualitative information is being systematically collected? What kind of qualitative information is NOT being collected? How is qualitative Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 153 information being integrated into the M&E system and, more specifically, synthesized with quantitative data (RM or AS data)? 6. [If applicable]: How are non-implementing partner entities (e.g. local institutions or local NGO partners) involved in the M&E system? To what extent do your data rely on staff from these non-IP entities? What is the capacity of these non-IP entities to perform monitoring? What challenges do they face? 7. How is data safeguarded? Qualitative v. Quantitative? 8. Is the data flow working well? Are there any challenges being faced (timeliness, data losses, data manipulation)? 9. How does the project staff collect annual monitoring data? What are the challenges? 10. How did they establish the base values? How do they set up targets? [Review some of the key annual monitoring indicators to assess the quality including one behavior change uptake indicator.] How could this process be improved? 11. How are community-based activities monitored for quality? For evidence of behavior change? What has been learned from these? What could be learned from these? 12. How has the team incorporated new FFP and FTF indicators into their monitoring systems? 13. How was gender equality considered in designing project monitoring system, indicators, reporting and data utilization? 14. What else can you share with us about the monitoring system? Complaint & Feedback Mechanism 1. Does the project have an operational complaints and feedback mechanism? 2. Who manages the system? Is it independent of project management? 3. How are complaints and feedback tracked? Are responses given, and if so is response time tracked? 4. How is information from the system used to inform project implementation? 5. Does the project have criteria for significant issues (e.g. fraud, waste, and abuse), and if so, how are these issues addressed? Data utilization 1. How is data being used? Who prepares reports or other products and who gets reports? Is information shared with others? Who? 2. Is data shared with the community? How? 3. What data do you find most useful and meaningful? 4. What challenges do you face to using data more effectively? 5. How is the project using data generated from routine monitoring? What decisions could be linked to the M&E system? 6. How does the M&E team see its role vis-a-vis data utilization? 7. What else can you share with us about how your team is utilizing data? R&I and Learning 1. How were the findings from the formative research (e.g. in year 1) incorporated into the M&E Plan? Who was involved in this process and how did it go? IMPEL | Implementer-Led Evaluation & Learning 154 Annex 6: Data Collection Instruments 2. To what extent did the Theory of Change (TOC) change as a result of the R&I process? 3. What worked well and what did not work well about the R&I approach? 4. What worked well and did not work well specifically around M&E during and immediately following the first year? 5. What else would you like to tell us about the R&I approach? Staffing & Staff Capacity 1. How are training needs and TA determined? For M&E staff vs. non-M&E staff? 2. Do you feel that field staff are adequately trained and able to collect data in alignment with what is outlined in the M&E Plan? Context Monitoring & Adaptive Management 1. What changes have occurred in the operating context (e.g., major events like disasters, slow onset events like climate change, government policy changes, etc.) that have affected project implementation, either positively or negatively? How did these specifically affect implementation? What did the project do to adapt to these changes? 2. How have these changes affected the food insecurity or resilience of targeted impact groups for each project? What other groups are becoming more food insecure or are having reduced resilience as a result of these contextual changes? 3. Given these changes in the operating environment, which project activities seem most relevant now in terms of having impact on the lives of targeted impact groups? 4. What suggestions do you have for adapting the project's strategy or implementation systems in order to be better able to respond to changes in the operating context? 5. Since the activity began implementation, have there been any interruptions or delays in the flow of resources, either food or cash or both? If so, what caused these delays, and what was the impact on project implementation? What did the project do to adapt? M&E Questions for Field-Based Staff: 1. What role does performance monitoring play in the activity? 2. What are your key roles/responsibilities for collecting data? Do you know what happens with the data you collect (after you submit the information)? 3. Do you feel like the data you’re gathering accurately reflects the situation on the ground? o For example, are those data points you are collecting accurately reflect the participation of women, youth, or other groups? If not, what is missing? o How does your team gather qualitative data? (Are anecdotes about things happening at trainings getting shared with the senior-level technical staff? How are these aggregated? Who analyzes and/or uses this information?) 4. If you could decide what information to gather about the effectiveness or other characteristics of the program, what information would you gather? Why (what’s different about what you would do as compared to what is currently happening)? 5. How involved are you in the analysis of monitoring data? What information do you receive, and when? How do you use this information once you have it? Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 155 o For example, are you given a report or raw data? What are you expected to do with that information? o What do you do if the information/reports you are receiving conflict with what you are seeing on the ground? Do you feel empowered to speak up to share your thoughts? How do you typically do this? 6. What training did you receive on M&E? o Data collection processes? o Informed consent and research ethics? o Data analysis methods o Other? 7. Operating Context o What changes have occurred in the operating context (e.g., major events like disasters, slow onset events like climate change, government policy changes, etc.) that have affected project implementation, either positively or negatively? How did these specifically affect implementation? What did the project do to adapt to these changes? o How have these changes affected the food insecurity or resilience of targeted impact groups for each project? What other groups are becoming more food insecure or are having reduced resilience as a result of these contextual changes? o Given these changes in the operating environment, which project activities seem most irrelevant now in terms of having impact on the lives of targeted impact groups? o What suggestions do you have for adapting the project's strategy or implementation systems in order to be better able to respond to changes in the operating context? o Since the Project began implementation, have there been any interruptions or delays in the flow of resources, either food or cash or both? If so, what caused these delays, and what was the impact on project implementation? o What did the project do to adapt? Local Implementation Office, extension workers, DAs ● Assess data collection system, records, data flow and information shared back. Ask for demonstration of what information they have access to. ● What data do you collect? Why do you collect this data? How do you record information? Review the paper forms ● For those people collecting data, what other roles do they have? Do they receive support to collect data? Do they receive instructions or training? ● What support do you need to collect data? ● What happens with the data you collect? ● What information do you receive from the woreda? From REST? From MoA/ MoH? ● How is data collected? How do you ensure that it is correct? ● What is done with the data you provide? Who do you provide the data to? ● How are decisions made using the data you provide? IMPEL | Implementer-Led Evaluation & Learning 156 Annex 6: Data Collection Instruments ● Demonstrate how measure: hectares, land under improved technologies, health data collected by HEWs- weighing of children, whether women are receiving ANC visits Trainers and Participants  Inquiry around trainings and behavior change.  Utility of trainings. Have trainings ever changed? What support do you receive? What do you need?  Assess registration logs. Interview HHs about the trainings, what they learned and how they use that information to assess and breakdowns in behavior change occurrence.  If possible, walk through training material with participants to assess understanding and quality of material  What would you like to learn in the trainings? Documents  Review training material or guides if available on M&E  (REST) website, newsletter and learning magazine that have these knowledge dissemination goals  Training material frontline extension agents  USAID DQA reports  Reports Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 6: Data Collection Instruments 157 Questions for CRS Accountability Manager 1. How long have you been in this role with CRS? 2. How would you describe your role with Budakadidi? 3. Can you describe the Budakadidi accountability system? a. Who is responsible for the system at each level? b. How are complaints and feedback tracked? Are responses given, and if so is response time tracked? c. How is information from the system analyzed and used to inform project implementation? What guidance is given to project staff on how to use the information to learn and adapt? d. Does the project have criteria for significant issues (e.g. fraud, waste, and abuse), and if so, how are these issues addressed? 4. How was this system developed? What information was used to inform the design of this system? a. Were consultations held with communities to determine their preferred feedback mechanisms? 5. What training has been conducted on the system? Who was trained? 6. How accessible is the system? a. What days is the Green Line available? b. What were the reasons for limiting the number of days it is available? c. Were these changes communicated to participants? d. How do participants without cell phones provide feedback? What about participants who don’t have suggestion boxes in their communities? 7. Can you describe how the suggestion boxes are used? a. How were the locations for the suggestion boxes chosen? b. Are there plans to add additional suggestion boxes to increase coverage? 8. What is the most frequent mechanism used for giving feedback (Green Line, suggestion boxes, verbally to staff, etc.)? 9. Is there a process for tracking and responding to verbal feedback? 10. How have you gauged the effectiveness of the accountability system so far? 11. What plans do you have to assess how well the feedback mechanisms are operating? (note: Budakadidi staff mentioned that the Accountability Manager was planning an assessment of the feedback mechanism). IMPEL | Implementer-Led Evaluation & Learning 158 Annex 7: Consent Scripts/Forms ANNEX 7: CONSENT SCRIPTS/FORMS Informateurs clés Bonjour ! Je m’appelle XXX. Je suis ici accompagné d’une équipe d’évaluation qui cherche à recueillir des informations pour mieux comprendre les activités que votre organisation met en œuvre dans le cadre des approches programmatiques et opérationnelles utilisées dans le projet (nom du projet), la qualité et l’efficacité des services et les changements qui doivent être apportés pour améliorer ces services. Nous vous prions de bien vouloir participer à l’évaluation en tant qu’informateur clé. Au cours de l’entretien, nous vous poserons des questions sur les activités de (nom du projet), y compris votre rôle dans la conception et la mise en œuvre des activités, les détails des activités liées à votre expertise technique, la manière dont les activités sont suivies et évaluées, et la collaboration continue avec vos collègues et autres partenaires d’exécution. Nous vous demanderons également votre point de vue sur la qualité et l’efficacité des activités, sur les difficultés rencontrées dans la mise en œuvre du programme et sur les recommandations visant à renforcer le (nom du projet). Votre expérience en tant que chef d’équipe, conseiller technique ou partenaire dans les activités du projet sera très utile à notre compréhension et à notre connaissance du projet. L’information recueillie dans le cadre de cette évaluation pourrait également servir à d’autres fins d’analyse dans l’avenir. L’entretien durera environ 1 h 30. Nous reviendrons probablement pour d’autres entretiens, qui seront de plus courte durée. Il n’y a pas de bonnes ou de mauvaises réponses aux questions que nous poserons au cours de cette entretien et des suivants. Vous pouvez choisir de ne pas répondre aux questions si vous préférez ne pas le faire. Votre participation à l’évaluation est entièrement volontaire. C’est à vous de choisir si vous voulez participer ou non. La non-participation n’entraînera aucune répercussion négative. Si vous participez à l’évaluation et que vous souhaitez arrêter, vous pouvez le faire à tout moment. Toutes les informations que vous nous fournirez seront traitées de manière strictement confidentielle. Pendant l’entretien, nous prendrons des notes manuscrites et l’entretien sera également enregistrée à l’aide d’un dictaphone. Les notes manuscrites et les enregistrements audio seront conservés en lieu sûr et ne seront partagés avec aucune personne autre que les membres de l’équipe d’évaluation. À la fin de l’évaluation, nous analyserons les données et produirons un rapport. Vous ne serez pas cité nommément dans un résumé, une présentation ou un rapport des résultats de l’évaluation. L’information enregistrée sera détruite une fois l’analyse des données terminée. Prenez tout le temps nécessaire pour prendre la décision de participer à cette évaluation. Si vous avez des questions ou des préoccupations au sujet de l’évaluation, que ce soit avant ou après la signature du présent formulaire, vous pouvez en aviser l’enquêteur principal de l’évaluation dont les coordonnées figurent ci-dessous. Pour toute question relative à l’évaluation, y compris les plaintes ou les questions concernant vos droits en tant que participant à l’évaluation, vous pouvez appeler l’Enquêteur (« chercheur ») sur le site d’évaluation : Mapatano Mala Ali. Téléphone : +243 09 9894 7313 Si vous avez des questions, n’hésitez pas à les poser à moi-même ou à un autre membre de l’équipe d’évaluation. Accepteriez-vous de participer à l’évaluation ? 1. Oui 2. Non Accepteriez-vous que nous enregistrions la conversation ? 1. Oui 2. Non Signature de la personne chargée de l’entretien : __________________Date : ____/_____/2019 Nom du participant : ________________________ Signature du participant : ________________________ Date : ____/_____/2019 Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 7: Consent Scripts/Forms 159 Informateurs de l’entretien approfondi Bonjour ! Je m’appelle XXX. Je suis ici accompagné d’une équipe d’évaluation qui cherche à recueillir des informations pour mieux comprendre les activités que votre organisation met en œuvre dans le cadre du projet (nom du projet), les approches programmatiques utilisées, la qualité et l’efficacité des services et les changements qui doivent être apportés pour améliorer ces services. Nous vous prions de bien vouloir participer à un entretien dans le cadre de l’évaluation. Au cours de l’entretien, nous vous poserons des questions sur les activités du projet (nom du projet), y compris votre rôle dans les activités, les détails des activités auxquelles vous participez, la formation que vous avez reçue, votre horaire de travail et vos responsabilités, votre satisfaction par rapport à votre rôle et vos responsabilités et votre collaboration continue avec vos collègues et partenaires opérationnels. Nous vous demanderons également votre point de vue sur la qualité et l’efficacité des activités, sur les difficultés rencontrées dans la mise en œuvre du programme et sur les recommandations visant à renforcer le projet (nom du projet). Votre expérience en tant qu’agent de terrain dans les activités du projet sera utile à notre compréhension et à notre connaissance du projet. Les informations recueillies dans le cadre de cette évaluation pourraient également être utilisées à d’autres fins d’analyse à l’avenir. L’entretien durera environ une heure. Si nous ne parvenons pas à poser toutes nos questions, il se peut que nous ayons à revenir plus tard pour un autre entretien. Il n’y a pas de bonnes ou de mauvaises réponses aux questions que nous poserons au cours de cet entretien et de ceux qui suivront. Vous pouvez choisir de ne pas répondre aux questions si vous le souhaitez. Votre participation à l’entretien est entièrement volontaire. C’est à vous de choisir si vous voulez participer ou non. La non-participation n’entraînera aucune répercussion négative liée à votre travail. Si vous acceptez de participer à l’évaluation et que vous souhaitez arrêter, vous pouvez le faire à tout moment. Toutes les informations que vous nous fournirez seront traitées de manière strictement confidentielle. Pendant l’entretien, nous ferons des prises de notes manuscrites et l’entretien sera également enregistré sur dictaphone. Les notes manuscrites et les enregistrements audio seront conservés en lieu sûr et ne seront partagés avec aucune personne autre que les membres de l’équipe d’évaluation. À la fin de l’évaluation, nous analyserons les données et produirons un rapport. Vous ne serez pas cité nommément dans un résumé, une présentation ou un rapport des résultats de l’évaluation. L’information enregistrée sera détruite une fois l’analyse des données terminée. Prenez tout le temps nécessaire pour prendre la décision de participer à cette évaluation. Si vous avez des questions ou des préoccupations au sujet de l’évaluation, que ce soit avant ou après la signature du présent formulaire, vous pouvez en aviser l’enquêteur principal de l’évaluation dont les coordonnées figurent ci-dessous. Pour toute question relative à l’évaluation, y compris les plaintes ou les questions concernant vos droits en tant que participant à l’évaluation, vous pouvez appeler l’Enquêteur (chercheur) sur le site d’évaluation : Mapatano Mala Ali. Téléphone : +243 09 9894 7313 Si vous avez des questions, n’hésitez pas à les poser à moi-même ou à un autre membre de l’équipe d’évaluation. Accepteriez-vous de participer à l’évaluation ? 1. Oui 2. Non Accepteriez-vous que nous enregistrions la conversation ? 1. Oui 2. Non Signature de la personne chargée de l’entretien : __________________Date : ____/_____/2019 Nom du participant : ________________________ Signature du participant : ________________________ Date : ____/_____/2019 IMPEL | Implementer-Led Evaluation & Learning 160 Annex 7: Consent Scripts/Forms Informateurs de l’entretien approfondi (aux villages) Bonjour ! Je m’appelle XXX. Je suis ici accompagné d’une équipe d’évaluation qui cherche à recueillir des informations pour mieux comprendre les activités que votre organisation met en œuvre dans le cadre du projet (nom du projet), les approches programmatiques utilisées, la qualité et l’efficacité des services et les changements qui doivent être apportés pour améliorer ces services. Nous vous prions de bien vouloir participer à un entretien dans le cadre de l’évaluation. Au cours de l’entretien, nous vous poserons des questions sur les activités du projet (nom du projet), y compris votre rôle dans les activités, les détails des activités auxquelles vous participez, la formation que vous avez reçue, votre horaire de travail et vos responsabilités, votre satisfaction par rapport à votre rôle et vos responsabilités et votre collaboration continue avec vos collègues et partenaires opérationnels. Nous vous demanderons également votre point de vue sur la qualité et l’efficacité des activités, sur les difficultés rencontrées dans la mise en œuvre du programme et sur les recommandations visant à renforcer le projet (nom du projet). Votre expérience en tant qu’agent de terrain dans les activités du projet sera utile à notre compréhension et à notre connaissance du projet. Les informations recueillies dans le cadre de cette évaluation pourraient également être utilisées à d’autres fins d’analyse à l’avenir. L’entretien durera environ une heure. Si nous ne parvenons pas à poser toutes nos questions, il se peut que nous ayons à revenir plus tard pour un autre entretien. Il n’y a pas de bonnes ou de mauvaises réponses aux questions que nous poserons au cours de cet entretien et de ceux qui suivront. Vous pouvez choisir de ne pas répondre aux questions si vous le souhaitez. Votre participation à l’entretien est entièrement volontaire. C’est à vous de choisir si vous voulez participer ou non. La non-participation n’entraînera aucune répercussion négative liée à votre travail. Si vous acceptez de participer à l’évaluation et que vous souhaitez arrêter, vous pouvez le faire à tout moment. Toutes les informations que vous nous fournirez seront traitées de manière strictement confidentielle. Pendant l’entretien, nous ferons des prises de notes manuscrites et l’entretien sera également enregistré sur dictaphone. Les notes manuscrites et les enregistrements audio seront conservés en lieu sûr et ne seront partagés avec aucune personne autre que les membres de l’équipe d’évaluation. À la fin de l’évaluation, nous analyserons les données et produirons un rapport. Vous ne serez pas cité nommément dans un résumé, une présentation ou un rapport des résultats de l’évaluation. L’information enregistrée sera détruite une fois l’analyse des données terminée. Prenez tout le temps nécessaire pour prendre la décision de participer à cette évaluation. Si vous avez des questions ou des préoccupations au sujet de l’évaluation, que ce soit avant ou après la signature du présent formulaire, vous pouvez en aviser l’enquêteur principal de l’évaluation dont les coordonnées figurent ci-dessous. Pour toute question relative à l’évaluation, y compris les plaintes ou les questions concernant vos droits en tant que participant à l’évaluation, vous pouvez appeler l’Enquêteur (chercheur) sur le site d’évaluation : Mapatano Mala Ali. Téléphone : +243 09 9894 7313 Si vous avez des questions, n’hésitez pas à les poser à moi-même ou à un autre membre de l’équipe d’évaluation. Accepteriez-vous de participer à l’évaluation ? 1. Oui 2. Non Accepteriez-vous que nous enregistrions la conversation ? 1. Oui 2. Non Signature de la personne chargée de l’entretien : __________________Date : ____/_____/2019 Nom de la personne chargée de l’entretien : ________________________ Village, date d’entretien Mid-Term Evaluation of the Budikadidi DFSA in the DRC Annex 7: Consent Scripts/Forms 161 Participants au groupe de discussion Bonjour ! Je m’appelle XXX. Je suis ici accompagné d’une équipe d’évaluation qui cherche à recueillir des informations pour mieux comprendre les services que vous recevez du projet (nom du projet), votre perception des services et les changements que vous recommandez pour améliorer ces services. L’objectif général de l’évaluation est d’examiner les approches programmatiques, la qualité et l’efficacité des activités mises en œuvre. Nous aimerions vous demander de participer à une discussion de groupe avec d’autres personnes de votre communauté. Au cours de la discussion, nous vous poserons des questions concernant les activités du projet (nom du projet). Votre expérience en tant que participant aux activités du projet sera très utile à notre compréhension et à notre connaissance du projet. L’information recueillie dans le cadre de cette évaluation pourrait également servir à d’autres fins d’analyse dans l’avenir. Votre participation à un groupe de discussion durera environ 1h45. Il n’y a pas de mauvaises réponses aux questions que nous allons poser au cours de la discussion. Vous pouvez choisir de ne pas répondre aux questions si vous le préférez. Votre participation à la discussion de groupe est entièrement volontaire. C’est à vous de choisir si vous voulez participer ou non. La non-participation n’affectera en rien les services que vous recevez habituellement. Si vous acceptez de participer à l’évaluation et que vous souhaitez arrêter, vous pouvez le faire à tout moment. Vous ne perdrez aucune de vos prestations régulières. Compte tenu de la nature de la discussion de groupe, il est impossible de garantir l’anonymat. Nous vous demandons, ainsi qu’aux autres participants à la discussion de groupe, de préserver la confidentialité des discussions au sein du groupe. Au cours de la discussion de groupe, ferons des prises de notes manuscrites. Les discussions seront également enregistrées à l’aide d’un dictaphone. Les idées et les positions exprimées par vous et les autres participants seront partagées aussi fidèlement que possible par les personnes chargées de la collecte des données. Les notes manuscrites et les enregistrements audio seront conservés en lieu sûr et ne seront partagés avec aucune personne autre que les membres de l’équipe d’évaluation. À la fin de l’évaluation, nous analyserons les données et produirons un rapport. Vous ne serez pas cité nommément dans un résumé, une présentation ou un rapport des résultats de l’évaluation. L’information enregistrée sera détruite une fois l’analyse des données terminée. Prenez tout le temps nécessaire pour prendre la décision de participer à cette évaluation. Si vous avez des questions ou des préoccupations au sujet de l’évaluation, que ce soit avant ou après la signature du présent formulaire, vous pouvez en aviser l’enquêteur principal de l’évaluation dont les coordonnées figurent ci-dessous. Pour toute question relative à l’évaluation, y compris les plaintes ou les questions concernant vos droits en tant que participant à l’évaluation, vous pouvez appeler l’Enquêteur de terrain(chercheur) sur le site d’évaluation : Mapatano Mala Ali. Téléphone : +243 09 9894 7313 Si vous avez des questions, n’hésitez pas à les poser à moi-même ou à un autre membre de l’équipe d’évaluation. Accepteriez-vous de participer à l’évaluation ? 1. Oui 2. Non Accepteriez-vous que nous enregistrions la conversation ? 1. Oui 2. Non Signature de la personne chargée de groupe de discussion: __________________Date : ____/_____/2019 Nom de la personne chargée de groupe de discussion : _____________________________________