Process Evaluation of the FIOVANA Resilience Food Security Activity in Madagascar September 2022 IMPEL | Implementer-Led Evaluation & Learning Associate Award ABOUT IMPEL The Implementer-Led Evaluation & Learning Associate Award (IMPEL) works to improve the design and implementation of Bureau for Humanitarian Assistance (BHA)-funded resilience food security activities (RFSAs) through implementer-led evaluations and knowledge sharing. Funded by the USAID Bureau for Humanitarian Assistance (BHA), IMPEL will gather information and knowledge in order to measure performance of RFSAs, strengthen accountability, and improve guidance and policy. This information will help the food security community of practice and USAID to design projects and modify existing projects in ways that bolster performance, efficiency, and effectiveness. IMPEL is a seven-year activity (2019–2026) implemented by Save the Children (lead), TANGO International, Tulane University, Causal Design, Innovations for Poverty Action, and International Food Policy Research Institute. RECOMMENDED CITATION IMPEL. (2022). Process Evaluation of the FIOVANA Resilience Food Security Activity in Madagascar. Washington, DC: The Implementer-Led Evaluation & Learning Associate Award. PHOTO CREDITS Charlie Forgham-Bailey / Save the Children DISCLAIMER This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the Implementer-Led Evaluation & Learning (IMPEL) award and do not necessarily reflect the views of USAID or the United States Government. CONTACT INFORMATION IMPEL Associate Award c/o Save the Children 899 North Capitol Street NE, Suite #900 Washington, DC 20002 fsnnetwork.org/IMPEL IMPEL@savechildren.org PREPARED BY: Process Evaluation of the FIOVANA RFSA in Madagascar Acknowledgments i ACKNOWLEDGMENTS The Causal Design research team would like to recognize the assistance of key individuals instrumental in facilitating the data collection, research, and findings central to this report, including Cara Raboanarielina and Daniel Rooney with Catholic Relief Services, and Mohamoud Ahmed, Corena Sharp, and Tom Spangler with IMPEL. Special thanks also to Mara Mordini, Justin Mupeyiwa, and Adam Trowbridge with USAID for their technical support during the inception phase. We also are grateful to Eliane Ralison and the field team and their tremendous work with data collection under difficult circumstances. IMPEL | Implementer-Led Evaluation and Learning ii Table of Contents TABLE OF CONTENTS Acknowledgments...............................................................................................i List of Tables......................................................................................................iii List of Figures.....................................................................................................iii Acronyms...........................................................................................................iv Executive Summary ...........................................................................................vi Methods ................................................................................................................................................ vi Findings.................................................................................................................................................. vi Cross-Cutting Findings................................................................................................................... vi Intervention-Specific Findings...................................................................................................... vii 1. Introduction...................................................................................................1 2. Methodology .................................................................................................2 2.1. Intervention Selection....................................................................................................................2 2.2. Primary Data...................................................................................................................................2 2.2.1. Key Informant Interviews..............................................................................................3 2.2.2. Validation Exercise........................................................................................................3 2.2.3. Ground-Truthing Sessions.............................................................................................3 2.2.4. Collaborating, Learning, and Adapting Workshop........................................................3 2.3. Secondary Data ..............................................................................................................................3 2.4. Data Analysis..................................................................................................................................4 2.5. Challenges and Limitations ............................................................................................................4 3. Findings .........................................................................................................5 3.1. Cross-Cutting, Contextual Findings................................................................................................5 3.1.1. Cyclones........................................................................................................................5 3.1.2. COVID-19.......................................................................................................................5 3.1.3. Gender Inclusion ...........................................................................................................6 3.1.4. Youth Inclusion..............................................................................................................7 3.1.5. People with Disabilities Inclusion..................................................................................9 3.2. Purpose 1: Health and Nutrition Status.......................................................................................10 3.2.1. Mother-to-Mother Groups .........................................................................................10 3.2.2. Cooking Demonstrations & Tsikonina.........................................................................13 3.2.3. School-Based Approach ..............................................................................................17 3.3. Purpose 2: Agriculture and Livelihoods........................................................................................19 3.3.1. Farmer Field Schools...................................................................................................19 3.3.2. Farmer Market Schools...............................................................................................24 3.3.3. Impact Investment Fund.............................................................................................25 Process Evaluation of the FIOVANA RFSA in Madagascar Table of Contents iii 3.4. Purpose 3: Enhanced Social and Ecological Risk Management ...................................................26 3.4.1. Locally Managed Marine Areas...................................................................................26 3.4.2. Disaster Risk Management Committees.....................................................................27 4. Conclusions and Recommendations.............................................................31 Annex A: FIOVANA Interventions......................................................................34 Annex B: Evaluation Questions and Methodology.............................................40 Annex C: Data Collection Tools .........................................................................48 LIST OF TABLES Table 1. Findings rubric (with illustrative findings reflective of cell)..........................................................31 Table 2. ADRA District and commune staff respondents Interviewed .......................................................42 Table 3. Fokontany-level respondents interviewed ...................................................................................43 Table 4. ADRA-I, ADRA country staff and implementing partner respondents interviewed......................44 Table 5. Validation exercises conducted per intervention .........................................................................45 LIST OF FIGURES Figure 1. Selected interventions and their Purposes..................................................................................41 IMPEL | Implementer-Led Evaluation and Learning iv Acronyms ACRONYMS AC Community Agent ADCOM Administrative Committee of the Adventist Development and Relief Agency ADRA Adventist Development and Relief Agency Ar Malagasy Ariary BHA USAID Bureau for Humanitarian Assistance CA Agent Communitaire/Community Animator (Coordinator) CAQDAS Computer-Assisted Qualitative Data Analysis Software CD/T Cooking Demonstrations and Tsikonina CHV Community Health Volunteer CISCO Circonscription Scolaire COSANS Comité de Santé/Local Food and Nutrition Security Committee CM Community Mother CNV Community Nutrition Volunteer CRS Catholic Relief Services CSB Centre Santé de Bases/Basic Health Center CU5 Children Under 5 DREN Direction Régionale de l’Éducation Nationale DRMC Disaster Risk Management Committee FC Fokontany Chief FFS Farmer Field School FMS Farmer Market School IGA/AGR Income Generating Activities IIF Impact Investment Fund IMPEL Implementer-Led Evaluation and Learning Associate Award IP Implementing Partner KII Key Informant Interview LF Lead Farmer LM Lead Mother LMMA Locally Managed Marine Area M&E Monitoring & Evaluation MM Model Mother M2M Mother-to-Mother Care Group PE Process Evaluation PWD Person With Disabilities Process Evaluation of the FIOVANA RFSA in Madagascar Acronyms v RFSA Resilience Food Security Activity SBA School Based Approach SME Small-to-Medium Enterprise TIV Teacher Influencer Volunteer WASH Water, Sanitation, and Hygiene WEG Women’s Empowerment Groups IMPEL | Implementer-Led Evaluation and Learning vi Executive Summary EXECUTIVE SUMMARY This Process Evaluation (PE) of the FIOVANA Resilience Food Security Activity (RFSA) that is being implemented by Adventist Development and Relief Agency (ADRA) in southern Madagascar examines and assesses the nature, quality, and process of FIOVANA implementation. Its core overarching evaluation question is: How well has FIOVANA been effectively implementing core and critical pieces of its intervention in order to address the RFSA objectives of enhanced food security and resilience? It’s intended to assist the United States Agency for International Development (USAID) Bureau for Humanitarian Assistance (BHA) and ADRA gauge the state of implementation, contend with identified challenges, and facilitate adaptive management as all parties maximize effectiveness. Methods This report focuses on eight key interventions spanning FIOVANA’s three purposes: Mother-to-Mother Groups (M2Ms), Cooking Demonstrations and Tsikonina (CD/T), School-Based Approach (SBAs), Farmer Field Schools (FFS), Farmer Market Schools (FMS), Impact Investment Fund (IIF), Locally Managed Marine Areas (LMMAs), and Disaster Risk Management Committees (DRMCs). It relies on three core sources of primary data: 1) Key informant interviews (KIIs) with implementing partners (IPs) throughout the implementation chain; 2) Validation exercises based on observation and informal interviews with participants; and 3) Ground-truthing sessions with ADRA technical staff, to garner feedback, questions, and additional relevant contextual information. In addition, the findings and recommendations of this report were informed by a joint Collaborating, Learning, and Adapting (CLA) workshop. Qualitative data was coded and analyzed using ATLAS.ti computer-assisted qualitative data analysis software (CAQDAS). The PE best represents a case study methodology in which one-time data collection was utilized to examine the implementation process and assess how intervening variables, including local context and conditions, influenced implementation to date. Findings are specific to FIOVANA, though some observations may indeed be relevant to RFSA implementation across contexts. Findings Cross-Cutting Findings Cyclone. By all accounts, the cyclones have had a devastating impact on RFSA-served communities and have delayed interventions by varying degrees over the past few months. The most acute impacts were the destruction of crops, FFS demonstration plots, and houses/buildings, all of which have significantly impacted the implementation of FFS, M2M, and CD/T interventions. The widespread destruction of crops pushed most participants and community members into purely survival activities due to a lack of food and/or income. This change in operating context (i.e., emergency conditions) had significant impacts on intervention implementation and participant motivation to participate in interventions during the period of emergency due to the cyclones. COVID-19. Overall, COVID-19 appears to have had a significant impact on the establishment of FIOVANA interventions. Respondents at the fokontany level suggested that there were only some slight Process Evaluation of the FIOVANA RFSA in Madagascar Executive Summary vii disruptions to ongoing (i.e., already established) group activities. However, animators (coordinators), supervisors, and country-level staff noted that restrictions on movement hindered sensitization and training, thereby delaying the establishment of some interventions. In areas where interventions were able to be established, respondents noted delay or disruption of meetings due to fear of gathering and/or large group restrictions. However, these disruptions appear short-lived, and group leaders were able to successfully adapt to the situation by gathering in smaller groups. Gender Inclusion. The means and extent of gender inclusion varies across interventions, given their nature and focus, with the exception of IIF, which appeared to have no mechanism accounting for gender. With some exceptions, Purpose 1 interventions appear to have little meaningful male involvement.1 Across interventions, strong gendered divisions of labor affected both types of participation in interventions. Men often assumed more physical roles, and, in some cases, the divisions affected women’s ability to conduct or participate in interventions (i.e., women were unable to speak up in public forums, conduct household visits, or attend DRMC meetings because of their domestic responsibilities). Youth Inclusion. Youth inclusion also varies across interventions, except for IIF and SBA interventions, which appear to have no specific mechanism accounting for youth. The number of youth involved in M2M, CD/T, FFS, FMS, and DRMC interventions varies by location, though where youth were present, they were perceived to be dynamic and active. A challenge in reaching youth was youths’ need to prioritize income-generating activities (IGAs), which reduced their available time to participate in FIOVANA interventions. People with Disabilities (PWD) Inclusion. Overall, there was little concrete information pertaining to the involvement of PWDs in FIOVANA interventions. A number of respondents, including Lead Mothers (LMs) and Model Mothers (MMs), Community Health Volunteers (CHVs), SBA respondents, and Community Agents (ACs), demonstrated an awareness of PWD rights and commented on the importance of PWD inclusion. A few respondents noted adaptations specific to allowing PWD participation. For example, some FFS and CD/T respondents noted that PWDs participated indirectly via their relatives. Recommendations: ● Consider providing additional training on inclusion for community workers led by gender, youth, and PWD sensitivity experts to enhance meaningful, inclusive participation and combat persisting stereotypes. Trainings should also include practical instruction on successful inclusive implementation. Intervention-Specific Findings Mother-to-Mother. There was some confusion around M2M findings due to LMs’ and MMs’ overlapping responsibilities with various groups. However, the M2M intervention’s positive elements include training materials, which were reported to be useful; food aid as a powerful motivator to encourage mothers to attend under-5 weighing sessions; anecdotal evidence of positive behavior change related to 1 Note that Male Leader Groups (MLGs), which were not selected as an intervention to be assessed by this PE, are the main avenue in which ADRA engages men in P1. IMPEL | Implementer-Led Evaluation and Learning viii Executive Summary Water Sanitation and Hygiene (WASH); and gender equality. However, other respondents recalled limited behavior change due to communities’ limited resources, limited time availability for M2M interventions, and a desire for more material and animator support to encourage M2M groups. Recommendations: ● Improve clarity among M2Ms, Women’s Empowerment Groups (WEGs), CD/Ts, and other interventions for LMs and other implementers involved. ● Clarify roles and responsibilities with group leaders via trainings and/or capacity building (formal or informal) to ensure that they understand specific intervention activities and goals. Cooking Demonstrations and Tsikonina. Though under-5 weighing sessions occur on a regular basis, the cyclone posed a major challenge for conducting cooking sessions due to crop destruction and reduced participant availability. Successes reported included positive recollections of trainings. Recommendations: ● Consider providing additional cash or in-kind ingredient support for CD/Ts, particularly during the cyclone recovery period, to ensure community members are able to participate in sessions. ● Consider reducing the number of days for Tsikonina sessions, given women’s considerable time constraints. Alternatively, conduct further assessment of ways to ensure that women are willing and able to participate in sessions.2 ● Provide additional leaflets and other informational materials for participants to take home as there appears to be high interest in these. ● To manage expectations and prevent potential mistrust, increase communication and clarity on distribution, including items, timelines, targeting criteria, and delays. ● To increase male participation in CD/T and reinforce gender equality, have men also lead cooking demonstrations.3 School-Based Approach. SBA interventions were at very early stages at the time of data collection, with many respondents noting that they had received awareness raising about SBA but had not yet received training on implementation. SBA leads had nonetheless selected land for school gardens and set up school clubs, though they had not initiated any work yet. Most respondents discussed school gardens but demonstrated little awareness of other aspects of the SBA interventions, such as WASH, which might reflect the early stages of the SBA intervention implementation, but could also suggest that agricultural interventions are perceived as most relevant. Recommendations: ● Adapt trainings and training materials to be more accessible to participants, including providing hard copies of information materials. Farmer Field Schools. FFS are established, and FFS interventions/responsibilities are well understood by Lead Farmers (LFs). However, the pace of implementation and uptake varies significantly between 2 During the Joint PE CLA workshop with ADRA and CRS conducted in Antananarivo from June 12–16, the P1 team noted that a reduction in attendance days for the Cooking Demonstrations was not recommended. Instead, they suggested better understanding the participation challenge and finding a middle ground for participants. 3 This recommendation came from ADRA P1 team members during the Joint PE CLA workshop. ADRA proposed that men lead cooking demonstrations and include this action as part of the MLG scorecard. Process Evaluation of the FIOVANA RFSA in Madagascar Executive Summary ix regions and specific fokontany due to uneven provision of trainings, materials, and seeds. Broadly, LFs and FFS group members appear to have received training and are applying improved techniques (specifically pesticides, compost, weeding, and planting). However, most LFs indicated that a significant challenge to their activities is seed and tool delivery delays, which have caused planting/harvesting setbacks and have been a source of demotivation. The cyclones had an acute impact on FFS implementation. In most cases, the cyclones completely destroyed crops on FFS demonstration plots and caused FFS interventions to be suspended due to reconstruction and recovery. Recommendations: ● Ensure timely seed and tool distribution to all FFS groups to 1) ensure that members can implement improved techniques and plant according to seasonality and 2) keep LF and FFS members motivated and engaged. ● Consider conducting an assessment of damages to FFS demonstration plots to identify areas in which participants need support for land identification or plot reconstruction. ● Consider following up with supervisors and animators in areas in which training and/or follow￾up has been reported to be missing or incomplete to assess barriers to delivery. ● Consider following up internally and/or with group leaders concerning issues related to member registration/participation and code assignment. Impact Investment Fund. Prospective business co-investment partners had been visited by ADRA to discuss the IIF co-investment. However, most (3 of 4) respondents suggested that there had been little to no follow-up after the initial visit and that the process by which to apply/register for the IIF co￾investment was unclear or had not progressed. IIF specialists suggested that the process for business co￾investment, selection, and approval was very lengthy due to internal ADRA processes and, as such, few approvals and disbursements have been executed. That said, IIF specialists also noted that processes were being improved and the implementation of the due diligence process has helped speed up Small￾to-Medium Enterprise (SME) selection and validation. Recommendations: ● Continue to streamline the SME selection and validation process internally by identifying and clearing bottlenecks in the IIF approval process. ● Improve prospective SME sensitization and follow-up, and consider utilizing animators to provide follow-ups and maintain communication. Locally Managed Marine Areas. There is insufficient data to assess LMMAs at this time due to their only nascent implementation status. Nevertheless, it appears some steps have been taken in the process of setting up LMMAs. For example, the P3 team and a respondent from MIHARI noted that a capacity building session for field agents took place and was followed by actions to collect information in the field for proper preparation of the establishment of the structure. Disaster Risk Management Committee. DRMCs are robustly implemented, with ample evidence of effective training, implementation, and community participation. The key to this appears to be effective community liaising, with—despite the array of potential interests—ADRA’s deft navigation of formal and informal stakeholders, including local authorities. ADRA, inherent challenges aside, has successfully IMPEL | Implementer-Led Evaluation and Learning x Executive Summary collaborated with local officials to establish DRMCs that are broadly inclusive, with a demonstrable positive impact on disaster preparation and recovery. The greatest obstacle to yet more effective participation, and possibly sustainability, is the regularly cited delays in compensation for participants of the training. Recommendations: ● Review the logistics involved in providing a code and compensating participants for trainings, as delays in compensation may erode trust and goodwill. Cross-Intervention Recommendations: ● Consider what incentive options are possible to motivate community workers and participants, given that motivation and time availability appears to be the greatest challenge to implementation quality and, consequently, impact. Also, increase communications among communities about the nature of community workers’ role and volunteer nature to avoid resentment or misperceptions. Process Evaluation of the FIOVANA RFSA in Madagascar Introduction 1 1. INTRODUCTION This Process Evaluation (PE) of the FIOVANA Resilience Food Security Activity (RFSA) that is being implemented by Adventist Development and Relief Agency (ADRA) in southern Madagascar is carried out under the United States Agency for International Development (USAID) Bureau for Humanitarian Assistance (BHA)-funded Implementer-Led Evaluation and Learning (IMPEL) Associate Award. As a PE, this report specifically examines and assesses the nature, quality, and process of FIOVANA implementation through April of 2022. It is designed and was conducted to answer the overarching evaluation question: How well has FIOVANA been effectively implementing core and critical pieces of its intervention in order to address the RFSA objectives of enhanced food security and resilience? This PE is intended to assist USAID/BHA and ADRA gauge the state of implementation, contend with identified challenges, and facilitate adaptive management as all parties maximize effectiveness. It also provides contextual information to inform FIOVANA’s endline impact evaluation. More precisely, this report gauges and assesses the systematic strengths and weaknesses of the implementation strategy and processes, as well as contextual factors that directly affect implementation. The most visible and relevant contextual factors that are examined intentionally include how COVID-19 and, separately, the recent cyclones have affected implementation. The evaluation also examines FIOVANA’s inclusivity approach regarding gender, youth, and people with disabilities (PWDs) and, more granularly, how implementation interacted with local governance structures and stakeholders, like fokontany chiefs (FCs), elders, and other formal structures. This report primarily relies on qualitative data from key informant interviews (KIIs) with personnel and stakeholders throughout the implementation chain. Each intervention also includes findings from a validation component, carried out intentionally to gauge the state of implementation and identify areas of special concern regarding implementation follow-through. This report also benefits from feedback and contextual information gathered through “ground-truthing” sessions held with ADRA and a Collaborating, Learning, and Adaptation (CLA) workshop attended by Causal Design, Catholic Relief Services (CRS), ADRA, USAID/Madagascar Mission, and BHA staff. The report concludes by providing recommendations to improve FIOVANA’s implementation in the future. Causal Design acknowledges that the ADRA team has persevered in the implementation of FIOVANA despite extreme setbacks, including the effects of a series of devastating cyclones in most activity areas and COVID-19 restrictions, which limited travel and communication. We, nevertheless, hope that our observations and recommendations are taken seriously by the implementing and relevant technical teams to augment learning and adaptive program management, as we believe the data collection and subsequent analysis yield important insights for all parties that can inform next steps. The report concludes by providing recommendations to improve FIOVANA’s implementation for the future, based on evidence discerned through careful data collection and analysis. IMPEL | Implementer-Led Evaluation and Learning 2 Methodology 2. METHODOLOGY The FIOVANA RFSA PE is based on the analysis of primary qualitative data (discussed more below) and predominantly constitutes a case study methodology with findings specific to FIOVANA and its current operating context. Results are neither generalizable nor address causality or outcome attribution, though some lessons learned and recommendations may indeed be relevant for other RFSAs being implemented in difficult environments. Data collection was conducted at one point in time (or over a month more precisely) to retrospectively examine implementation to date. Data collection instruments were designed to reveal the steps involved in implementation, identify moments and factors that intervened in the process, as well as adjustments or notable adaptations by implementers, if applicable. The sampling of diverse respondents allowed for a validation and robustness check on findings. Though a frequency of responses may be indicative of trends, patterns, and findings, in some instances, individual responses are noted if they are particularly illustrative or uniquely articulate of ground-level challenges. In the same vein, while acknowledging inherent limitations, this report leverages the advantages that extensive qualitative data presents. It not only presents a static set of observations but also references the complexity confronting implementing partners (IPs) operating in a crisis environment. Furthermore, it references the challenges that arise when implementing a mostly linear activity within a complex system constituted by diverse stakeholders with variable incentives and assesses language and responses indicative of assumptions, sentiments, and even grievances that might arise from the presence of outside intervention. 2.1. Intervention Selection Causal Design consulted with BHA Washington, the USAID/Madagascar Mission, and ADRA to determine a select set of priority interventions—among the universe of overlapping interventions—to prioritize for examination under this PE. The seven selected interventions, selected based on their representativeness of and significance in the FIOVANA RFSA as a whole are: ● Purpose 1: 1) Mother-to-Mother (M2M), 2) Cooking Demonstrations and Tsikonina (CD/T) ● Purpose 2: 2) Care Groups (CGs) ● Purpose 3: 3) Farmer Field Schools (FFS), 4) Farmer Market School (FMS), 5) Impact Investment Fund (IIF) ● Cross-cutting purposes: 7) School-Based Approach (SBA) 2.2. Primary Data This PE is based on qualitative inquiry. As it focuses on the implementation process, it necessitates examining of the links throughout the implementation “chain.” This was achieved through KIIs and a validation exercise. In addition, three ground-truthing sessions with ADRA staff were conducted for additional detail and clarification. Each component is detailed further below. A full list of respondents interviewed, validation exercises, and geographic sampling is presented in Annex B, and the tools used for data collection are presented in Annex C. Process Evaluation of the FIOVANA RFSA in Madagascar Methodology 3 2.2.1. Key Informant Interviews The main sources of data for this PE are KIIs with stakeholders across the implementation chain, from national-level ADRA and IPs to ground-level implementers such as community workers. Geographic sampling of community-level actors and participants covered communes and fokontany in all six regions targeted by FIOVANA, purposively selected to include both interior and littoral areas, and both rural and more urban areas. Causal Design staff conducted interviews with ADRA and IPs remotely over Zoom using simultaneous translation as necessary. As Causal Design staff were unable to travel to Madagascar due to the COVID-19-related travel restrictions, data collection at the fokontany and commune level was conducted by Madagascar-based Consultants Associates4 and supervised remotely. 2.2.2. Validation Exercise At BHA’s request, Consultants Associates also conducted a validation exercise to provide additional verification that interventions are taking place according to FIOVANA’s intended cascading system. The validation exercise, while narrow in scope, nevertheless provided additional qualitative data that built on findings from the KIIs. 2.2.3. Ground-Truthing Sessions Stakeholder feedback on findings and recommendations was solicited through ground-truthing sessions with ADRA technical staff. Four remote ground-truthing sessions, divided by purpose and intervention, were held over Zoom using simultaneous translation on May 24 and May 25, 2022. Sessions were highly fruitful and consisted of Causal Design presenting findings and initial recommendations, then garnering ADRA feedback, questions, and additional contextual information and details. 2.2.4. Collaborating, Learning, and Adapting Workshop At ADRA’s request, Causal Design facilitated a 4-day joint CLA workshop alongside CRS (implementers of the Maharo RFSA). Causal Design, CRS, ADRA, USAID/Madagascar Mission, and BHA staff attended the workshop. Additional context and refinements to recommendations that emerged from the workshop have been incorporated into this PE report. 2.3. Secondary Data ADRA also provided secondary data, including RFSA documents, annual reports, youth and gender strategies, technical documentation about the seven interventions, and studies conducted as part of the Refine and Implement year. Reviewing this secondary data, in combination with follow-up clarifications with FIOVANA technical staff, informed the tool design and construction of a preliminary codebook by identifying key themes anticipated to emerge in primary field data. 4 Consultants Associates conducted the FIOVANA Baseline Study and, as such, were familiar with the FIOVANA RFSA and context. IMPEL | Implementer-Led Evaluation and Learning 4 Methodology 2.4. Data Analysis Primary data, in the form of English language transcripts, was coded and analyzed using ATLAS.ti, Causal Design’s preferred CAQDAS program. Primary data was coded using both deductive and inductive approaches, whereby a codebook developed based on themes from secondary data was applied to the primary data and continuously modified to incorporate emergent themes. Regular communication among analysts in how codes were applied was key to ensuring inter-coder reliability and minimizing error. 2.5. Challenges and Limitations Travel restrictions due to COVID-19 prevented Causal Design staff from being able to travel to Madagascar for data collection, directly supervise field teams, and conduct enumerator training. In lieu, Causal Design conducted a training-of-trainers over 3 days and was in regular communication with the enumeration team over the course of training and data collection. This allowed both parties to ask questions and clarify any points of confusion, including responses, to maximize data quality. Language barriers also posed a challenge, as they necessitated the use of translators for remote interviews. Consequently, it increased the risk that some clarity, nuance, and details may have been lost in translation, though considerable steps were undertaken during enumerator training and during fieldwork supervision to minimize negative outcomes in this regard. Finally, given the scope and intention of the PE, it must be emphasized that this PE does not assess impact, effectiveness, or efficiency in any formal capacity, as its focus centers entirely on implementation. Furthermore, SBA, FMS, and LMMA interventions appeared to be at very early stages, thus making an assessment of implementation difficult. Process Evaluation of the FIOVANA RFSA in Madagascar Findings 5 3. FINDINGS This section presents the main findings from the PE. Cross-cutting findings affecting multiple interventions are presented first. These cross-cutting findings are related to contextual factors, in particular, cyclones, COVID-19, local governance in FIOVANA areas, and the inclusion of all genders, youth, and PWD in FIOVANA interventions. Findings by intervention and purpose are then presented, including intervention-specific findings on the cyclones, COVID-19, local governance, and inclusivity. As a qualitative analysis, findings are relayed using subjective language. Findings are contextualized around the number of respondents reporting a certain view with outliers noted. Quotations are sometimes provided, not in lieu of evidence and analysis but as a means to illustrate relevant phenomena or important observations. Quotations are attributed to respondents’ roles or organization, and community workers are identified by only their commune to preserve anonymity. 3.1. Cross-Cutting, Contextual Findings 3.1.1. Cyclones The kere or drought, while not an unexpected or exceptional circumstance given its perception by local communities as a recurring seasonal state of continuous hunger, was nonetheless noted to be particularly severe and posed a challenge to implementation in all areas sampled. The LF intervention was the most directly affected, though respondents from all interventions noted the kere/drought’s effect on participant time availability and motivation, as participants redirected their energy towards finding food or income generation. “Then there are the members because they are also miserable. They work for just an hour, and they get tired from hunger. They don't have enough food to eat, and their food is poor; they eat cassava powder.” (Male LF, Manakara) “The search for food, because if there is food, you can work properly. When there is not enough food, you cannot work even if you work diligently. It is limited because you don’t have enough energy to work.” (Male LF, Mananjary) 3.1.2. COVID-19 In most locations, COVID-19 appeared to have little impact on established FIOVANA interventions, and there was little awareness of COVID-19 in many rural areas. Respondents in Farafangana, Manakara, and Mananjary, in particular, noted that COVID-19 was not a major concern and did not affect ongoing interventions much. In places where COVID-19 was present or restricted group activity, group leaders either postponed meetings or activities until the restriction was lifted or were able to adapt and continue activities in smaller groups. IMPEL | Implementer-Led Evaluation and Learning 6 Findings Animators (coordinators), district supervisors, and country-level staff suggested that COVID-19 related movement and gathering restrictions had a significant effect on their ability to conduct training and gather groups. In many cases, this delayed the implementation or establishment of interventions in fokontany, as large gatherings to sensitize and establish groups were temporarily prohibited. Further, restrictions on movement limited the ability of ADRA to conduct staff trainings and travel to fokontany to sensitized and establish interventions. In a few areas, namely in Vohipeno, Vondrozo, and Vangaindrano, respondents noted that COVID-19 had disrupted, delayed, or suspended some interventions, including meetings and trainings. For example, one female Purpose 1 Community Animator (CA) from Vohipeno noted that some people were afraid of attending food distributions or going to markets to buy ingredients for CD/Ts. As they explained, “When we share food, many don't come because the food distributor is a stranger, so people are worried that the person is sick. I convince them that this person who comes to us has already been tested and is not sick. People get scared when they hear the death toll on the radio.” These COVID-19 challenges appear short-lived, and group leaders were to be able to find ways to conduct interventions while taking into account COVID-19 protocols. Animators and group leaders in these regions suggested that meetings were postponed for a couple weeks, but resumed either outside or inside while adhering to protocols. Moreover, group leaders adjusted schedules to do smaller group meetings, as one male LF in Mananjary detailed: “The measure we took at that time was to let only a few people come to do the activities at the school garden. For example, five members work in the morning; then, still, five members come in the afternoon. In other words, the number of people doing the activities was limited.” 3.1.3. Gender Inclusion The means and extent of gender inclusion varied per intervention examined, given its nature and focus. For IIF, there appeared to be no mechanism accounting for gender in Small-to-Medium Enterprise (SME) selection. For SBA, FFS, FMS, and DRMC interventions, women and girls were included to varying degrees, which often differed by location: ● M2M & CD/T: These interventions were largely targeted to women, with men generally showing little interest, with respondents commenting that it was difficult to sensitize men on these issues. However, a few male Community Health Volunteers (CHVs) did note that they made efforts to sensitize men, though they also suggested a need for further training on how to sensitize men, and a need for specific sensitizations for men only. One male CHV from Mananjary, recalled: “Sometimes, I try to talk discreetly with the men around my house. I ask these men their activities, from getting up in the morning until they have their coffee [when men drink coffee together in the morning]. Then, I also ask them what their wife accomplishes at this very moment. It is confirmed that the tasks of women are much more than those of men. This is how I can convince men.” Process Evaluation of the FIOVANA RFSA in Madagascar Findings 7 ● SBA: Men and women are included as teachers and parents to run school clubs, and both boys and girls participate in school clubs. ● FFS/FMS: Women’s participation varied, with some locations having more and others having fewer female members. ● DRMC: Women were represented in DRMCs, having been selected purposively, though the number of women and men varied. A few respondents indicated that fewer women participated in DRMCs because of their home responsibilities. One male FC from Vondrozo explained, “What differentiates young men from young women is that there is more dynamism among men. This is not to say that women are less dynamic, but they are more busy with household chores and divide their time between the family and the committee. We see, for example, during our meetings that women are often late.” Traditional gendered divisions of labor are evident across locations, including within FIOVANA interventions, which in some cases impeded equitable participation: ● M2M & CD/T: Respondents recalled that men may be involved in physical activities such as latrine construction, agricultural work, digging garbage pits, and carrying firewood for CD/Ts. However, respondents reported that men largely do not participate in sensitization as the topics were perceived to be only relevant to women. Furthermore, a few respondents noted that men’s participation and enthusiasm for assisting with WASH and CD/T interventions varied, making it necessary to resort to hiring laborers in some cases. ● SBA & FFS/FMS: Agricultural activities followed traditional gender norms and divisions of labor, with respondents citing women’s lack of strength and inability to conduct certain tasks. One male School Director from Vangaindrano noted that he planned school garden clubs to follow traditional gender lines: “My personal opinion, regarding the club, the participation of boys or girls depends on the type of activity. If it is an activity that uses force, boys are more active, such as carrying garden borders; if it is a less physical activity, women are more active.” ● DRMC: DRMC respondents reported that men typically took responsibility for more physically demanding tasks such as construction, while women were responsible for cleanup activities and sensitization in the village. A few respondents noted that women’s inability to speak up in public forums, including DRMC meetings and parents’ meetings, remained a challenge. As one male FC from Vondrozo recalled, “In terms of debate, it is often difficult to get the ideas of women, although we notice that when they give opinions, they are always brilliant ideas. And finally, women often ask questions during a meeting.” 3.1.4. Youth Inclusion The inclusion of youth varied per intervention examined. For IIF, there appeared to be no mechanism accounting for youth in SME selection, while SBA interventions, also at early stages, similarly did not appear to focus on youth inclusion, given their focus on children. For M2M, CD/T, FFS, and FMS interventions, the number of youth involved varied by location, with some reporting high youth membership or participation, while others had little or even none. IMPEL | Implementer-Led Evaluation and Learning 8 Findings ● Across M2M & CD/T: Reasons for having few youth members, mentioned by several respondents, include the fact that young people were less likely to be married and more likely to use contraception. However, one male CHV from Manakara noted that young mothers were more likely to have malnourished children as they were more likely to be single parents and rely on their grandparents to care for the children: “Some were still very young when they got pregnant while being single, and, as a result, the child's grandparents had to take care of the child; the number of people those grandparents had to be in charge of increased, so, if they couldn't really afford it, the child's diet became unbalanced.” ● FFS & FMS: Reflecting on why some FFS and FMS groups showed little youth participation, many respondents indicated that youth lack the motivation to participate in agricultural interventions and suggested a need for more sensitization and encouragement for youth to join. Nonetheless, a separate group for youth farmers was formed in Vangaindrano, and one FFS/FMS group was run by a young person. ● DRMC: Youth were purposely included when selecting committee members. One male FC from Vangaindrano noted that having youth on the board helped convince other youth to listen to their messages. Across interventions, where present, youth were also noted to be active, outspoken, and dynamic. Youth generally take responsibility for physical activities, such as hard agricultural labor, fetching water and firewood, and putting up posters. Some CD/T respondents perceived younger mothers to be more receptive to cooking lessons, with one male Community Nutrition Volunteer (CNV) from Farafangana noting that all MMs were young. However, a few respondents, including youth in interventions, noted obstacles due to youth’s need to prioritize income generation. One female MM from Vondrozo noted that young people had little time during the day as they worked jobs such as mining; consequently, she resorted to holding meetings early in the morning. A male FC from Vohipeno noted that, though youth were outspoken in DRMC meetings when they were present, youth were sometimes not able to attend DRMC meetings as they were not paid. He explained: “Well, young people still live with their parents. In other words, the young members of the committee are still under their parents' responsibility. So, when there is a meeting, they still have to ask their parents for permission. Their parents tell them to not attend the meeting since it doesn't make them earn money. That's why young people are often absent during the meetings. They're unlike the elders who are members of the association. When we want to attend a meeting, no one, not even our wife, can stop us.” Process Evaluation of the FIOVANA RFSA in Madagascar Findings 9 3.1.5. People with Disabilities Inclusion Overall, there was little concrete information pertaining to the involvement of PWDs in FIOVANA interventions. A number of respondents, including LMs and MMs, CHVs, SBA respondents, and CAs, showed an awareness of PWD rights and commented on the importance of PWD inclusion. A few respondents noted adaptations specific to allowing PWD participation. For example, some FFS and CD/T respondents noted that PWDs participated indirectly via their relatives. A few CHVs said that they accommodate PWDs by visiting them in their households when they could not attend events such as cooking demonstrations. In a few cases, respondents recounted ‘success stories’ of PWD participation, though their ability to participate hinged on the severity of their disability. Two such success stories are below: “There is one woman [with disabilities on the committee]… She is the same. Her disability does not prevent her from working; she just has a small deformity on her mouth. She is a youth representative, and when there is a campaign in the area, she is in charge of it. She lives there ... In any case, she does the job well.” (Male FC, Manakara) “[PWD FFS member] is motivated to do the activities, and he is not really disabled, some of his toes are missing, he is able to perform well the activities… Just like his fellow men, he also labors the soil.” (Male LF, Mananjary) However, overall, there appeared to be few PWDs involved in FIOVANA interventions, with many respondents across interventions noting no PWDs were involved. In some cases, this was cited to be due to a lack of PWDs in their communities. For IIF, there appeared to be no mechanism accounting for PWDs in SME selection. PWD inclusion was seen to be a particular challenge for agricultural interventions. While some FMS and FFS respondents said they would welcome PWDs in the group if they wanted to join, and some suggested ways PWDs could become involved in certain tasks, a few showed more resistance to including PWDs in agricultural interventions. As one male LF from Vondrozo stated, “It is very difficult for people with disabilities; they should not be involved in the organization because it is for work.” One male LF from Farafangana noted a need for PWD-adapted tools such as spades to allow PWDs to participate. Similarly, some SBA participants perceived physical disabilities as a potential obstacle for PWDs to assist with school gardens. Additional challenges for PWD inclusion mentioned by respondents included a need for more direct support for PWDs so they could join interventions, as voiced by a female MM from Vangaindrano, and speculatively, difficulty sensitizing those with non-mobility related disabilities (e.g., deafness, mental disabilities) as mentioned by a female MM from Vondrozo. Responses about PWD inclusion also illustrate that disability is largely understood to refer to physical disabilities, with mental disabilities little considered. IMPEL | Implementer-Led Evaluation and Learning 10 Findings 3.2. Purpose 1: Health and Nutrition Status Purpose 1: Sustained improvement in health and nutritional status of women of reproductive age, adolescent girls, and children under 5 years. 3.2.1. Mother-to-Mother Groups Assessing M2M interventions prove complicated, as many LMs and MMs interviewed revealed some confusion between their respective roles and activities. Many respondents were also involved in other groups and associations, such as CD/Ts and WEGs, and many spoke in detail about one particular intervention, including CD/Ts, under-5 weighing sessions, agricultural activities, and sensitizations on topics such as hygiene, domestic violence, and attending prenatal sessions. As such, it is difficult to extrapolate findings specific to the M2M intervention, though what is possible to ascertain is presented here. Validation Findings Validation data indicates that M2M groups have been established and conducted interventions in all locations surveyed, with two exceptions. In Mananjary, it was not possible to conduct a verification exercise. In Vondrozo (specifically Vohimary commune, fokontany Ratsimanahiantefaliandro), a LM had been selected, and names of mothers interested in joining the group have been recorded, but meetings and sensitizations have not begun. Where they are active, M2M groups consist of 15 members and meet twice or three times a month to raise awareness about childcare, child and maternal health and nutrition, and WASH, with the help of picture books LMs have been given. In Vangaindrano, it was also noted that members make contributions of 2000 Ariary each to buy ingredients for CDs and provide social support in case of births, deaths, and illness. Validation findings were important for making sense of KII findings, which indicated some confusion over what constituted training for LMs. While all LMs interviewed stated that they had not received capacity building for the M2M intervention, some nonetheless reported conducting activities, meeting MOTHER-TO-MOTHER GROUPS M2M is comprised of groups of women who come together to discuss issues on Maternal, Infant, and Young Child Feeding Nutrition, including challenges and how they resolve them. They support one another on issues of maternal nutrition and Infant and Young Child Feeding. These mothers are recruited by CHVs based on updated register information from the mapping. The groups constitute 10–15 participants and meet at least twice a month during intervention start-up and once a month after CHV visits. M2M sessions are facilitated by CHVs/CNVs, to discuss a wide range of nutrition and health topics. There are also specialized M2M groups of pregnant adolescents aged 15–19 to ensure adolescent mothers receive adequate peer support and can discuss issues specific to their cohort. Process Evaluation of the FIOVANA RFSA in Madagascar Findings 11 with CAs, and receiving sensitization equipment, such as posters and educational books. It appears that LMs, and possibly other community implementers, do not perceive meetings with CAs as “trainings,” as these are less formal and do not entail compensation for their time. Validation data suggest some initial success with respect to implementation. In Vohipeno, the high number of mothers affiliated with M2M was noted, and, in Manakara, a steady increase in mother membership was indicated. Though unconfirmed beyond these reports, validation data suggest successful uptake of sensitizations. In Manakara and Farafangana, it was noted that many children are now vaccinated as a direct result of M2M sensitizations. Pregnant women in Farafangana and Vohipeno began going to the hospital to receive antenatal care, and some mothers in Vohipeno brought their children to the hospital for healthcare in case of illness. Implementation Successes Given the confusion among participants around what constituted M2M interventions, and the fact that many participants spoke in detail about CD/Ts or weighing sessions, there is relatively little data that clearly points to the successes and challenges of M2M interventions. Some of the suggested successes and challenges noted by a few respondents are below: ● Those who had received training materials, such as books and posters, were positive in their responses and noted that these proved helpful, both in their own daily lives and with sensitization, particularly for illiterate mothers. ● In one case, a female MM from Vangaindrano had been successful in promoting kitchen gardens to provide healthy ingredients and also a source of income for mothers in her community: “I can say that the sensitization I did was achieved as I made effort to encourage the mothers to plant vegetables in order to earn some money. The problem for us is that there is no money to prepare the multicolor food. So I made a kitchen garden for them to see. They imitated it, so the members became eager to plant. Our cultivation grew wider, so we planted not only green vegetables but carrots, potatoes, cabbages also. We looked for markets to sell our products so there is some money and we can buy the other ingredients… The objective is to become a rongatry woman [women who ask for their rights].” ● Food aid is a powerful motivator for mothers to attend weighing sessions. As one female LM from Farafangana recalled: “Mothers actively participate in the CU5 (Children Under 5) weighing session ... They receive food aid based on maize flour; I find that this is what motivates them.” ● Several respondents also noted that they had seen instances of positive behavior change, such as reduced alcoholism, increased gender equality, and uptake of healthy WASH practices. As a female LM from Farafangana recalled, “In my current role, I really like and need this book on violence because nowadays we don't have to take or drink anything like that anymore. Do not touch drugs and alcohol because if you earn 500 Malagasy Ariary (Ar) or 700 Ar, you can buy rice for the children instead of spending on drugs or alcohol. Since visiting this hamlet, I have seen a IMPEL | Implementer-Led Evaluation and Learning 12 Findings change in attitude.” However, the exact cause of these behavior changes or specific FIOVANA interventions linked to these changes was not clear. ● A few respondents also noted spillover effects, where mothers who were not M2M members nonetheless listened in to sensitization meetings. Implementation Challenges Similarly to other interventions, time availability is a challenge, partially due to the cyclone (discussed in the Cyclone section above). One female LM from Farafangana noted that meetings were kept deliberately short as a result and noted that she was unable to conduct home visits due to her own childcaring responsibilities: “I'm so busy I can't make the home visit, I'm being honest. I have a lot of responsibility. I should take care of my family; my grandchild is still young and still needs care. So I prefer when people are together during the meeting, and I sensitize.” Some respondents recalled limited uptake of health and nutrition behaviors due to limited resources and awareness. Two respondents, a female MM from Manakara and a female LM from Farafangana, noted that it was difficult for them to follow up on group discussions and recommendations to grow kitchen gardens due to a lack of funds. One female MM from Mananjary explained in detail: “Talking about food security was very challenging because none of the mothers could afford it ... Actually, despite good awareness sessions, we cannot have a good result. Another challenge is talking about hygiene. We live in the countryside. Diapers are not often changed, and children do not often receive a bath. Of course, there is a slight progress after the teaching. So we have awareness activities, but the cost of living does not really allow mothers to comply with the teachings. At this time of the year, we are struggling to buy a cake of soap. We barely receive help, only some kind of a bar of soap and a liter of oil. Affordability can be a big issue.” A few respondents spoke of a need for more support from animators to maintain the motivation of M2M groups. One female MM from Mananjary lamented that the animator was not present at meetings, which could help with sensitization: “I also wish the animator were present during some meetings to teach or correct us should there be anything we do wrong. Also, it would be helpful because I won’t be the only leader. They already meet me every day. The other mothers will be glad to see we have a technician working with us.” One female MM from Vangaindrano noted that the Community Agent in their area left the fokontany, leading the group to somewhat dissolve: “The reason why our group was broken was our chief’s [animator] departure because she lived in our Fokontany. This group of ours is nothing if there is no leader. Some have not followed the payment of the monthly subscription, so the members left little by little.” Some respondents also voiced a desire for additional inputs and compensation to motivate M2M members to attend meetings and implement behavior change. For example, one MM from Vondrozo said: “Organizing a meeting with the mothers to do sensitization now can be a problem; maybe we should call them two or three times before they all come. They may ask for per diem or allowances for the meeting… Some people of our FKT, when there is a meeting or name registration, complain: ‘Our Process Evaluation of the FIOVANA RFSA in Madagascar Findings 13 names are still written there, but is there something to share or is there money to distribute?’” Some suggested that the lack of compensation created some negative sentiment, with one recalling that mothers in her area suspected her of diverting distributed aid. More specifically, one female LM from Farafangana requested pareos (traditional clothing) to formalize and distinguish the M2M members from other groups. 3.2.2. Cooking Demonstrations & Tsikonina Overall, there appeared to be little traction on the cooking demonstration and sensitization elements of CD/T interventions, other than weighing sessions. These issues were largely linked to limited time availability and access to ingredients, both linked to the cyclone. Validation Findings Validation data confirms that CD groups have been established, and demonstrations have been conducted in all locations surveyed except in the site in Vangaindrano, where CHVs have been trained but are waiting to receive equipment before carrying out interventions. Before the cyclone, CDs occurred when participants had free time, often once or twice a month. Participants contributed 200– 500 Ar for purchasing ingredients for the demonstrations. Validation data indicates that Tsikonina has occurred in Farafangana, Manajara, and Vohipeno of the surveyed districts. It usually lasts 6 days; however, it is sometimes shortened to three consecutive days. Food is prepared by the mothers with the intervention, led by CNVs, CHVs, or animators. The type of food prepared varies across locations and often includes jackfruit, breadfruit, cassava, banana, and rice mixed with other ingredients such as onions, peanuts, salt, oil, tomatoes, and jumbo, which is then fed to the babies. Under-5 weighing sessions occur at least once a month. In Manakara, it occurs on the first COOKING DEMONSTRATIONS AND TSIKONINA Cooking demonstrations contribute to FIOVANA’s goal to sustainably improve the health and nutritional status of women of reproductive age, adolescent girls, and children under 5 years old. Cooking demonstrations are a community-led intervention promoting the use of locally available or accessible food. Cooking Demonstrations aim to assist households in integrating nutritional recommendations into daily food processing and preparation. They provide households with the knowledge, skills, and confidence to better utilize locally available and affordable foods to improve their diet and nutritional status. They also impart the basics of food safety and hygiene, such as hand washing, covering prepared food, and proper handling and storage of food with available utensils. Tsikonina is a community-based nutrition activity for children aged 6–59 months (about 5 years) suffering from moderate malnutrition and at risk of developing malnutrition. Tsikonina provides an opportunity for neighborhood caregivers to come together and learn from each other about best practices to keep children healthy and well-nourished. IMPEL | Implementer-Led Evaluation and Learning 14 Findings and last day of Tsikonina, while in other districts, weighing takes place separately. In Vohipeno and Mananjary, sensitization and awareness teachings are delivered after CDs. Prior to the cyclone, the validation data suggest initial success. In Manakara, the number of children participating in CDs was steadily increasing, and community members in Farafangana and Vohipeno spoke of noticeable weight increases in their children following participation. However, cyclone Batsirai caused significant disruptions, including damaging WASH facilities in some CD/T sites, leading interventions to be suspended. Implementation Successes CHVs and CNVs were largely positive about the training, topics, and delivery, noting that the involvement of Basic Health Center (CSB) staff and doctors was helpful for sharing knowledge. In some cases, respondents noted that participants, as well as other community members watching cooking sessions, were “eager” and enthusiastic to learn new recipes and skills. One male CNV from Farafangana noted that they had run out of leaflets and booklets to distribute, as many people had been interested. Similar to other interventions, distribution was cited to be a key driver in encouraging attendance at weighing sessions and cooking demonstration interventions. The strength of this pull was illustrated by one quote from a female CHV from Vondrozo who recalled that they sometimes resorted to lying about distributions in order to encourage participation: “What makes people come to a meeting is the fact that there is something that is going to be distributed there… If you want them to come, you tell them: come for there is something that we give you freely… you have to lie to the people so as to make them come to the meeting. If you don’t do that trick, the people would not come.” Some CD/T and M2M respondents interviewed, as well as community members interviewed informally as part of the validation exercise, recalled that CD/Ts had led to weight gain among children involved in sessions, though this could not be verified. Implementation Challenges The most commonly reported challenge for CD/T interventions was a lack of food and income due to cyclone Batsirai. Respondents noted that crop destruction, in particular, meant that local nutritious foods were no longer available or are too expensive, meaning it was impossible and, respondents argued, inappropriate to conduct CD/T sessions. One female CNV from Vangaindrano recalled: “The flood [from the cyclone] destroyed all the maize, the cassava, the bananas, the breadfruit. The HH don't have enough money. It's hard to do Tsikonina right now. Doing Tsikonina now, with lack of food and lack of money, is difficult, even impossible. Today, most households are struggling to feed themselves. This may discourage mothers from doing Tsikonina.” Process Evaluation of the FIOVANA RFSA in Madagascar Findings 15 Though many CH/NVs reported having received materials for holding cooking demonstrations at the time of the interview, such as plates, spoons, pots, and cooking materials, they noted unexplained delays in receiving this equipment. In the meantime, most CH/NVs conducted CDs using equipment borrowed from community members, CSB, or other CH/NVs, though one female CHV from Vondrozo stated that they were waiting to receive equipment before carrying out CDs. Many respondents also noted that the materials they had received were not enough, which one female CHV from Mananjary noted meant that they had to limit the number of participants. A few CHV/Ns also complained about the equipment quality, with one female CNV from Vangaindrano reporting that the distributed scale had already broken and a male CHV from Mananjary complaining that the baby carrier on their infant scale was too wide. Some CHV/CNVs reported that the incentive payment they had received for training had not been enough to fully remunerate a day spent not working on their livelihoods (female CHV from Vohipeno). The incentive payment was also not enough to cover all transport costs and meals (male CHV from Manakara and male CNV from Farafangana, though the respondent from Farafangana noted that the stipend amount was increased after the first training session). One female MM from Manakara also explicitly stated that she had not conducted any household visits due to a lack of training and, consequently, a lack of confidence and soft skills awareness in how to conduct household visits. In her own words: “It is difficult to lie, there hasn't been a first follow-up, but we just ask each other during our meeting. I was first encouraged to visit each member’s home at least two times a month, but I hadn’t received any training, so I don’t know how to do that… I don't understand what I'm supposed to do. [Laughs] It’s so difficult for me. There should be some instructions teaching me what to do when I go to households. I may get into people's private lives; I may ask a lot of questions to them. It might not be what should be done, and I may end up making people angry.” Time availability for CHV/CNVs, LMs, MMs, and participants was often cited as a problem due to the need to attend funerals, disruptions and damages from the cyclone Batsirai (explained further in the Cyclone Batsirai section above), and women’s domestic responsibilities, a situation exacerbated by the length of 12-day Tsikonina sessions. As one male CHV from Manakara stated, “The schedule was a problem for some mothers since the culinary demonstration was done in the afternoon. They are tired from working in the morning, so they are not motivated. In addition, the culinary demonstration lasts for 12 days, and there is a lot to do during its fulfillment.” Another CHV, a female from Vondrozo, noted that only two mothers attended a sensitization meeting they held about breastfeeding due to community members’ need to work in the rice fields. One female CHV from Vondrozo noted that she had recently given birth and was unable to attend the CD/T training or conduct interventions. The high time demands as a CHV led one female from Vondrozo to complain that CHVs should be paid to undertake Tsikonina interventions: IMPEL | Implementer-Led Evaluation and Learning 16 Findings “Tsikonina remains a problem for us because people are very busy, and it demands a lot of time, 12 days of VAD; however, the CHV does not have a fixed salary job apart from agriculture, so the CHV activities are additional. It is a fact that the CHV does not have any motivation for this project, and the CHV complains a lot that the activity is not properly carried out due to a lack of motivation.” The need for financial contributions, exacerbated by crop destruction from the cyclone, also posed a barrier–sometimes prohibitive–for participants. For example, a male CHV in Manakara recalled that only 5 of the 14 mothers completed the Tsikonina activity as they could not afford the contribution of 300 Ar (approximately USD .075) each. One male CHV from Manakara also suggested that financial contributions could be difficult because men had decision-making power over household finances, and consequently, women needed to ask men for money to contribute to and attend CD/Ts. One female CNV from Vangaindrano indicated that participants and CNVs might hold negative perceptions of ADRA and FIOVANA due to disagreements about targeting. As she recalled, unexplained differences in CH/NVs and ADRA’s targeting could result in a level of burgeoning mistrust: “With ADRA, there is a problem. E.g., as part of food distribution, we CNV identified children in the community during the weighing session… We listed 282 children for food distribution, we sent this list to ADRA. After 3 weeks, we received a list [from ADRA] with only 44 beneficiary children. Parents no longer trust us. They think that we have discriminated against children who will not benefit from food. The only explanation we can give is that there is a problem with ADRA’s data processing software.” Some respondents also voiced challenges regarding outcomes, namely in ensuring CD/T participants implemented the techniques they had learned. A few reported having problems “convincing” mothers, which they attributed to a lack of education and preference for traditional healers, as well as a lack of resources to implement CD/T lessons. Process Evaluation of the FIOVANA RFSA in Madagascar Findings 17 One male CNV from Farafangana voiced concern that the benefits of changed cooking practices may not be felt by under-5 children due to large household sizes and lack of resources: “During HV, we can see that the households are very poor, they have nothing even they try hard to continue the Tsikonina at home. I am nervous to see that because they cook the jackfruit in their own way, without oil nor the other ingredients showed during practice, all ingredients are missing. It is really sad but we don’t know what to do… We have already given all the instructions they should apply. We can’t even give them what they need for the Tsikonina at home because we are ourselves in a bad situation. In addition, there was the cyclone so the household barely find some cassava powder to cook… [Also] What I have noticed during my home visits is that the whole family takes the child's food when there are more than 10 people in the house. This means that the child can't really enjoy the food that is dedicated to him.” Another male CHV from Mananjary noted that in some cases, expectations of receiving ingredients from FIOVANA could prevent mothers from making efforts themselves at home: “We noticed during our HV that some mothers have some ingredients at home. But they continue to practice their previous habit. Sometimes some mothers do not practice the instructions because they expect to have support from FIOVANA when they participate in the activity. They didn't think that they are the ones who have to bring the ingredients.” 3.2.3. School-Based Approach SBA interventions were at very early stages at the time of data collection, with respondents, namely school administrators and teacher influencer volunteers, noting that they had received awareness raising about SBA, but had not yet received implementation training. Many had nonetheless selected land for school gardens and set up school clubs, though they had not initiated programs otherwise. Most only discussed school gardens and showed little awareness of other aspects of the SBA interventions such as WASH, which both reflects the early stages of the SBA intervention and it could suggest that agricultural interventions are perceived to be the most relevant. SCHOOL-BASED APPROACH SBA interventions reach primarily pre-adolescents, as well as some adolescents. FIOVANA staff work with communities and school authorities to create school clubs. Under the facilitation of a Teacher Influencer Volunteer (TIV), groups of students are identified to promote key behaviors at the school level, such as improved feeding practices, dietary diversification, and WASH. Students are also encouraged to share these improvements with their families. Club interventions will include school gardens, cooking demonstrations, and nutrition classes, as well as meal tasting and judging. Heads of households will participate in meal tasting to foster intergenerational communication about food and nutrition. IMPEL | Implementer-Led Evaluation and Learning 18 Findings Validation Findings Teachers and parents confirmed that the location for school gardens has been identified in all districts surveyed. In all but one district (Vohipeno), school clubs had been established and club members selected. Each club consists of two teacher representatives, two parent representatives, and 22–35 students ranging from grades T2–T5. In all districts but Vohipeno and Vangaindrano, the teachers and parent representatives received a two- or three-day workshop that covered additional information regarding the establishment, operation, and goals of SBA interventions. Beyond the establishment, SBAs are not currently operational. In Vondrozo, the school group had selected the land and decided to cultivate vegetables, but no work had been done on the school yet. In Farafangana, training sessions had been originally scheduled for late March but had been delayed, with no reason given. No surveyed schools have received a school garden training session, and participants are waiting for the training to begin work. Progress is further stalled at the school surveyed in Vohipeno, which does not have any water infrastructure or a garden-like space, both meant to be selection criteria for SBA schools. Implementation Challenges Given the very early stages of the SBA interventions, few concrete successes and challenges were noted. As a success, respondents noted that they, as well as parents and students, were enthusiastic about the school gardens interventions and felt that these would be relevant and bring positive outcomes. Respondents also mentioned several challenges, some of which remained speculative and are detailed further below. ● Respondents often cited a lack of materials for school gardens, such as seeds and tools, and even asked enumerators whether FIOVANA would be distributing these items. Some added that the need to make contributions if materials were not distributed could be a barrier and put people off participating in SBA interventions. ● A few respondents reported experiencing challenges with the electronic format of the trainings they had received on SBA interventions. They recalled having difficulty keeping up with the slides and taking notes and that though they had been given soft copies of the presentation, they would prefer to receive these in physical form. As a result of these challenges, they felt they still had some gaps in their knowledge. ● Though all schools interviewed had selected land, one voiced concern for the security of the school garden plot, noting that their selected area was a little way away from the school. ● SBA respondents noted that they intended to work with CISCO and DREN as part of the SBA interventions. However, given the early stage of SBA interventions, details of this collaboration and what it would consist of were not clear. Furthermore, one respondent voiced a speculative concern that it would be difficult to work with Circonscription Scolaire (CISCO) and Direction Régionale de l’Éducation Nationale (DREN), fearing that they would not have enough time to dedicate to their school and SBA interventions. ● One respondent, a female TIV, noted that parents’ time availability could be a barrier to their involvement and support of SBA interventions. They recalled that at their school in Farafangana, only two parents volunteered as parent representatives. While other parents were supportive of Process Evaluation of the FIOVANA RFSA in Madagascar Findings 19 the intervention, they were too busy with their own work to assume responsibility for the SBA interventions. ● One TIV from Vondrozo noted that food shortages, such as those following the cyclone, could negatively impact SBA interventions as they led to children missing school and thus affected participation in school clubs and activities. 3.3. Purpose 2: Agriculture and Livelihoods Purpose 2: Household incomes and production are sufficient to access food and non-food essentials and build savings. 3.3.1. Farmer Field Schools FFS appear to be established, and FFS interventions/responsibilities are well-understood LFs. However, the pace of implementation and uptake varies significantly between regions and specific fokontany due to uneven provision of training, materials, and seeds. Broadly, LFs and FFS group members appear to have received training and are applying improved techniques (specifically pesticides, compost, weeding, and planting). However, most LFs indicated that a significant challenge to their interventions is seed and tool delivery delays, which have caused planting/harvesting setbacks and have been a source of demotivation. The cyclones had an acute impact on FFS implementation. In most cases, the cyclones completely destroyed crops on FFS demonstration plots and caused FFS interventions to be suspended due to reconstruction and recovery. Validation Findings Validation data illustrates that all districts have established FFS groups with participant numbers ranging from 15–23 per group. Before the cyclone, all FFS groups were active and met at least once, in some districts twice a week, to work on the demonstration plots, and practice learned techniques. Training topics include raised beds, basket composting, plant spacing, and organic insecticide. For members absent at sessions, a 500 Ar fine was levied. In an FFS group in Vondrozo, attendance forms are required for each member. Prior to the cyclone, the validation data indicated initial uptake and success. In Manakara, participants had started applying the techniques they learned during sensitization to each of their farms and were FARMER FIELD SCHOOLS The FFS is an approach where producers are empowered with improved production practices using demonstration sites to strengthen their production outcomes. A group of farmers (15–25 farmers) is supported through the entire production cycle, with all trainings following the physiological growth stage of the produce/product of focus. Demonstration sites are used as avenues to experiment and learn about the improved practices and their likely effects. IMPEL | Implementer-Led Evaluation and Learning 20 Findings seeing results. A participant used the example of a cassava plant yielding around 15kg of cassava roots utilizing the new techniques. In Vohipeno, the FFS group produced 85 cups [21,250 grams] of beans on their demonstration plots. The FFS group in Vangaindrano was able to sell harvest produced from their demonstration plots, and in Mananjary, they were able to harvest some small products before the first cyclone hit. The cyclone severely damaged demonstration plots and presented significant challenges to the progress of FFS groups. In Vohipeno, Farafangana, and Manakara, the validation data indicates that the crops were completely damaged by the cyclone. In Farafangana, no work has taken place on the demonstration plot since, and the FFS group in Manakara was just again starting work. Implementation Successes Nine of the ten LFs interviewed expressed that training (whether formal or informal) is/was received from animators and that they were transferring this knowledge to members on demonstration plots. Half of LF respondents cited specific and detailed descriptions of improved techniques for pesticides, composting/fertilizer, weeding, storage, and planting/harvesting varieties of crops used on demonstration plots. For example, two suggested: “There was also training given by the Purpose 2 animator. For example, when planting beans, the soil cannot be burned, make a plate-band, make a canal, when the soil rises in the plate-band, not planting following the ground up but the plate-band is done horizontally, and when planting, it should be done following a crossed row rather than traditional aligned sowing raw.” (Male LF, Vondrozo) “We learned about making fertilizer. We made some composted manure at that time, so we asked some people to dig a hole which length and width measured 1.20m. Next, we gathered some dry grass, green grass, and cow dung, then mixed it all. He also taught about how to plant beans and mustard leaves. That's what he taught us.” (Male LF, Mananjary) Five of ten LF respondents indicated that the use of improved techniques and seeds that were provided allowed them to increase production and/or sale, which helped to improve resilience and increase income or food availability. This was described best by two LFs in Vondrozo and Vohipeno: “For example, for the last cropping season, we are given bean seed, we planted bean, and the production belonged to the group. The production was very good and some was sold, and some was left for seed for the next growing season. The cyclone had arrived and all the members were in difficulty. The part destined to be seeded was sold. I told the other members that we should make contributions to buy seeds for the next cropping season. What I would like to say is that the bean production was what saved the members from starvation during the cyclone period.” (Male LF, Vondrozo) Process Evaluation of the FIOVANA RFSA in Madagascar Findings 21 “For example, we used to not remove weeds while growing rice. But when we applied the training or techniques we got, then, we saw that our yield increased. Before, we only harvested 10 rice-measures when we didn't remove weeds, but when we did, we harvested 18 rice-measures.” (Female LF, Vohipeno) Seven of ten LF respondents indicated that training, collaboration, and communication with animators was positive, and they felt supported. LFs from Vohipeno, Vangaindrano, and Manakara, in particular, spoke glowingly of the support they received from animators, for example: “He spares no time in coming to work with us. He is the one who bought our seeds the first time. And when he speaks and advises us, we members respect and do what he says to do. His enthusiasm to lead us makes us say that it is worth working with him, he teaches us agriculture.” (Female LF, Vangaindrano) Implementation Challenges The cyclones had a major impact on the implementation of FFS interventions over the past few months. Every LF respondent suggested that the cyclone had caused devastation in their communities. Flooding and wind destroyed crops, demonstration plots, and houses/meeting places, causing the postponement of FFS activities/meetings, setbacks in crop/livestock production, and, in many cases, member dropout and/or demotivation. All LFs cited crop or demonstration plot destruction as a setback or challenge to continuing to implement FFS interventions. In most cases, demonstration plot crops were completely destroyed by the cyclone, and LFs/FFS groups are just starting to resume activities. LFs in Mananjary (2) and Farafangana suggested they needed to find new land for the demonstration plot to continue interventions. One LF in Farafangana said that a member had provided a new place for the demonstration plot after the destruction of the original plot. Further, the impacts of the cyclone appear to have caused demotivation to engage in FFS interventions, including meetings and plot demonstrations/cultivation. The majority of LFs (6 of 10) suggested that participation had fallen because members were either discouraged by the destruction of crops or were too busy rebuilding houses and/or trying to find food and/or income opportunities to participate in FFS interventions. IMPEL | Implementer-Led Evaluation and Learning 22 Findings “There was a change in the participation of the Farmers' School's members. We were struggling making a living, so we went to look for some activities in order to earn money. There was no seeds, and we had no money to buy some. Still, we aren't discouraged; we plan to get back to our activities at the school garden. Some of the members quit; there were 18 of us in the beginning, but now, there are only 14 left because of the hardships.” (Female LF, Farafangana) “The members are not as eager to do the work on the school field, but due to the damage caused by the cyclone, everyone has prioritized the repair of the houses and the planting of rice, even though it is a bit difficult because of the flooding.” (Male LF, Vondrozo) The majority of LFs (7 of 10) and Purpose 2 animators suggested that incomplete or delayed seed delivery were challenges or barriers to the implementation of FFS. In some cases, LFs and animators expressed that delay in seed or seed voucher delivery caused issues as seeds came too late in the growing season to be effectively planted/cultivated. “Most of the seeds arrived too late, and we could not catch up with the growing season. It is an effort that we would like to ask FIOVANA to deliver the seeds to us in a timely manner.” (Male LF, Vangaindrano) “Another negative side is the delay in seed delivery. For example, we should normally plant beans in October; however, we only received the seeds in November.” (Female LF, Vohipeno) In other instances, LFs and animators suggested that seeds/seed vouchers were not delivered at all, thus delaying planting and work as the demonstration plot: “As I have said, we need seedlings; group members struggle to find seedlings. As we speak, there are no seedlings at all here. Group members have no seedlings, even for rice farming. They are all waiting on the project, and yet, the very purpose of the FIOVANA project is to change people’s mindsets. They should have savings, provisions, and seeds for the next crop year.” (Male CA, Vohipeno) “The lack of seeds may fail and disrupt the work of the FFS because we have requested them but have not received them. Pascal, the animator who works with us, took our request, but we don't know if it reached its destination.” (Male LF, Mananjary) Process Evaluation of the FIOVANA RFSA in Madagascar Findings 23 Two Purpose 2 Animators in Vangaindrano suggested that chicks had not been delivered to FFS groups that aimed to implement chicken farming. One animator said that the FFS group was trained and prepared for the chickens, but they never received them, and the group has now dissolved. In the other case, the animator suggested that the FFS group has been waiting for 2 years for chickens. Five of ten LFs noted that a lack of tools was a challenge to implementing improved techniques and was a demotivator for FFS members. Lack of material inputs was cited by LFs in Vangaindrano, Vondrozo, and Mananjary. Two LFs detailed the input needs and frustrations: “And this is exactly what we are complaining about; we are 15 members in the FFS group, and we have already sent a report asking for tools like shovels, but we haven't gotten anything yet. It is very frustrating, three of us have already stopped but were convinced and just came back.” (Male LF, Mananjary) “We lack materials such as watering cans, wheelbarrows, shovels, and spades. We have been promised these materials, but they have not always arrived until now. So, when it comes to improvement, the tools to work must be improved.” (Female LF, Vangaindrano) Five LFs (in Vangaindrano, Mananjary, and Manakara) suggested that training was insufficient or incomplete. Reasons for dissatisfaction with the training included: lack of clarity/instruction (pesticide), delay in meal compensation, topics of training (administrative rather than technical), tardiness of the trainer, and the need for additional technical training. Additionally, one LF in Farafangana suggested that they had not received any training or follow-up support from an animator, though it should be noted that the LF mentioned they were sick and missed a training session. Three Animators (in Vangaindrano, Vondrozo, and Manakara) and three LFs (in Vondrozo, Vangaindrano, and Mananjary) also cited general issues with participation/dropout in FFS groups due to various reasons, including time availability due harvesting season for clove/rice, working in the mines (Vondrozo), poverty/hunger (cyclone and non-cyclone related), and, as noted above, demotivation due lack of delivery of tools and seeds that were promised. Two animators (in Vondrozo and Vangaindrano) and one LF (in Farafangana) spoke of issues with the registration of members who want to participate in FFS interventions but do not have a code or have not been included in the census. One animator in Vangaindrano suggested that this can cause tension in the community and demotivation, as those without a code cannot participate in FFS interventions. IMPEL | Implementer-Led Evaluation and Learning 24 Findings 3.3.2. Farmer Market Schools FMS interventions appear to be in very early stages; only one respondent (in Manakara) suggested that the FMS group was established and had received training. Given the nascence of FMS interventions, respondents often spoke of other interventions, namely the FFS interventions they were engaged with. When they addressed the FMS, respondents indicated that they were interested in the FMS interventions and that recruitment from FFS members to join was or would be successful. That said, FMS respondents appear to have limited knowledge of the specific purpose and interventions of the FMS and indicated the need for FMS sensitization, establishment guidance, and training to get interventions off the ground. Validation Findings Validation data suggests that FMS groups are not yet active in any of the surveyed districts—no meetings or training have been conducted. All the surveyed districts are still in the process of announcing activity details, identifying potential participants, and delivering information about the FMS activity to interested parties, with the exception of Manakara, where a group of 22 members has been established. The validation data noted that a potential challenge to the creation of FMS groups is decreased production due to cyclone destruction. As noted in previous sections, the crops of many farmers were completely destroyed by cyclones, leaving farmers with a greatly reduced surplus of produce to sell. Implementation Successes and Challenges While only one FMS (of the FMS sampled) has been implemented thus far, there appears to be interest in FMS interventions. Further, LFs appear to be motivated to engage in FMS interventions and noted that they foresee no issues in recruiting members to participate in FMS interventions. Two LFs suggested that their leadership of the FFS made it easy to recruit members to FMS. Most FMS respondents indicated that they knew of the FMS but had not received support in the form of training or guidance on FMS interventions yet. It appeared that FMS respondents either had a very cursory understanding of FMS goals and tasks or no understanding at all, summarized best by two respondents from Vohipeno and Farafangana: FARMER MARKET SCHOOLS ADRA’s FMS approach operates on similar principles of discovery learning as used in FFS, in that smallholder farmers gain knowledge and skills on how to explore markets, what the market can offer, and how to make market decisions. FMS methods focus on experiential, practical, collective, and adult learning methods. ADRA will coordinate with private sector actors to improve farming households’ understanding and participation in market functions. FIOVANA will focus on leveraging the private sector to improve the enabling environment. FMS Agents will tailor sessions to the needs of activity participants and communities. The focus in FMS sessions with IGA groups will be on empowering participants to understand the demands of the market and knowing how to act. Process Evaluation of the FIOVANA RFSA in Madagascar Findings 25 “I haven't received any instructions or training; I just had a talk with [Purpose 2 Animator], since he helps us… We also need to be trained, because even me, the leader, have no idea of the tasks I should be performing, and we, members, know nothing about market opportunities.” (Female Management Committee Member, Vohipeno) “I'm really sorry if my explanation about it is not complete because we don't know much about FMS yet. FFS is what we have really experienced. It has been a week that we were told about the creation of an FMS group here in the Commune of Vohimasy. They said a group will be created, and it will search market first before producing. We already have an FFS group, and we decided to create an FMS group. No meeting was held yet; all these were just told to us verbally. I don't know how to explain it in details.” (Male LF, Farafangana) The male LF from Manakara indicated that the group was established, he had received some FMS training, and he had gone on a market information visit to Marofarihy and Manakara. However, the training he referred to appeared to be mostly related to FFS interventions (i.e., chicken farming techniques and vaccinations) rather than FMS-specific training. He suggested that: “The next problem is the fact that I wasn't given enough training, so the teachings I can give the members are limited.” 3.3.3. Impact Investment Fund ADRA visited prospective business co-investment partners that were interviewed to discuss the IIF co￾investment. However, all respondents suggested that there had been no follow-up after the initial visit and that the process to apply/register for the IIF co-investment was unclear or had not progressed. IIF specialists suggested that the process for business co-investment, selection, and approval, was very long due to internal ADRA processes, and, as such, few approvals and disbursements have been executed. IMPACT INVESTMENT FUND FIOVANA is targeting identified chains within the broader market that have the potential for positively impacting vulnerable people and seeks to address “market shortcomings” on both ends of these value chains. IIF addresses weaknesses among micro, small, and medium-sized enterprises further down the chain (e.g., buyers, transporters) and value chain additions (e.g., input suppliers, processors, service providers) that result in vulnerable target groups being underserved. As a concrete example, IIF could aim to support buyers/transporters who are limited to only buying small amounts due to the small carrying capacity of their vehicles. The IIF targets businesses that work directly with one of the identified FIOVANA value chains and are determined to be in a position to positively impact participants. IMPEL | Implementer-Led Evaluation and Learning 26 Findings That said, IIF specialists also note that processes are being improved and the implementation of the due diligence process has helped speed up SME selection and validation. Validation findings are not presented here because IIF was not included in the data validation exercise. Implementation Successes and Challenges Disbursements. IIF staff suggested that at least four SMEs had received disbursements (two in Manakara, at least one in Farafangana, and one in Vangaindrano), and several other approvals were in the final stages of approval at ADRA’s Administrative Committee (ADCOM) level. ADRA staff had visited all prospective IIF co-investment partners interviewed to follow up on the IIF co￾investment. However, most respondents said there had been no follow-up or communication past the initial meeting with ADRA. Three of the business owners appeared eager to move forward with the process but they were waiting for information from ADRA on the next steps for the co-investment. One respondent in Manakara suggested he had been visited three times, going through multiple steps in the IIF process, including sensitization, data collection, and quote provision. However, he suggested that he had not been contacted about moving the process forward since 2021 (at the earliest). SME selection and due diligence processes appear to have improved since the start of IIF interventions. One IIF staff respondent suggested that the IIF team has improved the original selection/vetting process (drafted by Miarakap IIF) and has since implemented a due diligence process to speed up the SME validation process. That said, the process of selection still appears to be lengthy and onerous. An IIF staff respondent suggested that environmental, banking, and tax compliance/registration processes are still lengthy (because often businesses are informal and must register), and businesses often do not have the capacity to produce solid business plans. One IIF staff suggested that the selection and approval process is currently taking 4 to 6 months to reach the final step of ADCOM approval. 3.4. Purpose 3: Enhanced Social and Ecological Risk Management Purpose 3 Enhanced Social and Ecological Risk Management. 3.4.1. Locally Managed Marine Areas LOCALLY MANAGED MARINE AREAS FIOVANA aims to support best practices for the implementation of LMMAs and coastal conservation initiatives. Working with traditional and local leaders and farmer and fisher groups, FIOVANA will promote mapping and the development of an action plan addressing the community’s most pressing coastal restoration and protection issues and current interventions. Where feasible, action plans will address restoration of coastal wetland systems to mitigate damage from storms and improve freshwater quality and incorporate good management of marine resources through LMMAs. Process Evaluation of the FIOVANA RFSA in Madagascar Findings 27 LMMAs are being initiated currently, and thus there is little evidence to gauge implementation quality. Based on responses that were coded, some early observations include the following: ● Sensitization interventions in focus fokontany are underway. o This appears to include some initial surveys, in particular, to target fokontany. ● There are regular references to committee formation efforts, though they appear not to have been fully constituted yet. In ground-truthing, ADRA acknowledged this status and reported that it anticipates implementation speeding up after the impending round of strategic planning workshops with the Mihari network. Validation findings are not presented here because the LMMA was not included in the data validation exercise. 3.4.2. Disaster Risk Management Committees DRMCs appear to be robustly implemented. Community mobilizers and supervisors were able to provide detailed responses across all implementation areas, and feedback was consistent and thorough. Validation Findings Validation data indicate that DRMC committees of ten members have been established in all visited districts and that the majority of committees include strong female representation. Each committee has received training lasting 4–6 days. Sensitization was focused on raising awareness in the community on disaster awareness, preparation, and recovery. Learned topics included the hoisting of emergency flags and their color significance, tree removal in areas close to houses, evacuation of damaged houses, gathering evacuees in a public place, and fortifying the roof with sandbags to protect it from the wind. Validation data also suggest initial successful uptake of training. Following cyclones that damaged their communities, DRMC committees demonstrated the capacity to initiate a response. Post-disaster DISASTER RISK MANAGEMENT COMMITTEES FIOVANA will use DRMCs to empower community members by building social capital and linking them to government and other stakeholders for strategic relationship building. ADRA initiated the process of cultivating government and stakeholders’ support during the design of the activity and they will continue engaging leaders early to ensure support for active community involvement. In places where DRMCs do not exist, FIOVANA DRMC agents will establish them, identifying motivated natural leaders representing youth, women, and other underrepresented groups. In places where DRMCs exist, we will evaluate the strength and diversity of committees and make recommendations. The activity will provide training to DRMCs on their roles and responsibilities, community mobilization and motivation techniques, and the importance of active participation of underrepresented groups. FIOVANA will coordinate with the National Office of Risk and Disaster Management to strengthen fokontany and commune-level DRMCs. IMPEL | Implementer-Led Evaluation and Learning 28 Findings surveyed DRMC committees in Farafangana, Mananjary, and Manakara submitted reports cataloguing the effects and damages of the cyclone on their communities. In Farafangana, the DRMC committee additionally restored the tranobe’s house to host disaster victims whose houses had been destroyed. The DRMC in Vangaindrano and Mananjary also conducted a community census in conjunction with ADRA and delivered it to them. An additional challenge the validation data notes is equipment delivery delays in Manakara and Vangaindrano. One respondent in Manakara reported that they do not have the necessary equipment, such as radios, waterproof clothing, rain boots, and mobile phones that would enable them to properly carry out their work. Implementation Successes First and foremost, responses coded specifically for DRMC demonstrate regular and consistent implementation. All Purpose 3 animators and supervisors detailed the specifics of forming or revitalizing committees, community sensitization, recruitment or participant selection, training, obtaining and distributing key materials (bullhorns, flags, etc.), and even the DRMC’s specific responses in the event of a catastrophic cyclone. Importantly, multiple responses, when probed for “effectiveness,” noted that there were either no or minimal casualties that resulted from recent cyclones, partly because of the warnings that DRMCs issued and the community’s response. As part of the implementation, moreover, supervisors and animators appear to communicate and coordinate effectively. Other important observations that contribute to the effectiveness of DRMC implementation include: ● Robust youth participation (this was most notable in the seedling nursery activity but is nevertheless worth acknowledging). ● Effective gender inclusion, with animators regularly reporting committees formed with strong female participation in particular (ratios of 7:3 and 6:4 male-to-female, for example). ● Effective spatial coverage, with representation from across the fokontany itself, facilitating both participation, but also the effective distribution of messaging outwards to community members. ● Broad community “buy-in” and participation, whether in addressing localized risks (weak roofs or threatening trees) or participation in road and drainage improvements. ● Effective linkages with local government and local governance mechanisms, including informal processes. This is highlighted in more detail below. Local governance engagement. Supervisors and animators appear to be deftly navigating the inherent challenges of implementation under conditions of overlapping authority structures, which include the presence and significance of traditional leaders and related customs. It is evident both from interviews and ground-truthing that ADRA places significant effort and thought into working with local leaders, including ampanjaka (approximates to “kings”) and mpanjaka (clans or families of local prominence). Some of this work is facilitated by partners (like FIANTSO), but it nevertheless comes down to the abilities of individual supervisors to navigate the intricacies of local structures and, sometimes, the individual preferences and expectations of local kings. This process is, of course, not perfect, as many respondents acknowledged that some local leaders are more responsive or amenable to ADRA activities than others (though DRMCs, as already prescribed by law and inclusive of FCs, were generally non-threatening). More crucially, animators need to Process Evaluation of the FIOVANA RFSA in Madagascar Findings 29 accommodate these special interests while delivering on an equitable and inclusive intervention that meets its objectives. This process of honoring the credibility of local authorities and customary decision￾making processes has proven critical in other contexts, even as it requires extreme tact and consideration, and it is likely to prove here as well.5 Implementation Challenges Many of the challenges confronting DRMC implementation are those affecting implementation across the region and other interventions, including the destructive nature of recent cyclones, difficult travel conditions for residents and animators/supervisors alike, time constraints, navigating local interests (as discussed above), and access to land (most relevant to the nursery activity and reforestation). These, therefore, require no further exploration, and there is evidence that Purpose 3 implementers are attempting to address these challenges as they can. Another obstacle that has come up elsewhere (Sections 1.7.1.3 and 1.8.1.3 in particular), however, is compensation. Despite this, the consistency and regularity of it being referenced by supervisors and animators at various levels, and despite respondents being unprompted (apart from asking about any challenges), underscores its significance.6 A male District Supervisor (DS) from Vohipeno recalled: “Some people do not come back anymore on day 2. That is the reason of the absence of the 15 to 20 people in the trainings that I said earlier… The problem is also that it is still a long procedure to get the compensation of the training… These are people who worked on February 21st, and it is only now [March 22] that we send the food request.” Delayed compensation—the most commonly expressed complaint—has a deleterious effect on ADRA credibility at the fokontany and individual level and creates frustration (as evident from responses) among trainers and animators themselves, as well as participants. Additionally, it diminishes the willingness of citizens to participate in interventions and delays the transfer of intervention ownership to locals. One male district supervisor from Vangaindrano explained: “It (delayed compensation) impacts the team’s motivation to focus on meetings and trainings, which was very troublesome. As soon as an SMS is received, I immediately check whether it is money or not. Honestly, it's really hard here, and no one complains about it… Our procedure here has too much delay.” 5 McNamara, Thomas. (2019) “The Limits of Malawian Headmen’s Agency in Co-Constructed Development Practice and Narratives,” Journal of East African Studies 13(3): 456–484 https://doi.org/10.1080/17531055.2019.1599196 6 The “remuneration” code, specifically, was among the most used within the “P3” Document Group (all interview transcripts related to Purpose 3), just below “cyclone,” indicative of its resonance among respondents. IMPEL | Implementer-Led Evaluation and Learning 30 Findings Ground-truthing with ADRA makes clear that it has examined the compensation question on certain levels and that it is highly cognizant of the risks of creating a dependency or that some complaints may, in fact, stem from unrealistic expectations of FIOVANA. Without diminishing these arguments, most recorded responses were among animators and referred more to a delay in processing compensation or accessing the necessary code (based on participation), which suggests that compensation was already offered, but that follow-through has been poor. Process Evaluation of the FIOVANA RFSA in Madagascar Conclusions and Recommendations 31 4. CONCLUSIONS AND RECOMMENDATIONS The ADRA team has persevered with implementing FIOVANA despite the extreme setbacks posed by cyclones in most RFSA areas and COVID-19 restrictions, which limited travel and communication. At the time of data collection, shortly after cyclone Batsirai, it appears that while some FIOVANA interventions have made good headway, including M2M, CD/Ts, and FFS, others remain stalled or are still in early stages, such as IIF, FMS, LMMAs and SBAs. Furthermore, the cyclone has severely disrupted interventions across the board, providing an additional challenge that impedes both implementation and the potential positive outcomes of interventions. Findings from this report can mostly be arranged as a 2x2 table that distinguishes between these more logistical versus ‘ideational’ implementation challenges (See Table 1). Table 1. Findings rubric (with illustrative findings reflective of cell) Practical/“Mechanical” “Ideational” Internal to ADRA • Training topics and modalities • Supply chains • Distribution • Census cokes • Working with local leaders to facilitate interventions; honoring traditional roles and values FIOVANA Communities • Resource shortages, due in part to the cyclone • Time availability, particularly for women and youth • Land availability • Business readiness • Desire for and frustration with time demands and lack of compensation/distribution • Confusion around FIOVANA activities and trainings • Links with animators • Targeting and exclusion Specific recommendations by intervention are as follows. Reviewing and amending these recommendations were a focus of the CLA workshop, and ADRA used breakout sessions to digest, amend, and build action plans for adapting to the implementation challenges and recommendations. Note that in some cases, it is difficult to provide concrete recommendations given the early stages of interventions and, therefore, lack of information on implementation so far. General ● Consider what incentive options are possible to motivate community workers and participants. This emerged by far as the most pressing issue at the CLA workshop, where the appropriateness, as well as the types of incentives, was discussed at length. Across interventions, motivation and time availability for participating in FIOVANA appear to be the greatest challenge to implementation quality and, consequently, outcomes and impact, a situation exacerbated by the cyclone emergency context. ADRA should thus consider what forms of incentive or motivation can be feasibly rolled out. Additional considerations: o Communication and managing expectations. It is important for ADRA to clearly communicate to the entire community the nature and goals of the intervention and the process for selection and targeting to help improve the transparency and motivation of participants to engage in interventions. IMPEL | Implementer-Led Evaluation and Learning 32 Conclusions and Recommendations o Type of incentive. Linked to the above, incentive options (e.g., community celebrations at key milestones, performance-based stipends/promotions for community workers, etc.) should be tailored to the intervention and fokontany context, continued engagement with communities on sources of motivation could be useful in this regard. o “Many hats.” In reality, many community workers hold multiple roles within their community and within the FIOVANA RFSA itself. The combination of these time demands and requirements on community workers should be accounted for when establishing incentives. o Sustainability. Questions of the long-term sustainability of incentives or payments should be tempered by considering the potential benefits, increased outcomes, and increased momentum that may result from interventions if community workers and participants are well-motivated to participate during the course of FIOVANA. Expectations can also be managed by communicating clearly to participants what incentives will consist of, their purpose, and their duration. M2M ● Improve clarity between M2Ms, Women’s Empowerment Groups (WEGs), CD/Ts, and other interventions for LMs and other implementers involved in these. ● Clarify roles and responsibilities with group leaders via trainings and/or capacity building (formal or informal) to ensure that they understand intervention activities and goals. CD/T ● Consider providing additional cash or in-kind ingredient support for CD/Ts, particularly during the cyclone recovery period, to ensure community members are able to participate in sessions. ● Consider reducing the number of days for Tsikonina sessions, given women’s considerable time constraints. Alternatively, conduct further assessment of ways to ensure that women are willing and able to participate in sessions.7 ● Provide additional leaflets and other informational materials for participants to take home as there appears to be high interest in these. ● To manage expectations and prevent potential mistrust, increase communication and clarity on distribution, including items, timelines, and targeting criteria, as well as delays. ● To increase male participation in CD/T and reinforce gender equality, have men also lead cooking demonstrations.8 SBA ● Adapt trainings and training materials to be more accessible to participants, including providing hard copies of information materials. 7 During the Joint PE CLA workshop with ADRA and CRS conducted in Antananarivo from June 12–16, the P1 team noted that a reduction in attendance days for the cooking demonstrations was not recommended. Instead, they suggested better understanding the participation challenge and finding a middle ground for participants. 8 This recommendation came from ADRA P1 team members during the Joint PE CLA workshop. ADRA proposed that men lead cooking demonstrations and include this action as part of the MLG scorecard. Process Evaluation of the FIOVANA RFSA in Madagascar Conclusions and Recommendations 33 FFS ● Ensure timely seed and tool distribution to all FFS groups to 1) ensure that members can implement improved techniques and plant according to seasonality and 2) keep LF and FFS members motivated and engaged. ● Consider conducting an assessment of damages to FFS demonstration plots to identify areas in which support for land identification or plot reconstruction may be needed. ● Consider following up with supervisors and animators in areas in which training and/or follow￾up has been reported to be missing or incomplete to assess barriers to delivery. ● Consider following up internally and/or with group leaders concerning issues related to member registration/participation and code assignment. FMS ● Consider accelerating efforts to sensitize and guide FMS leaders to establish FMS groups and to maintain enthusiasm for FMS interventions. ● Ensure that FMS-specific training is provided to FMS leaders and groups. IIF ● Continue to work to streamline the SME selection and validation process internally by identifying and clearing bottlenecks in the IIF approval process. ● Work to improve prospective SME sensitization and follow-up; consider utilizing animators to provide follow-ups and maintain communication. DRMC ● Review the logistics involved in providing a code and compensating participants for trainings, as delays in compensation may erode trust and goodwill. IMPEL | Implementer-Led Evaluation and Learning 34 Annex A: FIOVANA Interventions ANNEX A: FIOVANA INTERVENTIONS Mother-to-Mother M2M is comprised of groups of women who come together to discuss issues on Maternal, Infant, and Young Child Feeding Nutrition, including challenges and how they resolve them. They support one another on issues of maternal nutrition and Infant and Young Child Feeding. These mothers are recruited by CHVs based on updated register information from the mapping. The groups constitute 10– 15 participants and meet at least twice a month during intervention start-up and once a month after CHV visits. M2M sessions are facilitated by CHVs/CNVs, to discuss a wide range of nutrition and health topics. There are also specialized M2M groups of pregnant adolescents aged 15–19 to ensure adolescent mothers receive adequate peer support and can discuss issues specific to their cohort. A sub-section of women in the M2Ms who practice positive health and nutrition behaviors are classified as “Model Mothers.” MMs support bi-weekly group meetings by conducting targeted monthly home visits to reinforce learning and practice. MMs are given pictorial books of key practices developed by the Office of National Nutrition and adapted for the regional context. Linking with FIOVANA’s Gender component, during home visits, men, boys, and other members of the family are encouraged to participate in sessions. Anticipated outcomes include deeper engagement of men in childcare practices and reduced burden on women’s time. During quarterly meetings, MMs will gather with the local food and nutrition security committee (COSANS), CHVs/CNVs, and other stakeholders to discuss progress towards adopting health and nutrition practices, barriers, and solutions. As a complement to the M2M, the influencer groups of Community Mother support groups (CMs) have the explicit role of supporting M2Ms by collaborating with COSANS and CSBs; mobilizing communities to participate in health promotion activities; ensuring the M2Ms are supported; tracking health and nutrition of mothers and CU5s and reporting problem cases to CHV/CNVs. CMs organize community action days every quarter for the whole community to celebrate achievements and reinforce awareness. Cooking Demonstrations/Tsikonina Cooking Demonstrations Cooking demonstrations contribute to FIOVANA’s goal to sustainably improve the health and nutritional status of women of reproductive age, adolescent girls, and children under 5. Cooking demonstrations are a community-led intervention promoting the use of locally available or accessible food. The objectives are to: ● Assist households in integrating nutritional recommendations into daily food processing and preparation. ● Provide households with the knowledge, skills, and confidence to better utilize locally available and affordable foods to improve their diet and nutritional status. ● To impart the basics of food safety and hygiene, such as hand washing, covering prepared food, and proper handling and storage of food with available utensils. Process Evaluation of the FIOVANA RFSA in Madagascar Annex A: FIOVANA Interventions 35 The field agent and CNV will be trained with nutrition education first, and then in the technique of facilitating cooking demonstrations, including the evaluation of experiences (taste, smell, appearance, and color of the meals). These trainings will help cooking sessions run smoothly, and are truly participatory, going beyond the demonstration and encouraging everyone's active participation. Field agents and CNV will be provided with a recipe booklet, provided by the national nutrition office, to help them design appropriate dishes according to age and each session’s theme. Religious customs on food preparation should also be taken into consideration, though this may pose a challenge in mixed-religion groups. The health and nutrition team will organize one cooking demonstration per month per fokontany and sessions will be associated with monthly growth monitoring to identify the impact and motivate participants. The CD can also be associated with the session at FFS, such as agricultural fairs. All community members are invited to attend the session and learn. Apart from the close collaboration with the CNVs, local authorities, and traditional leaders, the involvement of men is key to the success of this approach. The number of participants per session should not exceed ten. The health and nutrition team will organize one cooking demonstration per month per fokontany, and sessions will be associated with monthly growth monitoring to identify the impact and motivate participants. To increase sessions’ effectiveness, manageability, and the chances of everyone learning something. Participants are asked to bring their own ingredients to the sessions. Tsikonina Tsikonina is a community-based nutrition activity for children aged 6–59 months (about 5 years) suffering from moderate malnutrition and at risk of developing malnutrition. Tsikonina provides an opportunity for neighborhood caregivers to come together and learn from each other about best practices to keep children healthy and well-nourished. Community health volunteers and ADRA field agents lead the intervention, and there is a linkage with a health center in case the child does not recover and needs a reference to seek professional care. Tsikonina depends heavily on food products supplied by other sectors and is based on the Positive Deviance/Hearth model. As part of this model, positive deviant practices (defined as practices that stray from the norm but in a beneficial way) are first identified. Once identified, a selection of recipes is prepared by field staff based on the recipes of the positive deviant mothers. If necessary, their recipes are modified to include more ingredients, more protein, or ingredients more accessible to every local mother. Once recipes are developed, women in the community meet over the course of 6 or 9 days to prepare the selected recipes for their children and discuss a key health message. Each community selects the session length and time of day (typically either 8 am or 2 pm) that is best for their situation, and caregivers meet each day with their children. Each session includes cooking a nutritious recipe together, feeding the children, and discussing a key health message. Every site should have an accessible latrine and tippy tap to promote good hygiene habits. Each session should be under 2 hours long. On the last day, the children’s weight is compared with their initial weight on the first day. Mothers are then encouraged to continue following the practices that they learned at home, with follow-up visits to their homes for 2 weeks. IMPEL | Implementer-Led Evaluation and Learning 36 Annex A: FIOVANA Interventions School-Based Approach SBA interventions reach primarily pre-adolescents, as well as some adolescents. FIOVANA staff work with communities and school authorities to create School Clubs. Under the facilitation of a TIV, groups of students are identified to promote key behaviors at the school level, such as improved feeding practices, dietary diversification, and WASH. Students are also encouraged to share these improvements with their families. Club interventions will include school gardens, cooking demonstrations, and nutrition classes, as well as meal tasting and judging. Heads of households will participate in meal tasting to foster intergenerational communication about food and nutrition. Farmer Field Schools The FFS is an approach where producers are empowered with improved production practices using demonstration sites to strengthen their production outcomes. A group of farmers (15–25 farmers) is supported through the entire production cycle with all trainings following the physiological growth stage of the produce/product of focus. Demonstration sites are used as avenues to experiment and learn about the improved practices and their likely effects. For FIOVANA, ADRA will modify the traditional model based on experience in Madagascar, which reveals that requesting farmers form groups is not always sustainable and can negatively affect group performance. We will use a phased approach to identify and work with already existing groups of farmers and gradually expand participants and groups based on interest and need. Phased Approach: In the first production year, field agents will engage groups in participatory, hands-on learning, with all members receiving training and follow-up support at least twice a month. Members will be supported in developing their own demonstration plots. During the second production year, field agents will shift work to monitoring and reinforcement with support from animators and the farmer leader volunteers, including on-farm exchange visits for cross learning. Field agents and animators will then focus on forming a second phase of farmer groups. All farmer groups will be connected with commune-level early warning system field agents for information on weather, flood, and cyclone alerts. At the same time, FIOVANA will seek other ways to improve information flows between critical actors, such as value chain participants, government representatives, farmers groups, and cooperatives, and between households and groups at the local level within communities themselves. Farmer Market Schools ADRA will use its own FMS approach with poor and extremely poor agricultural households. ADRA’s FMS approach operates on similar principles of discovery learning as used in FFS, in that smallholder farmers gain knowledge and skills on how to explore markets, what the market can offer, and how to make market decisions. FMS methods focus on experiential, practical, collective, and adult learning methods. ADRA will coordinate with private sector actors to improve farming households’ understanding and participation in market functions. FIOVANA will focus on leveraging the private sector to improve the enabling environment. FMS Agents will tailor sessions to the needs of activity participants and Process Evaluation of the FIOVANA RFSA in Madagascar Annex A: FIOVANA Interventions 37 communities. The focus in FMS sessions with IGA groups will be on empowering participants to understand the demands of the market and knowing how to act. The FMS approach is based on adult education principles and includes discovery learning that is applied to the market. The goal is for the smallholder farmers to understand the market needs, the value chain flow, the value of produce along the value chain, and available sales points to inform their production decision. This is to ensure producers are meeting the needs of the market. The farmers will literally travel to meet the value chain actors, and communicate with them to get all the information they require to make their own production decisions. The FMS process consists of: ● Firstly, a context analysis and community meetings to explain the FMS approach. ● After this, a group of 20–25 volunteer farmers forms an FMS group that goes through 14 facilitated technical sessions. Typically, sessions are half a day and take place weekly. ● Following the technical training sessions, the FMS trainees then start to travel along the value chains of their preferred crops to collect market intelligence. They do it themselves at their own cost. ● Farmers will use the market information to make production decisions while securing sales agreements, as much as possible. ● Finally, ADRA tracks sales made by the farmers. Impact Investment Fund FIOVANA is targeting identified chains within the broader market that have the potential for positively impacting vulnerable people and seeks to address “market shortcomings” on both ends of these value chains. Interventions such as FFS, FMS, and IGAs focus on production, producers, and their capacities/resources. By contrast, IIF addresses weaknesses among micro, small, and medium-sized enterprises further down the chain (e.g., buyers, transporters) and value chain additions (e.g., input suppliers, processors, service providers) that result in vulnerable target groups being underserved. As a concrete example, IIF could aim to support buyers/transporters who are limited to only buying small amounts due to the small carrying capacity of their vehicles. The IIF targets businesses that work directly with one of the identified FIOVANA value chains and are determined to be in a position to positively impact participants. The IIF team works directly and through other FIOVANA team members to identify these businesses. The IIF team then examines the businesses to assess the appropriateness of investment with them (e.g., checking their history of operation, willingness to co-invest, and willingness to co-invest in a venture that would impact FIOVANA’s target group). Businesses may also seek out the IIF to provide support. Applications are submitted and evaluated by a committee. IIF then works with approved businesses to develop business plans based on co-investment both from the business and IIF. IIF conducts follow-ups to ensure the businesses are operating as agreed. IMPEL | Implementer-Led Evaluation and Learning 38 Annex A: FIOVANA Interventions Disaster Risk Management Committees FIOVANA will use DRMCs to empower community members by building social capital and linking them to government and other stakeholders for strategic relationship building. ADRA initiated the process of cultivating government and stakeholders’ support during the design of the activity and it will continue engaging leaders early to ensure support for active community involvement. In places where DRMCs do not exist, FIOVANA DRM agents will establish them, identifying motivated natural leaders representing youth, women, and other underrepresented groups. In places where DRMCs exist, we will evaluate the strength and diversity of committees and make recommendations. The activity will provide training to DRMCs on their roles and responsibilities, community mobilization and motivation techniques, and the importance of active participation of underrepresented groups. FIOVANA will coordinate with the National Office of Risk and Disaster Management to strengthen fokontany and commune-level DRMCs. To encourage sustainability and local ownership, FIOVANA will take the following actions (based on lessons learned from previous DRMC activities): ● Presentation of the FIOVANA RFSA to the community, including the goal, purposes, expected outcomes, and interventions. The presentation day will be chosen in accordance with local leaders, traditional authorities, and the fokontany. All social classes will be invited to participate in a local development approach. ● The concept of Disaster Risk Management and Reduction will be presented to the community and will include a discussion on the perception of local threats and risks. ● An explanation of the election process and responsibilities of DRMC members and the community, encouraging community ownership at the outset. ● The chief de fokontany will lead the election process in accordance with national policies. All members of the community will be invited and eligible to apply. ● Elected members of the DRMC are presented to the community. Locally Managed Marine Areas FIOVANA aims to support best practices for the implementation of LMMAs and coastal conservation initiatives. Working with traditional and local leaders and farmer and fisher groups, FIOVANA will promote mapping and the development of an Action Plan addressing the community’s most pressing coastal restoration and protection issues and current interventions. Where feasible, Action Plans will address restoration of coastal wetland systems to mitigate damage from storms and improve freshwater quality, and incorporate good management of marine resources through LMMAs. The strategy looks beyond conservation, shifting ownership to local anglers and creating a local community engaged in the protection of coastal and marine resources as a safeguard for their livelihoods. Key activities include: ● Engaging stakeholders to ensure their support and commitment. ● Building local leadership and organizing key stakeholders. Process Evaluation of the FIOVANA RFSA in Madagascar Annex A: FIOVANA Interventions 39 ● Developing a sustainable fishery plan with management techniques, such as gear and fishing area restrictions and adoption of legal frameworks. ● Learning from established groups and sharing local best practices through exchange visits. ● Supporting the financial sustainability of LMMAs through locally tailored investments, fee-for￾service activities, and interventions used throughout FIOVANA, including the use of savings and loans associations. IMPEL | Implementer-Led Evaluation and Learning 40 Annex B: Evaluation Questions and Methodology ANNEX B: EVALUATION QUESTIONS AND METHODOLOGY This Annex provides additional information pertaining to the methodology of this PE, including the full list of evaluation questions; further details of field data collected, including sampling selection and respondent lists; and details of how ground-truthing sessions and the joint CLA workshop were held. Evaluation Questions The overarching research question, given the intent of the performance monitoring piece is: ● How well has FIOVANA been effectively implementing core and critical pieces of its intervention in order to address the RFSA objectives of enhanced food security and resilience? Sub-questions might include: ● Are implementation activities gender and context sensitive, and inclusive of key, local partners, leading to greater community engagement and adoption? ● What considerations within the implementation plan have been most effective at minimizing friction (or disagreement between stakeholders) and facilitating progress? ● Which factors, whether contextual or programmatic, interfere with more effective implementation? Additional sub-questions/probes identified by ADRA: ● Are implementation activities resilient to climate risks (namely cyclones)? ● Relatedly, how has implementation been affected by infrastructure limitations? (i.e. roads blocked by flooding) ● What are the ways, tangible and intangible, that COVID-19 has affected implementation? Sampling Intervention Sampling Since an exhaustive tracking of each activity under each purpose was not feasible for this PE, Causal Design worked directly with BHA/Washington and IPs to determine a sample of eight activities to examine. Given the focus of the PE, the aim was to select Interventions based on their representativeness and centrality to FIOVANA and its Theory of Change. Causal Design provided a short list of interventions to ADRA, and the ADRA team selected the final eight based on their understanding of the PE and its goals. Selected interventions and their Purposes are shown in Figure 1. Process Evaluation of the FIOVANA RFSA in Madagascar Annex B: Evaluation Questions and Methodology 41 Figure 1. Selected interventions and their Purposes Respondent Sampling Causal Design used a purposive sampling approach for KIIs and validation exercises. This entailed sampling respondents across the implementation chain from each RFSA intervention, interviewing respondents from ADRA, implementing partners, community workers, and participants (through the validation exercise). Causal Design also sampled across geographies, using a maximum variance sampling approach. Sampling focused on district and commune levels based on where specific activities were carried out, and with the goal of including both littoral and interior fokontany. The goal of distributing responses by geography is to gain a comprehensive rather than idiosyncratic depiction of implementation quality. As a qualitative inquiry, this PE cannot assume a representative sample. However, the use of both purposive and maximum variance sampling ensured the relevance of selected respondents and as much breadth as possible from the respondents and fokontany selected. Key Informant Interviews The subsequent tables present the total number of KIIs interviewed for this PE. Purpose 1 Purpose 2 Purpose 3 Mother-to-Mother Groups Farmer Field Schools Impact Investment Fund Cooking Demonstrations & Tsikonina School Based Activities Farmer Market Schools Disaster Risk Management Committee Locally Managed Marine Areas IMPEL | Implementer-Led Evaluation and Learning 42 Annex B: Evaluation Questions and Methodology Table 2. ADRA District and commune staff respondents Interviewed District Selected Commune Animators Supervisors/Specialists P1 P2 P3 Total P1 P2 P3 IIF Total Manakara Ambila 1 1 2 1 1 1 3 Marofarihy 1 1 Mananjary Ambohitsara Est 1 1 2 1 1 1 1 4 Sandrohy 1 1 2 Vohipeno Savana 1 1 2 1 1 1 3 Andemaka 1 1 2 Farafangana Anosy Tsararafa 1 1 2 1 1 1 1 4 Vohimasy 1 1 Vangaindrano Mahabe 1 1 2 1 1 1 3 Masianaka 1 1 2 Vondrozo Vohimary 1 1 1 1 1 3 Vondrozo 1 1 2 21 20 Process Evaluation of the FIOVANA RFSA in Madagascar Annex B: Evaluation Questions and Methodology 43 Table 3. Fokontany-level respondents interviewed District Selected Commune Fokontany Interview Selection by Fokontany SBA CD/T M2M FFS FMS IIF DRMC LMMA Total Manakara Ambila Ambila centre 1 1 2 Loharano 1 1 2 Marofarihy Ambotaka 1 1 2 Marofarihy 1 1 2 Mananjary Ambohitsara Est Ambohitsara est 1 1 2 Andranomahitsy 1 1 Sandrohy Ambalona 1 1 2 Sandrohy 1 1 2 Vohipeno Savana Ambohitsara I 1 1 Savana 1 1 2 Andemaka Andemaka centre 1 1 2 Vohitromby 1 1 2 Farafangana Anosy Tsararafa Antavia 1 1 2 Loharano 1 1 2 Vohimasy Ambohitrova 1 1 2 Vohimasy 1 1 2 Vangaindrano Mahabe Kidilanitra 1 1 2 Mahabe centre 1 1 2 Masianaka Antavimalahay 1 1 Nosy omby 1 1 2 Vondrozo Vohimary Bemandroso 1 1 2 IMPEL | Implementer-Led Evaluation and Learning 44 Annex B: Evaluation Questions and Methodology District Selected Commune Fokontany Interview Selection by Fokontany SBA CD/T M2M FFS FMS IIF DRMC LMMA Total Ratsimanahiantefaliandro 1 1 2 Vondrozo Antevongo 1 1 2 Vondrozo 1 1 2 45 Table 4. ADRA-I, ADRA country staff and implementing partner respondents interviewed Respondent Type Number ADRA-I and Country Staff 18 Implementing Partners 3 Total 21 Validation Exercise At BHA’s request, the PE also included a validation exercise, conducted by Consultants Associates and supervised remotely. The aim of the validation was to provide additional verification that interventions are taking place according to FIOVANA’s intended cascading system. The validation exercise, while narrow in scope, nevertheless provided additional qualitative data that build on findings from the KIIs. A validation tool was developed, which entailed a series of observations or informal KIIs with participants respondents. Enumerators were instructed to conduct at least one exercise or interview activity for each intervention in a random fokontany from all nine communes sampled. This was not always possible due to challenges with availability of key respondents on the ground at the time the exercise was conducted. Table 5 shows the validation exercises conducted across all sampled communes. Process Evaluation of the FIOVANA RFSA in Madagascar Annex B: Evaluation Questions and Methodology 45 Table 5. Validation exercises conducted per intervention Key ✔ Validation exercise conducted in one fokontany ✔✔ Validation exercise conducted in two fokontany x Validation exercise not conducted n/a Validation exercise not conducted because the intervention does not occur in that commune * Validation exercise not conducted because the intervention has not been conducted ** This exercise was not on the original validation tool, but the enumerator had a chance to observe this in the field in some instances. Activity Validation Exercises District Farafangana Manakara Mananjary Vangaindrano Vohipeno Vondrozo Fokontany: Antavia Marofarihy Andranomahitsy Kidilanitra Savana Antevongo SBA Speak to: Teacher or Parent Representatives ✔ ✔ ✔ ✔ * ✔ Observe: Teacher or Parent Representative Training X X * X * X Fokontany: Loharano Marofarihy Ambohitsara Est Nosy Omby Vohitromby Bemandroso CD/T Speak to: Community members and/or caregivers of children 6 to 59 months old ✔ ✔ ✔ ✔ * ✔ ✔ Observe: Cooking session X X X X * X X Fokontany: Loharano Marofarihy Ambalona Antavimalahay Savana Ratsimanahiantefaliandro M2M Speak to: M2M members ✔ ✔ X ✔ X * Observe: M2M session, pictorial books X X X X X * Fokontany: Vohimasy Ambila Andranomahitsy Mahabe Center & Nosy Omby Savana Antevongo IMPEL | Implementer-Led Evaluation and Learning 46 Annex B: Evaluation Questions and Methodology Activity Validation Exercises District Farafangana Manakara Mananjary Vangaindrano Vohipeno Vondrozo FIES Speak to: Demonstration plot members ✔ ✔ ✔ ✔✔ ✔ ✔ Observe: Demonstration Plot tour (Priority) ✔ ✔ ✔ ✔✔ X ✔ Observe: Demonstration Plot training session X X X X X X X Fokontany: Vohibano Marofarihy Ambalona N/A Andemaka Center Vondrozo FMS Speak to: FMS members * ✔ * X ✔ * Observe: Technical session, sales tracker * X * X - * Fokontany: Ambohitrova Marofarihy Ambalona Kidilanitra Vohitromby Ratsimanahiantefaliandro DRMC Speak to: DRMC members ✔ ✔ ✔ ✔ ✔ ✔ Observe: DRMC meeting X X X X X X Fokontany: N/A N/A N/A N/A N/A N/A LMMA Speak to: Fokontany Chief * * * * * * Process Evaluation of the FIOVANA RFSA in Madagascar Annex B: Evaluation Questions and Methodology 47 Ground-Truthing Sessions After all the primary data had been collected and analysis was underway, Causal Design organized a series of ground-truthing sessions on May 24 and 25, 2022, to share preliminary findings with ADRA technical teams and key staff. The aim of the sessions was to verify findings and obtain additional detail and context where necessary. As such, ground-truthing sessions served in part as a data collection exercise in themselves for Causal Design and proved highly fruitful. Sessions were held remotely on Zoom, with simultaneous translation, and lasted between 1 and 1.5 hours. Sessions were divided by Purpose and Intervention and purpose as follows: ● Session 1. M2M Groups, Cooking Demonstrations/Tsikonina, and School-Based Approach (Purpose 1 & Cross-Cutting) ● Session 2. Farmer Field Schools, Farmer Market School, and Impact Investment Fund (Purpose 2) ● Session 3. DRMCs (Purpose 3) Collaborating, Learning, and Adapting Workshop Causal Design organized and facilitated a 4-day joint CLA workshop, attended by technical teams from both ADRA and CRS. Each day included plenary presentations and discussions as well as breakout sessions. The majority of the discussion took place in French or Malagasy, with simultaneous translation for English-only speakers. The workshop was held between June 13 and June 16, 2022, after BHA and ADRA had provided an initial review of the FIOVANA PE report. Central themes for each of the four days were as follows: ● Day 1. Reflecting on and unpacking PE findings and recommendations ● Day 2. Identifying next steps/actions for incorporating process monitoring insights into activities ● Day 3. Discussing cross-activity learnings and adaptations ● Day 4. Chart the path forward and make action plans for adaptations and integration into other RFSA learning and M&E processes The CLA workshop, while primarily intended to facilitate the creation of action plans following the PE, also provided additional context for findings and refinement for recommendations, which were integrated into this PE report. FIOVANNA Process Monitoring – ADRA Implementation Staff & Partner KII Tool Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Causal Design, a research organization conducting a process monitoring evaluation of ADRA’ FIOVANA program. Specifically, we are examining the implementation of FIOVANA projects and factors that might be affecting implementation of these activities, either positively or negatively. We are specifically examining the following FIOVANA activities: ● School Based Activities (SBA) ● Mother to Mother Care Groups (M2M) ● Cooking Demonstrations & Tsikonina ● Farmer Field Schools (FFS) ● Farmer Market Schools (FMS) ● Impact Investment Fund (IIF) ● Locally Managed Marine Areas (LMMA) ● Disaster Risk Management Committees (DRMC) Goal of this interview: We have invited you to this interview as one of FIOVANA’s key implementers. We would like to hear your perspective on how the FIOVANA support has been delivered, and your experiences implementing activities under the FIOVANA program. We would like to hear about both facilitators and obstacles, with the aim of ascertaining how implementation could be improved in the future. We would also like to hear about the inclusion of women, youth, and people with disabilities; and effects of the climate events/cyclones and COVID-19 on program/activity implementation and activities. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? IMPEL | Implementer-Led Evaluation and Learning 48 Annex C: Data Collection Tool ANNEX C: DATA COLLECTION TOOLS [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Respondent Type Delete as appropriate ● ADRA CoP / DCoP / Learning Lead / Technical Advisors / Country Leads / Purpose Leads ● FIANTSO / FHI 360 / AVSF / MIHARI / TANGO Number of Respondents Interviewee Name/s Gender District Leave blank if not relevant Commune Leave blank if not relevant Fokontany Leave blank if not relevant Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 49 Interview Questions Background I’d like to start by hearing some background information about your involvement in the FIOVANA program. 1. What is your particular role within FIOVANA and how long have you been involved? 2. Which components of FIOVANA are you particularly involved in? Note to Enumerators: Probe for our select 8 activities above and concentrate on these. 3. From your understanding, what is the main purpose of your involvement and activities/[key activity] as part of the FIOVANA program? 3.1. What are your key roles and tasks? 3.2. What are your objectives as part of the FIOVANA program? 3.3. Do you think these tasks and objectives are reasonable given the context you/FIOVANA works in, and are you optimistic/pessimistic about realizing them? ■ Why?/Why not? Effectiveness of ADRA Support Note to Enumerators: Skip this section for ADRA CoP/Purpose Lead/Learning Lead/Technical Leads 4. What support for implementation have you received from ADRA so far? What did this consist of? (e.g. trainings, material or financial support) 4.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? Please be as specific as possible. (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing activities as part of the FIOVANA program. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 5. How did you make the transition from planning FIOVANA activities (thinking about them ‘in theory’) to their actual implementation? What are the essential, concrete ‘next steps’ involved in putting activities into practice? IMPEL | Implementer-Led Evaluation and Learning 50 Annex C: Data Collection Tool 5.1. Which steps or actions during this stage do you think are fundamental to making implementation work best? How have these steps/actions played out under FIOVANA? 5.2. What aspects of the transition from planning to actual implementation worked well, and which were more complicated? Why? How could they be improved? 6. What are some of the key considerations, in particular with regards to the communities where activities take place, that you included in your preparations to ensure implementation runs smoothly? (E.g. gender dynamics, poverty, migration, terrain, focus of economy (ie. agriculture, fishing)) 6.1. How do you accommodate them/factor them into your planning, or implementation now? Can you be specific? 7. Thus far, what have been some of your priorities, or requirements, for start-up/laying the groundwork for FIOVANA implementation? (E.g. securing partnerships, recruitment, establishing lines of communication, procurement) 7.1. What aspects of start-up/laying the groundwork has worked well, and which have been more complicated? Why? How could they be improved? 8. Other than these, what are your other critical priorities or requirements for implementing or rolling out your key activities? (e.g. communication, logistics, coordinating with communities) 8.1. Which aspects of establishing these have worked well, and which were more complicated? Why? How could these be improved? Adaptations 9. How does your implementation in practice compare to how you expected or planned it would roll out? 9.1. Can you give any specific examples of changes or adaptations made? 9.2. In particular, how did COVID-19 affect your implementation 10. What were the key outcomes of the refine-and-implementation period? 10.1. What specific adaptations did you make based on your learnings/observations during this phase? 10.2. If you could not adapt, why not? What resources or other support were you missing? 11. Are there any other expected or unexpected local conditions or obstacles that have affected implementation thus far? Please describe them. How did you adjust? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 51 11.1. If you did not/could not adapt, why not? What resources or other support were you missing? 12. Have you been unable to conduct any activities you wanted or that were planned? Why? 13. Other than the refine and implementation (R&I) period, what systems are currently in place to continue to adapt or ensure implementation? What do these look like? 13.1. Are there any examples of this in practice? 13.2. For Learning Lead: How are changes or ground level factors from the field incorporated into the ToC? Climate/Cyclone 14. What are the effects of the cyclone on your FIOVANA activities/implementation? Probe for both implementation and outcomes. 15. What changes of adaptations have had to make to the way you implement because of the cyclones? Why? 15.1. Have these adaptations been effective? 16. Were there any aspects of the implementation or activities that were completely derailed by the cyclones? Which ones, and how? 17. For activity-specific respondents only: In your opinion, what were the most important aspects or contributions of your activities in the cyclone context? 18. For all respondents: What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Linkages 19. Are there any other organizations or actors (local, formal or informal, like chiefs or women’s groups, government bodies, businesses or even other aid agencies) that are critical to you being able to effectively carry out your work? 19.1. Who/what are they, and how are they essential to effective implementation? 20. How effective would you describe these links with key partners at the moment? 20.1. What are some effective elements of these partnerships? What do you consider important for ensuring you work together to implement activities effectively? 20.2. What specific outcomes have emerged out of these partnerships? What factors were responsible for these outcomes? Please be as specific as possible. IMPEL | Implementer-Led Evaluation and Learning 52 Annex C: Data Collection Tool 21. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Gender 22. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e, influence over household decisions). In what areas of household decision-making do women have more or less influence or a say? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for FIOVANA under the IMPEL/Implementer￾Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 23. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for both men and women? 23.1. What does men and women’s participation in activities currently look like? In what aspects do they participate, and in how do they participate? ■ How does participation differ between men and women? 23.2. What considerations are important to ensuring equal access for both men and women when implementing programs (not just designing them!)? 23.3. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 24. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 24.1. What does PWD participation in activities currently look like? In what aspects do they participate, and in how do they participate? 24.2. What considerations are important to ensuring equal access for PWDs? 24.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 25. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for youth? Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 53 25.1. What does youths’ participation in activities currently look like? In what aspects do they participate, and in how do they participate? ■ How does participation differ for young men and young women? 25.2. What considerations are important to ensuring equal access for youth? 25.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to say or ask before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Alex Porter (alex.porter@causaldesign.com) or Matt Klick (matthew.klick@causaldesign.com). Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 54 Annex C: Data Collection Tool FIOVANA Process Monitoring – Cooking Demonstrations/Tsikonina KII Tool Background Information for Enumerators Note for Enumerators: This tool is to be used for either Cooking Demonstrations or Tsikonina. The tool is constructed to be able to be used for both activities, but each interview should be focused on only one of the two activities. Cooking Demonstrations/Tsikonina Cooking Demonstrations Cooking demonstrations contribute to FIOVANA’s goal to sustainably improve the health and nutritional status of women of reproductive age, adolescent girls and children under five. Cooking demonstrations are a community-led activity promoting the use of locally available or accessible foods. The objectives are to: - Assist households to integrate nutritional recommendations into daily food processing and preparation - Provide households with the knowledge, skills and confidence to better utilize locally available and affordable foods to improve their diet and nutritional status - To impart the basics of food safety and hygiene, such as hand washing, covering prepared food, and proper handling and storage of food with available utensils The field agent and Community Nutrition Agent (CNA) will be trained with nutrition education first, then trained in the technique of facilitating cooking demonstrations, including the evaluation of experiences (taste, smell, appearance, and color of the meals). These trainings will help to cooking sessions run smoothly, and are truly participatory, going beyond the demonstration and encouraging everyone's active participation. Field agents and CNA will be provided with a recipe booklet, provided by the national nutrition office, to help them design appropriate dishes according to age and each session’s theme. Religious customs on food preparation should also be taken into consideration, though this may pose a challenge in mixed-religion groups. The health and nutrition team will organize one cooking demonstration per month per fokontany and sessions will be associated with monthly growth monitoring to identify impact and motivate participants. The Cooking Demonstration can also be associated with the session at FFS (Farmers Field Schools), such as agricultural fairs. All community members are invited to attend the session and learn. Apart from the close collaboration with the Community Nutrition Agents, local authorities and traditional leaders, the involvement of men is key to the success of this approach. The number of participants per session should not exceed 10, in order to increase sessions’ effectiveness, manageability, and the chances of everyone learning something. Participants are asked to bring their own ingredients to sessions. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 55 Cooking Demonstrations/Tsikonina Tsikonina Tsikonina is a community-based nutrition program for children aged 6–59 months (about 5 years) suffering from moderate malnutrition and at risk of developing malnutrition. Tsikonina provides an opportunity for neighborhood caregivers to come together and learn from each other about best practices to keep children healthy and well-nourished. Community health volunteers and ADRA project field agents lead the activity and there is linkage with health center in case child doesn’t recover and reference to seek a professional care is needed. Tsikonina depends heavily on food produce supplied by other project sectors, and is based the Positive Deviance/Hearth (PD/Hearth) model. As part of this model, positive deviant practices (defined as practices that stray from the norm, but in a beneficial way) are first identified. Once identified, a selection of recipes is prepared by field staff based on the recipes of the positive deviant mothers. If necessary, their recipes are modified to include more ingredients, more protein, or ingredients more accessible to every local mother. Once recipes are developed, women in the community meet over the course of 6 or 9 days to prepare the selected recipes for their children and discuss a key health message. Each community selects the session length and time of day (typically either 8am or 2pm) that is best for their situation, and caregivers meet each day with their children. Each session includes cooking a nutritious recipe together, feeding the children, and discussing a key health message. Every site should have an accessible latrine and tippy tap, to promote good hygiene habits. Each session should be under 2 hours long. On the last day, children’s’ weight is compared with their initial weight on the first day. Mothers are then encouraged to continue following the practices that they learned at home, with follow-up visits to their home for two weeks. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including Cooking Demonstrations/Tsikonina. We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the Cooking Demonstrations/Tsikonina and FIOVANA’ support for Cooking Demonstrations/Tsikonina, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing activities as members of Cooking Demonstrations/Tsikonina; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in Cooking Demonstrations/Tsikonina; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions IMPEL | Implementer-Led Evaluation and Learning 56 Annex C: Data Collection Tool and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with FIOVANA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Activity Type ● Cooking Demonstrations ● Tsikonina Respondent Type ● Community Nutrition Volunteer (CNV) ● Community Health Volunteer (CHV) Number of Respondents Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 57 Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about Cooking Demonstrations/Tsikonina: 1. From your understanding, what is the main purpose of the Cooking Demonstrations/Tsikonina? 2. What would you consider main tasks/activities for the Cooking Demonstrations/Tsikonina to be? 2.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (i.e. Because they are most effective, most relevant/needed for the community?) Effectiveness of FIOVANA Support 3. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 3.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing activities through Cooking Demonstrations/Tsikonina: IMPEL | Implementer-Led Evaluation and Learning 58 Annex C: Data Collection Tool Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 4. Can you briefly describe what implementation of your key tasks/activities for the Cooking Demonstrations/Tsikonina looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 5. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 5.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 5.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 5.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 6. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 6.1. Can you give any specific examples of how these challenges affect your ability to implement? 6.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 6.3. How might these challenges be overcome in the future? Climate/Cyclones 7. How has the cyclone/s affected your ability to implement activities? 7.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? 8. How did the cyclone affect the effectiveness, impact, and potential outcomes of this Cooking Demonstrations/Tsikonina activities? Can you give specific examples? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 59 8.1. Did beneficiaries’ or members’ participation with this Cooking Demonstrations/Tsikonina change because of the cyclone? How and why? 9. In your opinion, what were the most important aspects or contributions of this Cooking Demonstrations/Tsikonina in the cyclone context? Why? 9.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why? Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 10. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out Cooking Demonstrations/Tsikonina activities? 10.1. How effective would you describe these links with key partners at the moment? Why? 10.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 10.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 10.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Now I would like to ask about the participation and inclusion of both genders, youth, and people with disabilities (PWD) in Cooking Demonstrations/Tsikonina. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: 11. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household IMPEL | Implementer-Led Evaluation and Learning 60 Annex C: Data Collection Tool decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 12. What does men and women’s participation in Cooking Demonstrations/Tsikonina currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 12.1. How does participation differ between men and women? Can you give any specific examples? 13. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 13.1. How successful do you consider the efforts to include both men and women in Cooking Demonstrations/Tsikonina to be successful? Why/why not? 13.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 14. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 14.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 14.2. What considerations are important to ensuring equal access for PWDs? 14.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 15. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. 15.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for young men and young women? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 61 15.2. What considerations are important to ensuring equal access for youth? 15.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119) Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 62 Annex C: Data Collection Tool FIOVANA Process Monitoring - Disaster Risk Management Committees (DRMC) KII Tool Background Information for Enumerators Disaster Risk Management Committees FIOVANA will use Disaster Risk Management Committees (DRMCs) to empower community members by building social capital, linking them to government and other stakeholders for strategic relationship building. ADRA initiated the process of cultivating government and stakeholders’ support during the design of the program, and will continue engaging leaders early to ensure support for active community involvement. In places where DRMCs do not exist, FIOVANA DRM agents will establish them, identifying motivated natural leaders representing youth, women, and other underrepresented groups. In places where DRMCs exist, we will evaluate the strength and diversity of committees and make recommendations. The program will provide training to DRMCs on their roles and responsibilities, community mobilization and motivation techniques, and the importance of active participation of underrepresented groups. FIOVANA will coordinate with the BNGRC to strengthen fokontany and commune-level DRMCs. To encourage sustainability and local ownership, FIOVANA will take the following actions (based on lessons learned from previous DRMC programs): ● Presentation of the FIOVANA program to the community, including the goal, purposes, expected outcomes, and interventions. The presentation day will be chosen in accordance with local leaders, traditional authorities, and the fokontany. All social classes will be invited to participate as part of a local development approach. ● The concept of Disaster Risk Management and Reduction will be presented to the community and will include a discussion on the perception of local threats and risks. ● An explanation of the election process and responsibilities of DRMC members and the community, encouraging community ownership at the outset. ● The chief de fokontany will lead the election process, in accordance with national policies. All members of the community will be invited and eligible to apply. ● Elected members of the DRMC are presented to the community. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including Disaster Risk Management Committees (DRMC). We specifically are examining how FIOVANA Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 63 projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the DRMCs and FIOVANA’ support for DRMCs, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing activities as members of a DRMC; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in DRMCs and their activities; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ IMPEL | Implementer-Led Evaluation and Learning 64 Annex C: Data Collection Tool Respondent Information Respondent Type ● Fokontany Chief ● Vice-Chairman of DRMC Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Leave blank for commune-level respondents Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about DRMCs: 16. From your understanding, what is the main purpose of the DRMC? 17. What would you consider your main tasks/activities for the DRMC to be? 17.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (ie. Because they are most effective, most relevant/needed for the community?) Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 65 Effectiveness of FIOVANA Support 18. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 18.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing activities through this DRMC: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 1. Can you briefly describe what implementation of your key tasks/activities for the DRMC looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 2. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 2.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 2.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 2.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 3. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 3.1. Can you give any specific examples of how these challenges affect your ability to implement? 3.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 3.3. How might these challenges be overcome in the future? IMPEL | Implementer-Led Evaluation and Learning 66 Annex C: Data Collection Tool Climate/Cyclones 4. How has the cyclone/s affected your ability to implement activities? 4.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? 5. How did the cyclone affect the effectiveness, impact, and potential outcomes of this DRMC’s activities? Can you give specific examples? 5.1. Did beneficiaries’ or members’ participation with this DRMC and their activities change because of the cyclone? How and why? 6. In your opinion, what were the most important aspects or contributions of DRMC in the cyclone context? Why? (Probe for activation and use of Early Warning Systems) 6.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why? Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 7. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out DRMC activities? 7.1. How effective would you describe these links with key partners at the moment? Why? 7.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 7.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 7.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 67 Inclusivity Now I would like to ask about the participation and inclusion of both genders, youth, and people with disabilities (PWD) in this DRMC. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: 8. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 9. What does men and women’s participation in this DRMC’s activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 9.1. How does participation differ between men and women? Can you give any specific examples? 10. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 10.1. How successful do you consider the efforts to include both men and women in DRMC activities to be successful? Why/why not? 10.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 11. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 11.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 11.2. What considerations are important to ensuring equal access for PWDs? IMPEL | Implementer-Led Evaluation and Learning 68 Annex C: Data Collection Tool 11.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 12. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. 12.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for young men and young women? 12.2. What considerations are important to ensuring equal access for youth? 12.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119). Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 69 FIOVANA Process Monitoring – Farmer Field School (FFS) KII Tool Background Information for Enumerators Farmer Field School The Farmer Field School (FFS) is an approach where producers are empowered on improved production practices using demonstration sites to strengthen their production outcomes. A group of farmers (15-25 farmers) are supported through the entire production cycle with all trainings following the physiological growth stage of the produce/product of focus. Demonstration sites are used as avenues to experiment and learn about the improved practices and their likely effects. For FIOVANA, ADRA will modify the traditional model based on experience in Madagascar which reveals that requesting farmers form groups is not always sustainable and can negatively affect group performance. We will use a phased approach to identify and work with already existing groups of farmers and gradually expanding participants and groups based on interest and need. Phased Approach: In the first production year, field agents will engage groups in participatory, hands-on learning, with all members receiving training and follow-up support at least twice a month. Members will be supported to develop their own demonstration plots. During the second production year, Field Agents will shift work to monitoring and reinforcement with support from animators and the Farmer Leader volunteers, including on-farm exchange visits for cross learning. Field Agents and animators will then focus on forming a second phase of farmer groups. All farmer groups will be connected with commune-level early warning system (EWS) field agents for information on weather, flood and cyclone alerts. At the same time, FIOVANA will seek other ways to improve information flows between critical actors such as value chain participants, government representatives, farmers groups and cooperatives, and between households and groups at the local level within communities themselves. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including FFS. We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the Farmer Field Schools and FIOVANA’ support for Farmer Field Schools, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s IMPEL | Implementer-Led Evaluation and Learning 70 Annex C: Data Collection Tool support has been delivered; your experiences implementing activities as members of a Farmer Field School; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in Farmer Field School and their activities; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 71 Respondent Information Respondent Type ● Leader Farmers Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about Farmer Field Schools: 19. From your understanding, what is the main purpose of this Farmer Field School? 20. What would you consider What would you consider main tasks/activities for the Farmer Field School to be? 20.1. If more than one activity specified: Which of these task/activities do you consider the most important for reaching your goals? Why? (ie. Because they are most effective, most relevant/needed for the community?) Effectiveness of FIOVANA Support 21. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 21.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) IMPEL | Implementer-Led Evaluation and Learning 72 Annex C: Data Collection Tool Effectiveness of Implementation Now I would like to hear about your experiences implementing activities through this Farmer Field School: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 22. Can you briefly describe what implementation of your key tasks/activities for the Farmer Field School looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 23. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 23.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 23.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 23.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 24. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 24.1. Can you give any specific examples of how these challenges affect your ability to implement? 24.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 24.3. How might these challenges be overcome in the future? Climate/Cyclones 25. How has the cyclone/s affected your ability to implement activities? 25.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 73 26. How did the cyclone affect the effectiveness, impact, and potential outcomes of this Farmer Field School’s activities? Can you give specific examples? 26.1. Did beneficiaries’ or members’ participation with this Farmer Field School and their activities change because of the cyclone? How and why? 27. In your opinion, what were the most important aspects or contributions of this Farmer Field School in the cyclone context? Why? 27.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 28. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out Farmer Field School activities? 28.1. How effective would you describe these links with key partners at the moment? Why? 28.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 28.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 28.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Now I would like to ask about the participation and inclusion of both genders, youth, and people with disabilities (PWD) in this Farmer Field School. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: IMPEL | Implementer-Led Evaluation and Learning 74 Annex C: Data Collection Tool 29. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 30. What does men and women’s participation in this Farmer Field School’s activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 30.1. How does participation differ between men and women? Can you give any specific examples? 31. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 31.1. How successful do you consider the efforts to include both men and women in Farmer Field School activities to be successful? Why/why not? 31.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 32. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 32.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 32.2. What considerations are important to ensuring equal access for PWDs? 32.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 33. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 75 33.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for young men and young women? 33.2. What considerations are important to ensuring equal access for youth? 33.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119). Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 76 Annex C: Data Collection Tool FIOVANA Process Monitoring – Farmer Market School (FMS) KII Tool Background Information for Enumerators Farmer Market School ADRA will use its own Farmer Market School (FMS) approach with poor and extremely poor agricultural and fishing households. ADRA’s FMS approach operates on similar principles of discovery learning as used in Farmer Field Schools (FFS), in that smallholder farmers gain knowledge and skills on how to explore markets, what the market can offer and how to make market decisions. FMS methods focus on experiential, practical, collective and adult learning methods. ADRA will coordinate with private sector actors to improve farming households’ understanding and participation in market functions. FIOVANA will focus on leveraging the private sector to improve the enabling environment. FMS Agents will tailor sessions to the needs of program participants and communities. The focus in FMS sessions with IGA groups will be on empowering participants to understand the demands of the market and knowing how to act. The FMS approach is based on adult education principles and includes discovery learning that is applied to the market. The goal is for the smallholder farmers to understand the market needs, the value chain flow, value of produce along the value chain, and available sales points to inform their production decision. This is to ensure producers are meeting the needs of the market. The farmers will literally travel to meet the value chain actors, communicate with them to get all the information they require to make their own production decisions. The FMS process consists of: ● Firstly,a context analysis and community meetings to explain the FMS approach. ● After this, group of 20-25 volunteer farmers form an FMS group that goes through 14 facilitated technical sessions. Typically, sessions last half- a-day and take place weekly. ● Following the technical training sessions, the FMS trainees then start to travel along value chains of their preferred crops to collect market intelligence. This is done by themselves and on their own cost. ● Farmers will use the market information to make production decisions while securing sales agreements, as much as possible. ● Finally, ADRA tracks sales made by the farmers. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 77 We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including FMS. We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the Farmer Market School and FIOVANA’ support for Farmer Market Schools under the FIOVANA program, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing activities as members of a Farmer Market School; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in Farmer Market Groups and their activities; and effects of climatic events (i.e. cyclone(s)) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English IMPEL | Implementer-Led Evaluation and Learning 78 Annex C: Data Collection Tool Enumerator/s Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Respondent Type ● FMS Management Committee Member(s) (President, Advisor, or Communication/Marketing Manager) Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about the Farmer Market Schools: 1. From your understanding, what is the main purpose of this Farmer Market School? 34. What would you consider main tasks/activities for the Farmer Field School to be? 1.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (ie. Because they are most effective, most relevant/needed for the community?) Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 79 Effectiveness of FIOVANA Support 3. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 3.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing Farmer Market School activities: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 4. Can you briefly describe what implementation of your key tasks/activities for the Farmer Market School looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 5. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 5.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 5.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 5.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 6. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 6.1. Can you give any specific examples of how these challenges affect your ability to implement? 6.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 6.3. How might these challenges be overcome in the future? IMPEL | Implementer-Led Evaluation and Learning 80 Annex C: Data Collection Tool Climate/Cyclones 7. How has the cyclone/s affected your ability to implement activities? 7.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? 8. How did the cyclone affect the effectiveness, impact, and potential outcomes of this Farmer Market School’s activities? Can you give specific examples? 8.1. Did beneficiaries’ or members’ participation with this Farmer Market School and their activities change because of the cyclone? How and why? 9. In your opinion, what were the most important aspects or contributions of this Farmer Market School in the cyclone context? Why? 9.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 10. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out Farmer Market School activities? 10.1. How effective would you describe these links with key partners at the moment? Why? 10.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 10.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 10.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 81 Inclusivity Now I would like to ask about the participation and inclusion of both genders, youth, and people with disabilities (PWD) in this Farmer Market School’s activities. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: 11. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (ie, influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 12. What does men and women’s participation in this Farmer Market School’s activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 12.1. How does participation differ between men and women? Can you give any specific examples? 13. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 13.1. How successful do you consider the efforts to include both men and women in Farmer Market School activities to be successful? Why/why not? 13.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 14. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 14.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 14.2. What considerations are important to ensuring equal access for PWDs? IMPEL | Implementer-Led Evaluation and Learning 82 Annex C: Data Collection Tool 14.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 15. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. 15.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for young men and young women? 15.2. What considerations are important to ensuring equal access for youth? 15.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119). Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 83 FIOVANA Process Monitoring – Impact Investment Fund (IIF) KII Tool Background Information for Enumerators Impact Investment Fund FIOVANA is targeting identified value chains within the broader market that have potential for positively impacting vulnerable people, and seeks to address “market shortcomings” on both ends of these value chains. Activities such as FFS, FMS and IGAs focus on production, producers and their capacities/resources. By contrast, the IIF addresses weaknesses among businesses/MSMEs further down the value chain (e.g. buyers, transporters) and those businesses that are not directly “in” the value chain but can provide value (e.g. input suppliers, processors, service providers), that result in vulnerable target groups being better served. As a concrete example, the IIF could aim to support buyers/transporters who are limited to only buying small amounts due to the small carrying capacity of their vehicle. The IIF targets businesses which work directly with one of the identified FIOVANA value chains and are determined to be in a position to positively impact beneficiaries. The IIF team works directly and through other FIOVANA team members to identify these businesses. The IIF team then examines the businesses to assess the appropriateness of investment with them (e.g. checking their history of operation, willingness to co-invest, willingness to co-invest in a venture that would impact FIOVANA’s target group). Businesses may also seek out the IIF to provide support. Applications are submitted and evaluated by a committee. IIF then works with approved businesses to develop business plans, based on co-investment both from the business and IIF. IIF conducts follow-ups to ensure the businesses are operating as agreed. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including the Impact Investment Fund (IIF). We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. Goal of this interview: We have invited you to this interview to share your views on the work of the Impact Investment Fund (IIF) and FIOVANA’ support for the Impact Investment Fund, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences as partners in the Impact Investment Fund; and facilitators and obstacles you have encountered while partnering with the Impact Investment Fund. IMPEL | Implementer-Led Evaluation and Learning 84 Annex C: Data Collection Tool We would also like to hear about your customer base; and effects of climatic events (i.e. cyclone/s) on your business and on implementation of the IIF grant. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Respondent Type ● IIF business co-investment partner Number of Respondents Interviewee Name/s Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 85 Respondent Type ● IIF business co-investment partner Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Leave blank for commune-level respondents Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about you experience with Impact Investment Fund: 35. From your understanding, what is the main purpose of the Impact Investment Fund? 36. What would you consider your role in the IIF co-investment to be? (Probe for: how will/do you use the investment? who will benefit from the investment? Effectiveness of Implementation Now I would like to hear about your experiences partnering with the Impact Investment Fund: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 37. Can you briefly describe the process of applying and receiving your IIF investment looks like? 37.1. How did you hear about the IIF investment opportunity? IMPEL | Implementer-Led Evaluation and Learning 86 Annex C: Data Collection Tool 37.2. What did the process for applying for the IIF investment look like? Do you feel it was easy or hard? (Probe for explanation and details) 37.3. What did the process of receiving the IIF investment look like? Do you feel it was easy or hard? (Probe for explanation and details) 38. What support for using the IIF investment have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 38.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. terms of the agreement; co-investment requirements; any instances of delays) Note for enumerators: Probe for specifics around delays if cited by respondent 39. What kind of challenges have you encountered while using your IIF investment (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 39.1. Can you give any specific examples of how these challenges affect your ability to use the investment? 39.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 39.3. How might these challenges be overcome in the future? Climate/Cyclones 40. How has the cyclone/s affected your ability to use your IIF investment? 40.1. What changes or adaptations have you had to make to the way you run your business because of the cyclone? 41. How did the cyclone affect the how you use the IIF investment? Can you give specific examples? 41.1. Did your customers’ needs change because of the cyclone? How and why? Linkages 42. What other actors or businesses (e.g suppliers, distributors, government), etc.) do you work with as part of your business operations? 42.1. What specific outcomes or benefits have emerged out of these partnerships? Can you give any specific examples? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 87 42.2. Have you received an investment from any other organizations? (Probe for: grants or funds received from NGOs or any other organizations) 43. Has the IIF investment affected your partnerships in any way? 43.1. In what ways has the IIF investment affected your partnerships (Probe for: new partnerships, strengthening or weakening in existing partnerships) Inclusivity 44. Who do you consider main customers for your business? 44.1. Has your customer base changed since you received the IIF investment? (Probe for: increase or decrease in customers, new customers) ■ Is this change because of or related to the IIF investment? ■ What types of new customers have been using your services/products? 45. Do you have more men or women customers? What portion of your customer base is female? 45.1. Have you made an effort to increase your women customers? Why or why not? 45.2. What are the barriers to increasing your women customers? 46. Does your business have any youth customers (under the age of 29)? What portion of your customer base is youth? 46.1. Have you made an effort to increase your youth customers? Why or why not? 46.2. What are the barriers to increasing your youth customers? 47. Does your business have and customers that have disabilities? What portion of your customer base is people with disability? 47.1. Have you made an effort to increase your ability to serve customers with disabilities? Why or why not? 47.2. What are the barriers to increasing your ability to serve customers with disability? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119) Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 88 Annex C: Data Collection Tool FIOVANA Process Monitoring – Locally Managed Marine Area (LMMA) KII Tool Background Information for Enumerators Locally Managed Marine Area FIOVANA aims to support best practices for the implementation of Locally Managed Marine Areas (LMMAs) and coastal conservation initiatives. Working with traditional and local leaders, and farmer and fisher groups, FIOVANA will promote mapping and the development of an Action Plan addressing community’s most pressing coastal restoration and protection issues and current interventions. Where feasible, Action Plans will address restoration of coastal wetland systems to mitigate damage from storms and improve freshwater quality, and incorporate good management of marine resources through LMMAs. The strategy looks beyond conservation, shifting ownership to local fishers and creating a local community engaged in the protection of costal and marine resources as a safeguard for their livelihoods. Key activities include: • Engaging stakeholders to ensure their support and commitment. • Building local leadership and organizing key stakeholders. • Developing a sustainable fishery plan with management techniques, such as gear and fishing area restrictions and adoption of legal frameworks. • Learning from established groups and sharing local best practices through exchange visits. • Supporting the financial sustainability of LMMAs through locally tailored investments, fee￾for-service activities, and interventions used throughout FIOVANA including the use of savings and loans associations (SLAs). Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including Locally Managed Marine Areas (LMMAs). We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the Locally Managed Marine Areas and FIOVANA’ support for Locally Managed Marine Areas, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing activities as members of a Locally Managed Marine Area; and facilitators and obstacles you have encountered while implementing Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 89 these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in Locally Managed Marine Areas and their activities; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ IMPEL | Implementer-Led Evaluation and Learning 90 Annex C: Data Collection Tool Respondent Information Respondent Type ● Fokontany Chief Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Leave blank for commune-level respondents Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about Locally Managed Marine Areas: 1. From your understanding, what is the main purpose of the Locally Managed Marine Area? 2. What would you consider your main tasks/activities for the LMMA to be?? (Probe for: formation of Locally Managed Marine Area plans; meetings commune-level committee counterparts) 2.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (ie. Because they are most effective, most relevant/needed for the community?) Effectiveness of ADRA Support 3. What support for implementation have you received from ADRA so far? What did this consist of? (e.g. trainings, material or financial support) Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 91 3.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing activities as part of the Locally Managed Marine Area: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 48. Can you briefly describe what implementation of your key tasks/activities for the LMMA looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 49. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 49.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 49.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 49.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 50. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 50.1. Can you give any specific examples of how these challenges affect your ability to implement? 50.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 50.3. How might these challenges be overcome in the future? Climate/Cyclones 7. How has the cyclone affected your ability to implement activities? IMPEL | Implementer-Led Evaluation and Learning 92 Annex C: Data Collection Tool 7.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? 8. How did the cyclone affect the effectiveness, impact, and potential outcomes of this Locally Managed Marine Area’s activities? 8.1. Did beneficiaries’ or members’ participation with this Locally Managed Marine Area and their activities change because of the cyclone? How and why? 9. In your opinion, what were the most important aspects or contributions of this Locally Managed Marine Area in the cyclone context? 9.1. What were the most important aspects of the FIOVANA program as a whole as a whole in the cyclone context? Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 10. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out LMMA activities? 10.1. How effective would you describe these links with key partners at the moment? Why? 10.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 10.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 10.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 93 11. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 12. What does men and women’s participation in this LMMA’s activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 12.1. How does participation differ between men and women? Can you give any specific examples? 13. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 13.1. How successful do you consider the efforts to include both men and women in LMMA activities to be successful? Why/why not? 13.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 14. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 14.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 14.2. What considerations are important to ensuring equal access for PWDs? 14.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 15. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. 15.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? IMPEL | Implementer-Led Evaluation and Learning 94 Annex C: Data Collection Tool ■ How does participation differ for young men and young women? 15.2. What considerations are important to ensuring equal access for youth? 15.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119) Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 95 FIOVANA Process Monitoring – Mother to Mother Care Groups (M2MCG) KII Tool Background Information for Enumerators Mother to Mother Care Groups Mother to Mother Care Groups (M2MCG or M2M) are groups of women who come together to discuss issues on MIYCN including challenges and how they resolve them. They support one another on issues of maternal nutrition and IYCF (Infant and Young Child Feeding). These mothers are recruited by Community Health Volunteers (CHVs) based on updated register information from the mapping. The groups constitute 10-15 participants and meet at least twice a month during project start and once a month after CHV visits. M2M sessions are facilitated by CHVs/Community Nutrition Volunteers (CNVs), to discuss a wide range of nutrition and health topics. There are also specialized M2M groups of pregnant adolescents, aged 15-19, to ensure adolescent mothers receive adequate peer support and can discuss issues specific to their cohort. A sub-section of women in the M2Ms who practice positive health and nutrition behaviors are classified as “Model Mothers” (MMs). MMs support bi-weekly group meetings by conducting targeted monthly home visits, to reinforce learning and practice. MMs are given pictorial books of key practices developed by the ONN and adapted for the regional context. Linking with Fiovana’s Gender component, during home visits, men, boys, and other members of the family are encouraged to participate in sessions. Anticipated outcomes include deeper engagement of men in childcare practices and reduced burden on women’s time burden. During quarterly meetings, MMs will gather with COSANS, CHVs/CNVs and other stakeholders to discuss progress towards adopting health and nutrition practices, barriers, and solutions. As a complement to the M2M, the influencer groups of Community Mother support groups (CMs) have the explicit role of supporting M2Ms by collaborating with COSANS and CSBs; mobilizing communities to participate in health promotion activities; ensuring the M2Ms are supported; tracking health and nutrition of mothers and CU5s (Children Under 5) and reporting problem cases to CHV/CNVs. CMs organize community action days every quarter for the whole community to celebrate achievements and reinforce awareness. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA’s FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including M2M Care Groups. We specifically are examining how FIOVANA projects are being IMPEL | Implementer-Led Evaluation and Learning 96 Annex C: Data Collection Tool implemented in practice, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the Care Groups and FIOVANA’ support for Care Groups, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing activities as members of a M2M Care Group; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in M2M Care Groups and their activities; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with FIOVANA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 97 Respondent Information Respondent Type ● Model Mother Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Fokontany Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about M2M Care Groups: 51. From your understanding, what is the main purpose of this M2M Care Group? 52. What would you consider main tasks/activities for the M2M Care Group to be? 52.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (i.e. Because they are most effective, most relevant/needed for the community?) Effectiveness of FIOVANA Support 53. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 53.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) IMPEL | Implementer-Led Evaluation and Learning 98 Annex C: Data Collection Tool Effectiveness of Implementation Now I would like to hear about your experiences implementing activities through this M2M Care Group: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 54. Can you briefly describe what implementation of your key tasks/activities for the M2M Care Group looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) 55. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 55.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 55.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 55.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 56. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 56.1. Can you give any specific examples of how these challenges affect your ability to implement? 56.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 56.3. How might these challenges be overcome in the future? Climate/Cyclones 57. How has the cyclone/s affected your ability to implement activities? 57.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 99 58. How did the cyclone affect the effectiveness, impact, and potential outcomes of this M2M Care Group’s activities? Can you give specific examples? 58.1. Did beneficiaries’ or members’ participation with this M2M Care Group and their activities change because of the cyclone? How and why? 59. In your opinion, what were the most important aspects or contributions of this M2M Care Group in the cyclone context? Why? 59.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why? Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 60. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out M2M Care Groups activities? 60.1. How effective would you describe these links with key partners at the moment? Why? 60.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 60.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 60.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Now I would like to ask about the participation and inclusion of both genders, youth, and people with disabilities (PWD) in this M2M Care Group. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: IMPEL | Implementer-Led Evaluation and Learning 100 Annex C: Data Collection Tool 61. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 62. What does men and women’s participation in this M2M Care Group’s activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 62.1. How does participation differ between men and women? Can you give any specific examples? 63. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both men and women? Can you give specific examples of actions taken to be inclusive? 63.1. How successful do you consider the efforts to include both men and women in M2M activities to be successful? Why/why not? 63.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 64. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 64.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 64.2. What considerations are important to ensuring equal access for PWDs? 64.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 65. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for youth? Please be as specific as possible. Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 101 65.1. What does youths’ participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for young men and young women? 65.2. What considerations are important to ensuring equal access for youth? 65.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119). Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 102 Annex C: Data Collection Tool FIOVANA Process Monitoring – School Based Activities KII Tool Background Information for Enumerators School Based Activities School-based activities reach primarily pre-adolescents, as well as some adolescents. FIOVANA staff work with communities and school authorities to create School Clubs. Under the facilitation of a Teacher Influencer Volunteer, groups of students are identified to promote key behaviors at the school level, such as improved feeding practices and dietary diversification and WASH (Water Sanitation and Hygiene). Students are also encouraged to share these improvements with their families. Club activities will include school gardens, cooking demonstrations and nutrition classes, as well as meal tasting and judging. Heads of households will participate in meal tasting to foster intergenerational communication about food and nutrition. Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Consultants Associates, a Madagascar-based data collection firm, in partnership with Causal Design, a research organization. We are conducting a process monitoring evaluation of ADRA FIOVANA program, which aims to enhance food security and resilience in six districts of Madagascar: Manakara, Mananjary, Vohipeno, Farafangana, Vangaindrano, Vondrozo. The FIOVANA Program includes a range of activities in a range of sectors, including School Based Activities. We specifically are examining how FIOVANA projects are being implemented, and anything that might be affecting implementation of these activities, either positively or negatively. We will be examining impact and effectiveness at a later time. Goal of this interview: We have invited you to this interview to share your views on the work of the School Based Activities and FIOVANA’ support for School Based Activities, with the aim of providing recommendations for the FIOVANA program. We would like to hear your perspective on how FIOVANA’s support has been delivered; your experiences implementing School Based Activities; and facilitators and obstacles you have encountered while implementing these activities. We would also like to hear about the inclusion of women, youth, and people with disabilities in School Based Activities and their activities; and effects of climatic events (i.e. cyclone/s) on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with FIOVANA, and it will be up to their discretion to share them with you and others. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 103 Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Respondent Type ● School Leaders Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune IMPEL | Implementer-Led Evaluation and Learning 104 Annex C: Data Collection Tool Respondent Type ● School Leaders Fokontany Observations Please provide details about the interview setting, interviewee’s responsiveness, and any other relevant information Interview Questions Background Firstly, we would like to hear some background about School Based Activities: 66. From your understanding, what is the main purpose of the School Based Activities? 67. What would you consider main tasks/activities for the School Based Activities to be? 67.1. If more than one activity specified: Which of these tasks/activities do you consider the most important for reaching your goals? Why? (i.e. Because they are most effective, most relevant/needed for the community?) Effectiveness of FIOVANA Support 68. What support for implementation have you received from FIOVANA so far? What did this consist of? (e.g. trainings, material or financial support) 68.1. What are your reflections, positive or negative, in the way this support was delivered? Could anything have been improved, and why? (e.g. format of trainings; any instances of delays) Effectiveness of Implementation Now I would like to hear about your experiences implementing activities through this School Based Activities: Note to Enumerators: Ask for each of the activities mentioned in Q2 above; if more than 3 main activities were mentioned, ask about the 3 most important. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 69. Can you briefly describe what implementation of your key tasks/activities for the School Based Activities looks like? (Probe for: who is this work/activity for, when/how often do you do the tasks/activity, do you use any equipment for the tasks/activities?) Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 105 70. What would a ‘successful’ or ‘smooth’ implementation of each these activities look like? 70.1. What factors (such as specific actions on your part; other actors; specific resources) contribute to it being a success? 70.2. How typically does this successful/smooth implementation happen when you are conducting activities? What might prevent implementation from being fully successful/smooth? 70.3. Note to Enumerators: Try to steer answers towards how activities are implemented, not just focusing on impact as a success. 71. What kind of challenges have you encountered while implementing your activities (prior to or other than cyclone-related challenges, we will touch on those specifically later)? (Probe specifically for COVID-19) 71.1. Can you give any specific examples of how these challenges affect your ability to implement? 71.2. How have you adapted to these obstacles/challenges? ■ If you did not/were not able to adapt, why not? What resources or other support were you missing? 71.3. How might these challenges be overcome in the future? Climate/Cyclones 72. How has the cyclone/s affected your ability to implement activities? 72.1. What changes or adaptations have you had to make to the way you implement because of the cyclone? 73. How did the cyclone affect the effectiveness, impact, and potential outcomes of School Based Activities? Can you give specific examples? 73.1. Did beneficiaries’ or members’ participation with School Based Activities change because of the cyclone? How and why? 74. In your opinion, what were the most important aspects or contributions of School Based Activities in the cyclone context? Why? 74.1. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Why? Note to Enumerators: Be aware that not all respondents may be able to reflect on the FIOVANA program as a whole or have enough information to answer this question with lots of detail. It may be helpful to mention some of the other activities being examined in this IMPEL | Implementer-Led Evaluation and Learning 106 Annex C: Data Collection Tool Process Monitoring, but do not feel the need to spend too much time on this question if respondents are unsure how to answer. Linkages 75. What other actors (local, formal, informal, governance, business, NGOs/civil society groups, or NGOs) do you work with or are involved in carrying out School Based Activities? 75.1. How effective would you describe these links with key partners at the moment? Why? 75.2. What are some effective or successful elements of these partnerships? (e.g. communication; availability; support; partners’ knowledge) 75.3. What specific outcomes have emerged out of these partnerships? Can you give any specific examples? ■ What factors do you think were responsible for these outcomes? Please be as specific as possible. 75.4. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Inclusivity Now I would like to ask about the participation and inclusion of both genders and people with disabilities (PWD) in School Based Activities. Gender Before asking about the activities specifically, I would like to ask a general question about women’s agency in this fokontany and areas targeted by the FIOVANA program: 76. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e. influence over household decisions). In what areas of household and community decision-making do women have more or less influence? Can you give specific examples? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for the IMPEL/Implementer-Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 107 77. What does boys and girl’s participation in School Based Activities currently look like in practice? In what aspects of activities do they/don’t they participate in? Please give examples. 77.1. How does participation differ between men and women? Can you give any specific examples? 78. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for both boys and girls? Can you give specific examples of actions taken to be inclusive? 78.1. How successful do you consider the efforts to include both boys and girls in School Based Activities to be successful? Why/why not? 78.2. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 79. How have you tailored/organized and conducted your activities to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 79.1. What does PWD participation in activities currently look like? In what aspects of the activity do they participate in? ■ How does participation differ for men and women with disabilities? 79.2. What considerations are important to ensuring equal access for PWDs? 79.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Is there anything else you would like to ask or say before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119) Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ IMPEL | Implementer-Led Evaluation and Learning 108 Annex C: Data Collection Tool FIOVANA Process Monitoring – Supervisor and Animator KII Tool Communication Protocol and Informed Consent Introduction and program information: Hello, my name is ________________. I work with Causal Design, a research organization conducting a process monitoring evaluation of ADRA’ FIOVANA program. Specifically, we are examining the implementation of FIOVANA projects and factors that might be affecting implementation of these activities, either positively or negatively. We are specifically examining the following FIOVANA activities: ● School Based Activities (SBA) ● Mother to Mother Care Groups (M2M) ● Cooking Demonstrations & Tsikonina ● Farmer Field Schools (FFS) ● Farmer Market Schools (FMS) ● Impact Investment Fund (IIF) ● Locally Managed Marine Areas (LMMA) ● Disaster Risk Management Committees (DRMC) Goal of this interview: We have invited you to this interview as one of FIOVANA’s key implementers. We would like to hear your perspective on how the FIOVANA support has been delivered, and your experiences implementing activities under the FIOVANA program. We would like to hear about both facilitators and obstacles, with the aim of ascertaining how implementation could be improved in the future. We would also like to hear about the inclusion of women, youth, and people with disabilities; and effects of the climate events/cyclones and COVID-19 on program/activity implementation and activities. We would like to emphasize that we are interested in hearing about how these activities are being rolled out in practice on the ground, which may differ from how the activity was originally planned or you were trained. We are interested in hearing your opinions and judgment about implementation successes and challenges. We aim for the interview to take up around 1 hour of your time. Managing Expectations: We cannot guarantee that every suggestion raised during this interview will be acted upon, but all suggestions will be analyzed and those that occur frequently may be taken forward as recommendations to program leadership. The analysis results will be shared with ADRA, and it will be up to their discretion to share them with you and others. Informed Consent: With your permission, I will be recording this interview. What you say during this interview will be shared with a team of analysts, but nobody else. We will remove your name and other personally identifying information from the recording, and your name will not appear in the analysis or any report. So please, speak openly to us – your voice is very important to help us learn about the successes and challenges of this program. For interviews with more than one respondent: We would like to hear from all of you, so I encourage you all to speak openly, even if you may disagree with what another participant is saying. I ask that you Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 109 please speak one by one, so we can record all of your answers accurately. We will try to make sure everyone has a chance to speak. Do you agree with being interviewed by us, and to our conversation being recorded? [CONFIRM CONSENT] Thank you very much for agreeing to take part in this interview. Do you have any questions before we begin? Interview Information Enumerator/s Interview Date Original Interview Language French Malagasy English Approx Interview Length Delete as appropriate <30 mins 30mins 45 mins 1hr 1hr+ Respondent Information Respondent Type ● District Supervisor ● Commune Animator ● IIF Specialist Number of Respondents Interviewee Name/s Gender Age Delete as appropriate 18 - 30 years old 30 - 60 years old 60+ years District Commune Purpose Delete as appropriate P1 / P2 / P3 Observations Please provide details about the interview setting, interviewee’s IMPEL | Implementer-Led Evaluation and Learning 110 Annex C: Data Collection Tool Respondent Type ● District Supervisor ● Commune Animator ● IIF Specialist responsiveness, and any other relevant information Interview Questions Background I’d like to start by hearing some background information about your involvement in the FIOVANA program. 26. Which components of FIOVANA are you particularly involved in? Note to Enumerators: Probe for our select 8 activities above and concentrate on these. 27. What is your particular role within FIOVANA and how long have you been involved? 28. From your understanding, what is the main purpose of your involvement and activities/[key activity] as part of the FIOVANA program? 28.1. What are your key roles and tasks? 28.2. What are your objectives as part of the FIOVANA program? 28.3. Do you think these tasks and objectives are reasonable given the context you/FIOVANA works in, and are you optimistic/pessimistic about realizing them? ■ Why?/Why not? Effectiveness of Implementation Now I would like to hear about your experiences implementing activities as part of the FIOVANA program. Note to Enumerators: If respondents talk only about the effectiveness or impact of the activities, steer them back to the topic of how activities are implemented, or their role in and experience of carrying out these activities. 29. How did you make the transition from planning FIOVANA activities (thinking about them ‘in theory’) to their actual implementation? What are the essential, concrete ‘next steps’ involved in putting activities into practice? 29.1. Which steps or actions during this stage do you think are fundamental to making implementation work best? How have these steps/actions played out under FIOVANA? Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 111 29.2. What aspects of the transition from planning to actual implementation worked well, and which were more complicated? Why? How could they be improved? 30. What are some of the key considerations, in particular with regards to the communities where activities take place, that you included in your preparations to ensure implementation runs smoothly? (E.g. gender dynamics, poverty, migration, terrain, focus of economy (i.e. agriculture, fishing)) 30.1. How do you accommodate them/factor them into your planning, or implementation now? Can you be specific? 31. Thus far, what have been some of your priorities, or requirements, for start-up/laying the groundwork for FIOVANA implementation? (E.g. partnerships, communication, etc.) 31.1. What aspects of start-up/laying the groundwork has worked well, and which have been more complicated? Why? How could they be improved? 32. Other than these, what are your other critical priorities or requirements for implementing or rolling out your key activities? (e.g. communication, logistics, coordinating with communities) 32.1. Which aspects of establishing these have worked well, and which were more complicated? Why? How could these be improved? Adaptations 33. How does your implementation in practice compare to how you expected or planned it would roll out? 33.1. Can you give any specific examples of changes or adaptations made? 33.2. In particular, how did COVID-19 affect your implementation 34. What were the key outcomes of the refine-and-implementation period? 34.1. What specific adaptations did you make based on your learnings/observations during this phase? 34.2. If you could not adapt, why not? What resources or other support were you missing? 35. Are there any other expected or unexpected local conditions or obstacles that have affected implementation thus far? Please describe them. How did you adjust? 35.1. If you did not/could not adapt, why not? What resources or other support were you missing? 36. Have you been unable to conduct any activities you wanted or that were planned? Why? IMPEL | Implementer-Led Evaluation and Learning 112 Annex C: Data Collection Tool 37. Other than the refine and implementation (R&I) period, what systems are currently in place to continue to adapt or ensure implementation? What do these look like? 37.1. Are there any examples of this in practice? Climate/Cyclone 38. What are the effects of the cyclone on your FIOVANA activities/implementation? Probe for both implementation and outcomes. 39. What changes or adaptations have had to make to the way you implement because of the cyclones? Why? 39.1. Have these adaptations been effective? 40. Were there any aspects of the implementation or activities that were completely derailed by the cyclones? Which ones, and how? 41. In your opinion, what were the most important aspects or contributions of your activities in the cyclone context? 42. What were the most important aspects of the FIOVANA program as a whole in the cyclone context? Linkages 43. Are there any other organizations or actors (local, formal or informal, like chiefs or women’s groups, government bodies, businesses or even other aid agencies) that are critical to you being able to effectively carry out your work? 43.1. Who/what are they, and how are they essential to effective implementation? 44. How effective would you describe these links with key partners at the moment? 44.1. What are some effective elements of these partnerships? What do you consider important for ensuring you work together to implement activities effectively? 44.2. What specific outcomes have emerged out of these partnerships? What factors were responsible for these outcomes? Please be as specific as possible. 45. What limitations do you see in your partnerships at the moment? How could they be improved, or what would you change moving forward? (e.g. communications, type of support, expanding to other/different partners, geography or sectoral knowledge). Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 113 Inclusivity Gender 46. The baseline findings (refer to this in more detail if that is not clear) suggest that women in this region possess a high degree of ‘agency’ (i.e, influence over household decisions). In what areas of household decision-making do women have more or less influence or a say? ■ If more information needed: Causal Design was commissioned to conduct a baseline study for FIOVANA under the IMPEL/Implementer￾Led Design, Evidence, Analysis and Learning (IDEAL) activity. Our baseline findings on gender suggested that women had more agency than CRS/ADRA’s previous gender analyses suggested. 47. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for both men and women? 47.1. What does men and women’s participation in activities currently look like? In what aspects do they participate, and in how do they participate? ■ How does participation differ between men and women? 47.2. What considerations are important to ensuring equal access for both men and women when implementing programs (not just designing them!)? 47.3. What limitations or gaps still exist in implementation efforts to ensure equal access? How might they be overcome? PWD 48. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for people with disabilities (PWDs)? 48.1. What does PWD participation in activities currently look like? In what aspects do they participate, and in how do they participate? 48.2. What considerations are important to ensuring equal access for PWDs? 48.3. What limitations or gaps still exist in implementation efforts to ensure equal access for PWDs? How might they be overcome? Youth 49. How have you tailored/organized and conducted your activities and implementation to ensure equal access, benefits, and participation for youth? Note to Enumerators: ‘Youth’ refers to adolescents and young people aged 10 to 29. 49.1. What does youths’ participation in activities currently look like? In what aspects do they participate, and in how do they participate? ■ How does participation differ for young men and young women? IMPEL | Implementer-Led Evaluation and Learning 114 Annex C: Data Collection Tool 49.2. What considerations are important to ensuring equal access for youth? 49.3. What limitations or gaps still exist in implementation efforts to ensure equal access for youth? How might they be overcome? Is there anything else you would like to say or ask before we conclude the interview? Thank you so much for your time! We greatly appreciate your perspective on all of this! If you have any further questions or concerns, please do not hesitate to contact Eliane Ralison (+261 331165119) Approx Interview Length Delete as appropriate <30 mins / 30mins / 45 mins / 1hr / 1hr+ Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 115 District-Level Checklist Cover Page Enumerator Name District ACTIVITY Possible Validation Exercises Location (Note Commune and Fokontany, and note how close it is to where the activity takes place) Quick notes or observations (This could be around weather, trouble finding people, obstacles or positives) SBA  Speak to: Teacher or Parent Representatives  Observe: Teacher or Parent Representative Training Cooking Demos & Tskikonina  Speak to: Community members and/or caregivers of children 6 to 59 months old  Observe: Cooking session M2M  Speak to: M2M members  Observe: M2M session, pictorial books . FFS  Speak to: Demonstration Plot members IMPEL | Implementer-Led Evaluation and Learning 116 Annex C: Data Collection Tool ACTIVITY Possible Validation Exercises Location (Note Commune and Fokontany, and note how close it is to where the activity takes place) Quick notes or observations (This could be around weather, trouble finding people, obstacles or positives)  Observe: Demonstration Plot tour (Priority)  Observe: Demonstration Plot training session FMS  Speak to: FMS members .  Observe: Technical session, sales tracker DRMC  Speak to: DRMC Members  Observe: DRMC meeting LMMA  Speak to: Fokontany Chief Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 117 Observation Checklist School Based Activities Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews SBA Teacher or Parent Representatives Ask about: - Participation in SBA training - Information gleaned/effectiveness of trainings Observations SBA Teacher or Parent Representative Training Observe: - Assess Volunteer Teachers interaction, engagement and demeanor - Location of training - Participants present and their characteristics - Assess content, its relevance and delivery IMPEL | Implementer-Led Evaluation and Learning 118 Annex C: Data Collection Tool Cooking Demonstrations/Tskikonina Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews Community members and/or caregivers of children 6 to 59 months old Ask about: - Participation in Cooking Demonstrations/Tskikonina activities and frequency of these - Conduct of CHV/CNV - Information gleaned/effectiveness of meetings/trainings Observations Cooking session Observe: - Assess CHV/CNV and participants’ interaction, engagement and demeanor - Location of session - Participants present and their characteristics - Assess content, its relevance and delivery Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 119 M2M Care Groups Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews M2M Members Ask about: - Participation in Model Mother/Care Group activities and frequency of these - Conduct of Model Mothers - Information gleaned/effectiveness of meetings/trainings Observations M2M Training Observe: - Assess Model Mother and participants’ interaction, engagement and demeanor - Location of training - Participants present and their characteristics - Assess content, its relevance and delivery IMPEL | Implementer-Led Evaluation and Learning 120 Annex C: Data Collection Tool Farmer Field School Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews Demonstration Plot members Ask about: - Participation to date and frequency of meetings/trainings with Lead Farmer - Specific learnings/effectiveness of trainings - Relevance and ability to implement learnings Observations Demonstration Plot Observe: - Relative location of plot compared to community - State of plot (tilled, under cultivation, abandoned, fenced, etc.) - Inquire into nature and stage of demonstration(s) - Probe for: crop type, irrigation/seed/fertilizer focus, etc.? Demonstration Training Observe: - Assess Leader Farmer and participants’ interaction, engagement and demeanor - Location of training - Participants present and their characteristics - Assess content, its relevance and delivery Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 121 Farmer Market School Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews FMS Members Ask about: - Participation to date and frequency of meetings/trainings with FMS agent - Specific learnings/effectiveness of trainings - Relevance and ability to implement learnings Observations Technical Session Observe: - Assess FMS agent and participants’ interaction, engagement and demeanor - Location of training - Participants present and their characteristics - Assess content, its relevance and delivery IMPEL | Implementer-Led Evaluation and Learning 122 Annex C: Data Collection Tool Disaster Risk Management Committees Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews DRMC members Ask about: - The nature of their participation - Key lessons from the consultations - Any observed changes in implementation that have been realized - Challenges or criticisms of process Observations DRMC meeting Observe: - Participants present and their characteristics - Participants’ interaction, engagement and demeanor - Location of meeting - Assess content, its relevance and delivery - NOTE: If any physical assets are being/were built, try to see these in person and assess their existence, state (complete, under construction, etc), and condition Process Evaluation of the FIOVANA RFSA in Madagascar Annex C: Data Collection Tools 123 Locally Managed Marine Areas Enumerator Name Date District Commune Fokontany Observations Please provide details about the interview setting, participants’ responsiveness, and any other relevant information Informal Interviews / Observations Notes (If unable to complete, note this and reasons why e.g. lack of time, no respondents available, etc.) Informal Interviews Fokontany Chief Ask about: - Community consultations, communication about LMMAs, needs assessments IMPEL | Implementer-Led Evaluation and Learning 124 Annex C: Data Collection Tool