Joint Mid-Term Review of the UBALE and Njira Projects USAID-funded Title II Development Food Assistance Projects in Malawi led and Catholic Relief Services (UBALE) and Project Concern International (PCI) Award Numbers: AID-FFP-A-14-00006 & AID-OAA-A-15-00019 (UBALE) AID-FFP-A-15-00003 & AID-OAA-A-15-00018 (Njira) JMTR Methodology (JMTR Report, Volume II) 5 June 2017 (Final) JMTR Methodology 5 June 2017 i Acknowledgement The members of the JMTR team would like to gratefully acknowledge the support provided by the staff of the UBALE and Njira Projects. Special thanks go to the staff from all of the consortium partners who went out of their way to provide logistical support to enable the team to implement an intensive schedule with a large number of reviewers and observers. We would also like to thank the technical and implementing staff from all of the partners for supporting the field work, participating in various events during the review, and providing insights while answering our multitudinous questions. Both projects were able to recruit translators for the exercise who performed admirably in helping team members understand the context and responses from participants. Innumerable project participants and beneficiaries have also provided useful responses to our questions and participated in stimulating discussions. Our gratitude goes out to them as well. Hopefully we've listened well, our observations are grounded in reality, our assessment is accurate, and our recommendations are useful. Sincerely, Rupert Best, Independent Consultant Aaron Chassy, Catholic Relief Services HQ Amos Chigwenembe, Catholic Relief Services Malawi Mike DeVries, Independent Consultant Charlotte Germain-Aubrey, USAID Bureau for Democracy, Conflict and Humanitarian Assistance HQ Mary Pat Kieffer, Project Concern International HQ Mark Kumbukani Black, CARE Malawi Mike Manske, USAID Food for Peace HQ Lucy Mungoni, USAID Malawi Kllian Mutiro, USAID Food For Peace Southern Africa Amy Sunseri, Project Concern International HQ Marit Wilkerson, USAID Bureau of Food Security HQ Cover pictures taken by Douglas La Rose This report was made public by support from the U.S. Agency for International Development (USAID). The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the Government of the United States of America JMTR Methodology 5 June 2017 ii TABLE OF CONTENTS Page I. EXECUTIVE SUMMARY FOR VOLUME II 1 II. INITIAL SCOPE OF WORK FOR THE MALAWI JMTR 2 III. UPDATED JMTR PROTOCOL WITH INFORMATION ON ACTUAL PROCESS USED 27 IV. DATA COLLECTION TOOLS 64 V. DATA COLLECTION SCHEDULES 116 VI. PERSONS INTERVIEWED AND SITES VISITED 146 VII. FINAL WORKSHOP PLANS FOR THE VERIFICATION WORKSHOPS 167 VIII. JMTR PROCESS REVIEW 173 JMTR Methodology 5 June 2017 iii ACRONYMS AND ABBREVIATIONS ACPC Area Civil protection Committee (found at the TA level) ADC Area Development Committee (found at the TA Level) ADDRMO Assistant District Disaster Risk Management Officer ANC Ante-Natal Care ANCC Area Nutrition Coordinating Committee (found at the TA level) CAHW Community Animal Health Worker CBM Community-Based Management (for WASH) CBDA Community-Based Distribution Agent CBO Community-Based Organization CCA Climate Change Adaptation CCFLS Community-led Complementary Feeding and Learning Sessions CDF Community Development Funds (USAID funds disbursed through the OAA) CLAN Community Leaders for Action on Nutrition (found at the GVH level) CNFA Cultivating New Frontiers in Agriculture CoP Chief of Party CPC Civil Protection Committee (found at the GVH, TA and District-levels) CRS Catholic Relief Services CSB Corn Soy Blend DFAP Development Food Assistance Program DiNER Diversity and Nutrition for Enhanced Resilience DIP Detailed Implementation Plan DNCC District Nutrition Coordinating Committee DoDMA Department of Disaster Management Affairs EI Emmanuel International FFA Food for Assets FGD Focus Group Discussion GoM Government of Malawi GVH Group Village Headman (the lowest GoM structure encompassing a number of villages) HSA Health Surveillance Assistant IGA Income-Generating Activity IMCI Integrated Management of Childhood Illness IPTT Indicator Performance Tracking Table ITSH Inland Transport Storage and Handling IYCF Infant and Young Child Feeding JMTR Joint Mid-Term Review LoA Life of Activity M&E Monitoring and Evaluation MCHN Maternal and Child Health and Nutrition MEAL Monitoring, Evaluation, Accountability & Learning MEL Monitoring, Evaluation and Learning MoAIWD Ministry of Agriculture, Irrigation and Water Development MoGCDSW Ministry of Gender, Children, Disability and Social Welfare MoH Ministry of Health MUSCCO Malawi Union of Savings and Cooperatives JMTR Methodology 5 June 2017 iv MVAC Malawi Vulnerability Assessment Committee MWK Malawi Kwacha NASFAM National Smallholder Farmers Association of Malawi NGO Non-Governmental Organization OAA Office of Acquisition and Assistance ONSE Organized Network of Services for Everyone's Health PCI Project Concern International PLW Pregnant and Lactating Women PM Project Manager PMP Performance Monitoring Plan PM2A Preventing Malnutrition in Children Under 2 Approach PMU Project Management Unit PREP Pipeline and Resource Estimate Proposal PSP Private Service Provider PVCA Participatory Vulnerability Capacity Assessment PVO Private Voluntary Organization RMNCH Reproductive Maternal, Newborn and Child Health SANE Strengthening Agriculture and Nutrition Extension Services SBCC Social and Behavioral Change Communications SILC Savings and Internal Lending Community SMART Skills for Marketing and Rural Transformation TA Traditional Authority (a GoM structure encompassing a number of GVHs) TEEC Tools for Enterprise and Education Consultants ToC Theory of Change ToT Training of Trainers TWG Technical Working Group UBALE United in Building and Advancing Life Expectations VCPC Village Civil Protection Committee (found at the GVH level) VDC Village Development Committee (found at the GVH level) VNRMC Village Natural Resource Management Committee (found at the GVH level) VSLA Village Savings and Lending Association WALA Wellness and Agriculture for Life Advancement WASH Water, Sanitation and Hygiene WE/VSL Women's Empowerment/Village Savings and Lending JMTR Methodology 5 June 2017 Joint Mid-Term Review (JMTR) of the UBALE and Njira Projects JMTR METHODOLOGY JMTR Report, Volume II EXECUTIVE SUMMARY FOR VOLUME II This Volume II of the report for the Joint Mid-Term Review (JMTR) of the UBALE and Njira Projects in Malawi in 2017 summarizes the methodology that was used to conduct the JMTR. This resource document for future reviews is being provided as a separate document from the JMTR Summary Report (Volume I) which summarizes the priority recommendations that emerged from the JMTR Process. The section which immediately follows provides the Scope of Work for the JMTR which initiated the planning for the exercise. This product was developed jointly by FFP with representatives from PCI and CRS headquarters. The second section provides the JMTR Protocol that was used to operationalize the exercise. The protocol included here has been updated with the actual processes that were used, since adjustments were made throughout the process. It represents a good starting point for planning the next JMTR or similar exercise. The protocol was initially developed by the team leader for the JMTR. The third section in this volume provides the data collection tools, mainly topical outlines and interview checklists, which were used by various team members over the course of the exercise. These are included as an annex to the JMTR Protocol, but they have been provided here as a separate section in Volume II for ease of access to planners of future reviews. The next section provides the various schedules that were used to implement the JMTR including the overall schedule (which is also an annex in the JMTR protocol) as well as field work schedules for the interviews and focus group discussions (FGDs) held in each district. Over the course of the JMTR, the field work scheduling process was continually improved. The field work schedule for Balaka, the last district which the JMTR team visited, represents the best version of a field work schedule. The fifth section lists the actual interviews and FGDs held by the ten JMTR team members for the UBALE review and by the nine JMTR team members for the Njira review. The sixth section of Volume II provides the workshop plans for the two Verification Workshops that were conducted. Improvements were made in the plan for the Njira Workshop which was conducted after the UBALE Workshop, so the workshop plan for Njra represents the better version of a plan for a Verification Workshop. Finally, the last section of Volume II summaries the results of a JMTR process review that was conducted with JMTR team members and representatives from the two projects to glean good practices and lessons learned from the Malawi experience to inform the planning for future reviews. JMTR Methodology 5 June 2017 2 I. INITIAL SCOPE OF WORK FOR THE MALAWI JMTR JMTR Methodology 5 June 2017 3 SCOPE OF WORK FOR JOINT MIDTERM REVIEW OF FFP DEVELOPMENT FOOD ASSISTANCE PROGRAMS IN MALAWI UBALE AND NJIRA PROGRAMS JMTR Methodology 5 June 2017 4 A. Background In Fiscal Year 2014, the U.S. Agency for International Development (USAID) Office of Food for Peace (FFP) awarded funding to private voluntary organizations (PVOs) to implement multi-year development food assistance projects in the most food-insecure regions of Malawi. FFP issued two awards: one to Catholic Relief Services’ (CRS) UBALE project in Nsanje, Chikwawa and Blantyre Districts; and the second to Project Concern International’s (PCI) Njira project in Balaka and Machinga Districts. The main purpose of these two projects is to improve long-term food security and nutrition in southern Malawi. Through close collaboration with Government of Malawi (GoM) ministries and other stakeholders, the purposes of UBALE are designed to reduce chronic malnutrition and food insecurity and build resilience among vulnerable populations in three of the most food-insecure, chronically malnourished, and disaster-prone districts of Southern Malawi. UBALE works towards: 1) smallholder farming households sustainably increasing productivity of nutritious and profitable farm products; 2) vulnerable rural households successfully engaging with markets; 3) reducing stunting among children under five; 4) households and communities being more resilient to shocks in a cross-cutting manner; and 5) underlying systems and structures are sustainably contributing to reducing chronic malnutrition and food insecurity while building the resilience of 250,000 households in Nsanje, Chikwawa and Blantyre Rural districts. The Purposes of the Njira project are to: 1) Increased Income from Agricultural and Non-agricultural Activities; 2) Improved health and nutrition of pregnant and lactating women and children under five; and 3) Improve capacity to prepare for, manage and respond to shocks. Program activities include, for example, development of community-level plans for supporting climate smart agriculture, support for adoption of quality agricultural inputs and improved farming practices, establishment of informal village savings and lending groups, provision conditional food rations, training on improved child feeding and household hygiene practices, support for improved access to reproductive health services, and training of government and community counterparts on effective disaster risk mitigation. In total, Njira is aimed at directly benefitting 63,472 households and 244,248 individuals. i. Overview of UBALE Program The overall goal of UBALE is to reduce chronic malnutrition and food insecurity and build resilience among vulnerable populations in rural Blantyre, Chikwawa, and Nsanje districts of Malawi. Purpose 1: Vulnerable households’ income increased Purpose 2: Improved nutritional status among pregnant and lactating women, and children under the age of two years. Purpose 3: Communities are empowered to contribute to their own sustainable development All the three purposes are pursued with the overarching goals of strengthening local government capacity, empowering youth and promoting gender equality. JMTR Methodology 5 June 2017 5 UBALE Target Population by Program Element UBALE will target all 284 Group Village Headman (GVH, a cluster of villages) communities in all 26 rural Traditional Authorities (TAs) in these three districts, of which 147 GVHs in 17 TAs were part of WALA, wrapping around existing food-security and nutrition programs with complementary program elements to ensure all communities have access to the full package of UBALE interventions. Unlike WALA, UBALE will target all villages within each GVH to provide greater saturation and reach HH missed by WALA that are still highly vulnerable. UBALE will reach 248,200 HH, touching the 90% of the rural population in the target districts. By delivering activities through government and private-service providers, UBALE can achieve saturation coverage while instilling government and community ownership from the beginning. UBALE Approaches The factors contributing to chronic food insecurity and malnutrition in Southern Malawi are complex and intertwined. Poor soil fertility, insufficient agricultural production and weak access to markets limit both the availability of food and the possibility to earn income, which in turn affects households’ ability to use health services. Poor health, caused by under-nutrition, diarrheal disease, malaria, chronic illness or infection limits households’ ability to engage in productive activity. These mutually reinforcing elements are subject to chronic and cyclical shocks, some strongly linked to climate change, which destroy or diminish assets; damage health status; and divert time, energy and resources from productive activity. Underlying all of these are community systems and structures that often disadvantage women and girls, and fail to provide all community members with a voice, ensure that the most vulnerable are cared for, or effectively govern community￾owned assets. Catholic Relief Services (CRS) and UBALE partners have developed a theory of change to describe the intertwined development pathways through which the program will assist its beneficiaries to address these factors, ultimately becoming more healthy and food-secure. Thus the program goals is: Reduced chronic malnutrition and food insecurity, and increased resilience among vulnerable populations in the districts of rural Blantyre, Chikwawa, and Nsanje in Malawi, supported by three purposes. The theory of change posits that improvements to agricultural production will be motivated by the prospect of increased income that comes with access to markets (Purpose 1). This purpose describes how UBALE will systematically move households from a situation of greater vulnerability and less risk tolerance (Recover), to being able to consolidate and stabilize their assets (Build), on to a point where they can risk higher-level, income-oriented investments (Grow). This evidence-based Pathway to District # of GVH Rural HH (total pop) Health & Nutr. Ag & Markets DRM Total HH r. Blantyre 77 99,900 64,600 59,900 80,000 89,900 Chikwawa 75 120,700 82,400 59,700 81,000 108,600 Nsanje 132 55,200 51,400 26,400 25,100 49,700 Total 284 275,800 198,400 146,000 186,100 248,200 Figure 1: Ubale Beneficiary Targeting JMTR Methodology 5 June 2017 6 Prosperity model defines how interventions are phased, targeted and layered depending on households’ livelihoods status. The improvements in food security thus achieved will both facilitate and benefit from improved health and nutrition status for women and children (Purpose 2). To mitigate against the risk that households nevertheless fail to thrive because they are insufficiently prepared for shocks, UBALE also assists households and communities to better prepare for risks and better manage and rebound when shocks do arrive (Purpose 3). Underlying all program activities is the intentional strengthening of foundational systems and structures that govern how communities are able to address their own development issues. The objectives of all interventions rely on this sub-purpose to be effective and sustainable. Throughout each of these purposes, improved gender equity is also considered to be a necessary precondition for arriving at the desired outcomes. Strengthened participation by, and engagement with youth, who form a significant portion of the population but are often undervalued, is also expected to increase the long-term impact of the program. Purpose 1: Vulnerable households’ income increased. UBALE’s theory of change posits that prosperous farming households have the motivation, information, and resources to manage their farms and households in a way that values the inputs of men and women and to produce products for both table and markets accessing credit and insurance to more fully engage in value chain activities and participating in both on and off-farm employment opportunities to smooth risk. To improve farm-management practices both women and men need, based on extensive research by CRS, skills in group organization and management, savings and financial management and household budgeting, marketing basics, how to harness innovation, and protecting the natural-resource base for agricultural production. These SMART Skills are coupled with messages and activities intended to increase joint decision-making within the household, for better decision-making and more equitable allocation of household resources. In addition to general household needs, farming households also need to improve the management of their farming systems. They will acquire information on improved production techniques and climate￾smart agriculture through strengthened government agriculture extension. This will be assisted by the USAID-funded for MODES: Modernization of on Demand Extension Services project, providing on demand extension services to a projected 800,000 smallholder households and 2.3 million radio listeners in Malawi. UBALE assumes that with the new Learning Center model, provision of work tools and linkages to research and the private sector, these services will adequately support Lead Farmers to conduct outreach in their communities. They will acquire resources in the form of planting material through DiNER fairs or as members of Care Groups, and with SILC savings. The motivation comes from the prospect of increased income through sales, while Lead Farmer activities will also demonstrate the effectiveness of promoted technologies. While increasing the volume of production is important, farmers also need to diversify what they produce, as a measure to mitigate vulnerability to weather-related shocks while ensuring access to nutrient-dense foods and generating commodities for markets. In addition to the information and resources made available through Purpose 1, information and motivation will be generated through participation in Care Groups and participation in Marketing Clubs. In order to maximize the benefits of JMTR Methodology 5 June 2017 7 improved production, farmers will also acquire the information, motivation and resources to reduce contamination and post-harvest losses, with additional financial resources being generated through links to SILC. In order to increase income, farming households need access to markets, financial services, and diversified, independent livelihoods streams. UBALE’s theory of change posits that its efforts to help farmers get organized and strengthen linkages throughout selected value chains will add enough stimuli that market forces will effectively take over and make the systems sustainable. It also assumes that strengthened value chains will become more capable of absorbing production from vulnerable, smallholder farming households. In that sense it assumes that numerous stakeholders in the targeted value chains will willingly participate and engage with the program. Evidence suggests that when interventions are aligned with the natural interests of those stakeholders, which UBALE has taken care to do, then that participation is likely. In order for farmers to increase their sales, they need to be organized. Thus UBALE will form and strengthen marketing clubs, clusters and associations. They also require market-price information, for which UBALE will leverage and improve access to existing government and private systems, including through the USAID-funded MODES program. Finally, the program must also help to ensure that there is sufficient demand for farmers’ production, which is the justification for efforts to smooth targeted value chains at all levels and strengthen their links to participating farmers, especially vulnerable ones. Once farmers have reached a point where they can take on more risk, they need access to finance to invest in their production. At the most basic level, they can access SILC savings or loans. Once they are more organized, they will benefit from linkages and support to access loans from MFIs or banks and insurance products. While CRS and partners are confident that all farming households can benefit from improved market access, it is also evident that not all households or household members have access to enough resources (land, labor) to become food-secure through farming alone. And all farming households can better withstand shocks if they have multiple, independent livelihoods streams. For this reason, UBALE will also seek employment and income-generating opportunities, which may be part of targeted value chains or new value chains, that can benefit in particular participants with limited productive resources. Additional livelihoods opportunities may be generated through the expansion and strengthening of business-development services. Purpose 2: Improved nutritional status among pregnant and lactating women, and children under the age of two. In order to address issues of good health and nutrition, UBALE follows the evidence-based approach of targeting the first 1,000 days of life from conception through a child’s second birthday. The theory of change states that their health will be improved if capacities of government and community structures are strengthened to provide services and support to households, and of the households themselves to apply optimal behaviors for feeding and care. Woven through all of this is the need to improve women’s agency so that they are empowered to take decisions for their own health and that of their children. UBALE takes advantage of the GoM’s commitment to Scaling Up Nutrition (SUN), which provides guidance and models for MCHN and WASH initiatives. It assumes that government and community actors, including donors and other development programs in the program area, will fulfil their JMTR Methodology 5 June 2017 8 commitments to strengthen government and community capacity in the areas they cover. It also assumes that MoH systems for recuperative services for cases of acute malnutrition are functioning, as are MCHN services such as ante-natal care, immunization, deworming and the provision of iron and Vitamin A and that the MoH continues to support good management and control of chronic illnesses such as HIV by providing access to proper treatment and care. It further assumes that other initiatives for WASH are supporting CLTS, in particular that they fulfil their commitments to trigger the targeted communities. UBALE considers that the government-led services require training and accompaniment to better direct and oversee MCHN activities, ensure a continuum of MCHN services at all levels, and deliver quality services. At the community level, UBALE will ensure that the CLAN, as the structure that coordinates multi￾sectoral efforts to improve nutrition for children, effectively mobilizes support for a functioning network of Care Groups that provides information and support to households with pregnant or lactating women or children under two. Where Purpose 1 largely addresses access and availability, Care Groups address utilization by providing information and motivation to support optimal feeding practices, and link to Purpose 1 in explaining the importance of dietary diversity. CCFLS supports the behavior change needed to sustain these practices by building caregivers’ confidence to design meals and prepare a wide range of nutrient-dense foods. Community support for good feeding and care practices must also be complemented by support for proper hygiene and sanitation promoted through Care Groups and strengthened government systems. UBALE supports the GoM strategy using CLTS to boost demand for sanitation through capacity building of key players at sub-district level. Evidence is accumulating that environmental enteropathy also plays a significant role in chronic malnutrition of small children. UBALE assumes that this will eventually be substantiated and that guidance will be developed for a few key behaviors that can be adopted by households to prevent it. In the meantime, the program will take advantage of existing investments into Community Based Childcare Centers to model key targeted behaviors and begin to build an experience base in targeted communities that will facilitate their eventual adoption by households. The three principal household factors negatively affecting the nutritional status of children under two that UBALE partners identified are access to nutritious food, an understanding of optimal feeding and care behaviors and how to apply them, and young women’s ability to enter into pregnancy in a state of optimal health. To address access, complementing the improved production of food in Purpose 1, the program will make a conditional ration available to all mother-child pairs with children under two in the coverage area during the lean season. The blanket coverage is consistent with the principles of PM2A, while the timing (distribution over six months of the year) supports GoM policy regarding supplementary feeding. Additional food support to vulnerable households is provided through FFA activities carried out under Purpose 3. Through individual visits by Care Group Lead Mothers, primary and secondary caregivers will acquire the tailored skills and support (information and means) to apply optimal health and nutrition practices, with focus on ENAs. JMTR Methodology 5 June 2017 9 The health of women before they become pregnant being a known factor influencing the health of their children, UBALE’s theory of change assumes that by increasing young women’s understanding of their own bodies and pre-conception health, they will be empowered to make better decisions and negotiate on their own behalf. Offering the training to young men as well is intended to increase their understanding and appreciation of their potential partners. UBALE does not consider this training to be sufficient to effect the desired level of change, but it assumes that coupled with other gender-related messaging through e.g. SMART Skills, youth clubs and gender champions, as well as the efforts of the Malawian government and traditional leaders, it will make an important contribution to transforming the position of young women. UBALE partners consider that women need a strong sense of agency to have the skills and self￾confidence to influence how health-related decisions are made, for herself and for her children. While a five-year program is necessarily limited in how much change can be engendered, UBALE has given priority to two elements: first, to ensure that women of reproductive age, whether or not they already have children, have the necessary knowledge and confidence to make sound decisions about their bodies and their health, and secondly to create a supportive environment where spouses, grandmothers and other influential individuals take women’s health seriously and facilitate the application of targeted behaviors. Purpose 3: Communities are empowered to contribute to their own sustainable development UBALE believes that communities’ capacity to govern themselves and take ownership of their own development, along with the presence of social cohesion and inclusion, are significant determinants of the sustainability of program outcomes. UBALE’s approach is to work through and with existing structures wherever possible, strengthening them to more effectively provide support and services to individuals and families after the program finishes. While elements such as good governance and social inclusion can be difficult to measure, factors considered important at the national level (good governance practices, strong civil society) are also relevant to local communities. Without these, any gains made at the individual level are not likely to be sustainable. For this reason, Purpose 3 is foundational: it seeks to strengthen the underlying systems and structures that underpin all of UBALE’s positive outcomes. At the community level, UBALE addresses the structures that are necessary for the program to function and meet its objectives. Structures are essentially all groups or bodies with which UBALE interacts that have a mandate for the common good: these include Village Development Committees, Care Groups, Water User Committees, PSP Networks, marketing clubs, youth clubs and more. Through training and accompaniment UBALE will provide them with the skills and knowledge to effectively carry out their mandate, while demonstrating the benefits that accrue to everyone when the voices of women, youth, disabled, elderly, and others are heard and their concerns addressed. To improve communities’ ability to prepare for and mitigate the effects of shocks, they need to put structures and systems in place to prepare for and respond to disasters, protect the natural-resource base from climate change- and other weather-related shocks, and ensure that systems for decision￾making take women’s voices into account. This purpose assumes that the GoM policy encouraging community-level structures for civic protection and natural-resources management are effective and will continue to be encouraged throughout the program’s duration; it further assumes that other programs (in particular ECRP/DISCOVER) will similarly develop the VCPC structures in their coverage JMTR Methodology 5 June 2017 10 area. Finally, this purpose assumes that communities will be able to mitigate and respond to disasters of localized importance, and that government and donors will mount a response to any disasters that overwhelm local capacities. The nature of rural communities and, in Malawi, their particular exposure to climate-related shocks (exacerbated by environmental degradation and climate change) means that they require systems to care for community-level natural resources that include development, protection and maintenance of community-level productive assets. UBALE’s theory of change posits a seamless complementarity between the outcomes in Purpose 1 for improved natural-resource management, risk management and diversification at household level and those in Purpose 3, which address the same issues at community level. Community NRM plans will identify the key communal assets that need rehabilitation or protection, and facilitate communities to undertake the necessary actions to achieve those aims. For actions requiring significant inputs of labor, the program will facilitate their rehabilitation through food for work. UBALE’s theory of change holds that the program’s outcomes cannot be achieved without transformational change in the power dynamics between women and men within households and communities, and this applies across all Purposes. It also recognizes that gender roles are not static, and seeks to build on and leverage changes already taking place in many communities thanks to government policy, the attitudes of some traditional leaders, migration, media and other factors. Strong narratives already exist in Malawian society that emphasize solidarity within the household and mutual support between husbands and wives in the face of life’s difficulties. UBALE believes that by leveraging these developments, and applying peer-to-peer approaches wherever possible, individuals and communities will identify and adopt changes in behavior and attitudes that they consider to be appropriate and desirable, thus rendering them ultimately more equitable. ii. Overview of Njira Program The overall goal of Njira is to improve food security among vulnerable households in Balaka and Machinga Districts. Specifically, the project targets rural communities suffering from food insecurity; at risk of suffering from food insecurity; and communities with a high prevalence of chronic malnutrition. Njira’s Purposes and Sub-purposes are as follows: Purpose 1: Increased Income from Agricultural and Non-agricultural Activities Sub-Purpose 1.1: Increased Sustainable Nutrition Friendly and Market Oriented Agriculture Production Sub-purpose 1.2: Increased agricultural and non-agricultural sales Purpose 2: Improved health and nutrition of pregnant and lactating women and children under five Sub-Purpose 2.1: Improved nutrition and health practices Sub-Purpose 2.2: Increased utilization of reproductive, maternal, new-born and child health (RMNCH) prevention and treatment services Sub-Purpose 2.3 Increased utilization of hygiene, sanitation and water facilities Purpose 3: Improve capacity to prepare for, manage and respond to shocks Sub-Purpose 3.1: Improved community and household assets for disaster mitigation Sub-Purpose 3.2 Improved institutional support of Disaster Risk Management (DRM) JMTR Methodology 5 June 2017 11 structures at all levels Sub-Purpose 3.3 Enhanced community empowerment in managing disasters Key cross-cutting priorities include gender equity, governance and environmental management. Njira also works to build capacity building for local groups and organizations and ensure appropriate sustainability and exit plans. Project Concern International (PCI) is the prime awardee and has overall responsibility for management and implementation of the Njira project. PCI directly implements project activities in Balaka District and coordinates closely with its sub-awardee Emmanuel International (EI) which leads implementation of project activities in Machinga District. Njira Target Population by Program Element Figure 2 (below) shows the population served by Njira disaggregated by program element. Purpose 1 (P1) seeks to provide increased access to financial services through village saving and loan (VSL) and new linkages with public and private sector actors, and improve agricultural productivity and participation in markets for 36,929 beneficiaries through the agriculture sector capacity program element. Purpose 2 (P2) aims to support the suite of services within the First 1,000 Days Approach reaching 240,225 beneficiaries in the nutrition program element, 103,165 beneficiaries in the maternal and child health (MCH) program element, 51,497 beneficiaries in the water supply and sanitation (WSS) program element, and 50,701 beneficiaries in the family planning and reproductive health (FPRH) program element. Purpose 3 (P3) is intended to reduce risk and improve the resilience of targeted communities through activities in the natural resources and biodiversity (NRB) program element which will reach 32,542 beneficiaries. P3 also reaches 27,125 individuals through the capacity building, preparedness and planning (CBPP) program element and 9,310 beneficiaries through the social assistance program element. Njira Approaches Central to Njira’s Theory of Change (TOC) is the use of Tailored Pathways to empower beneficiaries to better access the wide variety of resources necessary for lasting food security. This approach was taken in response to previous efforts that tended to apply standardized interventions across geographic areas and target populations rather than tailoring interventions in response to context-specific opportunities and challenges. Njira is an attempt to improve on this approach by empowering communities and individual participants to readily access the resources and services needed to achieve lasting improvements in food and livelihood security. Pathways supported under each of the Purpose areas encourage adoption of community-driven strategies based on available assets, evidence of what works within distinct livelihood and agroecological zones, and a participatory approach to continual learning. Figure 2: Njira Participant Targeting JMTR Methodology 5 June 2017 12 Under Purpose 1, Njira will reach 36,929 participants with agriculture and livestock activities aimed at enhancing production, access to nutritious foods and opportunities for improved livelihood security. Activities carried out under P1 support one or more potential “pathways”. These pathways include: 1) increased agriculture and livestock yields; 2) improved access to agriculture and livestock markets; 3) adoption of agroforestry and other climate-smart agricultural practices; and 4) increased household production of nutritious foods. Njira intends to ensure sustainable, nutrition-friendly and market￾oriented household production by providing training and improved access to producer groups and lead farmers. The project also works with VSL groups and proven agro-dealers to improve access to capital and encourage investment in improved agricultural inputs such as drought-tolerant and disease-free seeds and planting materials. Njira also engages lead farmers in the establishment of demonstration plots throughout the target area to encourage replication of improved production practices. Purpose 1 activities also support improved livestock production through a “pass-on” approach to expanded goat and chicken production, and increased opportunities for women’s engagement in pigeon rearing. Profitable engagement in agricultural production is supported through establishment of Njira marketing clubs and linking these groups with private sector buyers of agricultural produce in target communities. Activities carried out under Purpose 2 directly support the Government of Malawi’s “1,000 Special Days” Initiative. Priority focus is on utilizing formative research to promote adoption of improved maternal and child health and nutrition practices among 240,225 direct beneficiaries. The P2 “pathways” are aimed at achieving: 1) improved MCHN practices; and 2) increased access to and uptake of quality services. Care Groups serve as a primary platform for promoting improved practices and behaviors among Njira participants. Care Groups are designed to achieve scale and sustainability of MCHN interventions, particularly SBCC activities supporting adoption of essential nutrition actions (ENA), improved infant and young child feeding (IYCF) practices, and community complementary feeding sessions. The Care Group model is based on the participation of trained Mother Leaders who are responsible for sharing lessons learned with 10-15 neighborhood groups to facilitate behavior change at the household level. Njira has adapted and expanded on the traditional Care Group Model by supporting the establishment of Father Care Group (FCG). These groups are intended to complement mother-led Care Groups by creating awareness and providing training for men to exert a positive influence on household nutrition practices. PCI Malawi has also supported local NGOs in the formation of Youth Groups aimed at advocating for gender equity, improved reproductive health and avoidance of harmful traditional practices. Together, these groups utilize the Care Group methodology to extend beyond nutrition and promote behavior change in critical areas such as family planning, improved access to and uptake of health services, and improved access to water and sanitation infrastructure. Njira also intends to enhance resilience to shocks and stresses among 35,770 households by strengthening their capacity for disaster risk mitigation. The “pathways” through which the project promotes improved capacity include 1) strengthened community mobilization and engagement; and 2) mainstreaming information and analysis relevant to disaster risk reduction among activities of all stakeholders. Under Purpose 3 Njira staff work closely with Civil Protection Committees (CPC) at the Village, Area, and District Levels to implement a training curriculum on disaster risk mitigation that closely aligns with the policies and priorities of the Department of Disaster Management Affairs (DoDMA). A strong emphasis is on working with Village, Area and District CPCs to improve their capacity for good governance. P3 activities also include development of context-specific risk mapping exercises to enable communities to understand, monitor, and formulate collective responses to potential hazards affecting food and livelihoods security. Analysis provided through hazard mapping is intended for use in development of community-based early warning systems, contingency plans, and disaster risk reduction strategies. JMTR Methodology 5 June 2017 13 Table 1: Key Elements of Ubale and Njira Programs Ubale Njira Goal Reduced chronic malnutrition and food insecurity, and increased resilience among vulnerable populations in the districts of rural Blantyre, Chikwawa, and Nsanje in Malawi Improved food security among vulnerable households in Balaka and Machinga Districts Purposes P1: Vulnerable households’ income increased P2: Improved nutritional status among pregnant and lactating women, and children under the age of two. P3: Communities are empowered to contribute to their own sustainable development P1: Increased Income from Agricultural and Non-agricultural Activities P2: Improved health and nutrition of pregnant and lactating women and children under five P3: Improve capacity to prepare for, manage and respond to shocks Geographic Target Areas Rural Blantyre  8 TAs  69 GVH Chikwawa  11TAs  104 GVH Nsanje  8 TAs  102 GVH Balaka  5 TAs  35 GVH Machinga  5 TAs  43 GVH Total Direct Beneficiaries 259,000 households 244,248 beneficiaries Budget $65,000,000 $30,000,000 Commodities (in-kind) 10,600 MT (CSB+ and vegetable oil) 4,866 MT (beans, CSB, vegetable oil) B. Objectives of the Joint Midterm Review The primary purpose of the Joint Midterm Review (JMTR) is to assess the effectiveness of UBALE’s and Njira’s designs and implementation processes. For each project this will include determination of: quality and effectiveness of service delivery systems, how closely activities adhere to original, approved plans; factors influencing the efficiency and quality of project outputs; the level of participant satisfaction with interventions and outputs in target communities; and whether the underlying assumptions associated with the Theory of Change remain relevant. Examination of each of these elements will enable identification of lessons learned, effective means of replicating and scaling up best practices, and necessary modifications to program designs and/or implementation mechanisms to enable achievement of project goals. To understand the factors that either promote or inhibit achievement of project objectives, JMTR team members will interview members from various stakeholders and compare and contrast achievements across different activity sites, beneficiary groups, awardees, and geographic regions. JMTR Methodology 5 June 2017 14 When reviewing UBALE and Njira implementation processes the JMTR will seek to assess the quality of outputs and their likelihood of producing the planned outcomes articulated in the LogFrame, measured by indicators presented in the Indicator Performance Tracking Table (IPTT). At this end, the JMTR team will also review the status of the outcomes that are necessary to achieve the project purposes but produced by external actors. The JMTR will review the indicators to assess how well they are capturing outcomes and whether there are other major outcomes (intended or unintended) need to be captured. In addition, the JMTR team will seek to identify factors in the operating context that appear to promote or impede the intended results, including whether there have been substantive changes in the assumptions supporting project Theories of Change. The objectives of the JMTR are to: 1. Review the theories of change of both projects’ to assess the overall validity of the program strategy, assess the hypotheses, risks, and assumptions and identify potential changes needed (if any) to ensure achievement of project purposes. 2. Assess the quality of program inputs, implementation and outputs in terms of meeting terms agreed with FFP and their acceptability and usefulness to the targeted communities. Identify factors (both internal and external) that appear to enhance or detract from the efficiency, quality, acceptability, and usefulness of the activities’ implemented and the likelihood they will contribute to achievement of project purposes. 3. Present, through qualitative methods and available routine monitoring data, evidence of changes (intended and unintended) associated with program activities and outputs, assess how well the observed changes support the logic of the results frameworks, and identify factors (both internal and external) in the implementation or context that impede or promote the achievement of target results. 4. Review the level and effectiveness of coordination and collaboration with external organizations that are critical to achieve the projects’ purposes and goal. This includes actors that provide complementary services necessary to achieve the project outcomes at the various levels of objective hierarchy, actors that will provide essential services to sustain the outcomes after the end of the two projects, actors that influence people’s access to goods and services, and organizations that promote or impede “enabling environment”. 5. Review systems for capturing and documenting lessons learned and assess the extent to which they are used in project implementation, including feedback from the perspective of stakeholders and participants. Assess processes and use of monitoring and evaluation evidence for adjusting project strategies. Assess how well the project is seeking out, testing and adapting new ideas and approaches to enhance project effectiveness or efficiency. 6. Determine the extent to which outcomes, systems, and services are designed and implemented to continue after the projects end and assess progress made on implementing sustainability strategies. The review will identify the organizations, services and relationships that need to be maintained JMTR Methodology 5 June 2017 15 after the project ends in order to sustain the outcomes, and will analyze the threats to these to formulate recommendations for the project. 7. Determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from project activities. Assess the extent to which project interventions support greater capacity for local governance and address sources of environmental risk. 8. Develop recommendations for mid-course corrections to address JMTR findings and strengthen UBALE’s and Njira’s program designs and/or implementation mechanisms. This will include potential opportunities for improved coordination with government, civil society and/or private actors in the projects’ respective target areas. To accomplish Objective 1, the JMTR team will review the theories of change of the two projects; 2015/2016 National Food Insecurity Response Plan along with other policies and priorities of the Government of Malawi, and recent analysis of the Malawi Vulnerability Assessment Committee (MVAC) to determine what (if any) changes may be needed to project activities and implementation mechanisms to ensure the logframe and program approach are valid for the local context, and ensure that achievement of project objectives and goals is feasible. The team will also review recent trip reports from visiting technical staff and feedback from the USAID Malawi Mission – including OFDA representatives – regarding partner responses to the ongoing drought. While a comprehensive assessment of partner responses to the drought is beyond the scope of the JMTR, the team will consider the effectiveness of adaptations made to program designs and/or implementation mechanisms to help target communities mitigate the food security impacts of the drought. To accomplish Objective 2, team members will review technical guidance on key processes and approaches used by the projects, observe learning sessions (e.g. agricultural demonstrations, technology transfer, growth monitoring sessions, and mothers’ group sessions, disaster management planning), training sessions, interview facilitators and participants to assess the quality of technical interventions. The review team will also consider:  Staffing and activity resources  Communication – internal and external  Community participation  Coordination, Collaboration and Integration (internal and external)  Participant targeting (especially overlap/consistency with other sectors)  Management of food and non-food commodities  Transfers of entitlements (food, non-food,)  Branding/Marking  Partnerships and linkages  Consortium management  Routine monitoring and evaluation for all activities  Cash flow management  Exit/Sustainability strategies JMTR Methodology 5 June 2017 16 Regarding Objective 3, the JMTR team will review quarterly and annual reports, analyze performance monitoring data collected by the projects (if necessary), as well as review the performance management plan and IPTT. JMTR team members will look for evidence, through qualitative methods (i.e. interviewing project participants, visiting their homes, and activity sites) and available routine monitoring data, of whether and how members of target groups have changed their ideas, attitudes, and practices in any way since program initiation, and acquire potential reasons for why some beneficiaries have started applying project promoted practices while others have not. For each technical sector, team members will examine the extent to which activities and outputs demonstrate a commitment to the cross-cutting issues of gender, governance and the environment. For Objective 4, the JMTR team will review the list of projects and the services they provide to the Ubale and Njira target population, the government services, discussions with the project management and technical teams, and USAID Mission and review of the Theories of Change. To answer these questions, the JMTR team members will review the SBCC strategies implemented by FFP partners in target districts, interview direct and indirect beneficiaries, review monitoring data, examine the quality of the SBCC sessions, assess the knowledge of the communicators/facilitators, and the appropriateness of the messages and other communication materials. To answer the first question, the JMTR team will look for early signs of change in practices and behaviors, especially among individuals and households who directly benefit from project services. For example, these would include:  Changes in practices, attitudes, and knowledge that individuals have made themselves or observed in others  Changes in nutritional status of the children under five based on the growth monitoring data, and growth charts  Changes that have occurred in the ecological, social or household environment  Factors (especially those related to the program) that seem to have promoted the changes  Barriers to changes intended by the program  Evidence of potential harm from program activities  Conditions that have or would likely promote or discourage continuation of observed changes Regarding Objective 5, the JMTR team will interview project leadership, technical staff, the leadership of partner organizations and knowledge management staff; and review documents and reports. Emphasis will be placed on the collection, analysis and management of data to enable iterative learning and evidence-based improvements to project design and implementation mechanisms. To accomplish Objective 6, the JMTR will review the sustainability strategy, interview staff and management, interview project participants, and private and public stakeholders. Specific attention will be paid to the appropriateness, efficiency and efficacy of capacity building activities targeted to participants and local partners. For Objective 7, the JMTR team will interview program staff, participants and non-participants and review the gender assessment report, baseline study results, monitoring reports and other relevant program documents to determine how effectively the projects are responding to the cross-cutting JMTR Methodology 5 June 2017 17 themes of gender, governance and the environment. The MTR will talk to male and female participants to understand how the activities promoted by the projects and their participation in these activities effecting their lives and livelihoods. Finally, to accomplish Objective 8, MTR members will interview key staff and stakeholders to capture their perspectives on potential improvements to project design and implementation mechanisms and apply knowledge of best practice in other food and livelihood security initiatives to formulate recommendations for improving the effectiveness of both DFAPs. C. Key Review Questions, Illustrative Methods and JMTR Team Composition The Joint Midterm Review will assess the efficiency, effectiveness and acceptability of project design and implementation mechanisms and utilize findings to develop recommended improvements in support of desired outcomes. To do so, the JMTR will consider the perspectives of various stakeholders (e.g., implementing staff members, participants, other community members, local experts, external partners, etc.) on project interventions and implementation mechanisms to understand what they think is working, what adjustments should be made, and why. i. Key Review Questions Specific questions considered by the JMTR team will align with one or more of the specific objectives stated in Section B. The following questions are illustrative, and are meant to inform, rather than constrain, primary data collection by the JMTR team. The JMTR will be an iterative review and the focus of interviews and data analysis will be continually refined throughout the process. The overarching questions for the JMTR are: 1. To what extent has the project adhered to the initial technical approach, service delivery approach, implementation plan, outputs, and participant targets included in the initial technical narrative? What factors have promoted or inhibited adherence to initial plans? How have problems and deterrents been managed? (Objectives 1-3) 2. What are the strengths and challenges of the project design and management processes so far? Are the assumptions made in the theory of change that informs program design still valid? What key factors of project management are associated with positively and negatively influencing the quality and sustainability of planned outputs? How have UBALE and Njira management and technical specialists used data to inform programmatic decisions, referral and follow up? To what extent have program design and management processes incorporated measures to improve sustainability of program outputs and outcomes? What processes have been instituted to improve data collection and data quality? (Objectives 3-6) 3. In each technical sector, what are the strengths and challenges to the program inputs, implementation of activities and processes and the quality and sustainability of outputs? How well do implementation processes adhere to internationally acceptable technical standards (for sustainable social and behavior change communication, experiential learning or learning-by-doing) and proven approaches, methodology and processes? Are SBCC strategies messages consistent, JMTR Methodology 5 June 2017 18 adequately contextualized, adequately disseminated and communicated? What commercial opportunities have been created to enable beneficiaries to participate into value chains? How they could be improved? (Objectives 1,2, 6) 4. What changes do community members and other stakeholders associate with project activities? Which factors appear to promote the apparent changes, and which have deterred intended changes? How do the changes correspond to those hypothesized by the projects’ results framework? (Objectives 3,6,7) 5. How effective are program designs and implementation mechanisms in addressing the cross-cutting issues of gender, governance and the environment? What (if any) challenges have projects encountered in these areas that may not have been anticipated in the project design, and how have the projects responded? To what extent do project activities and implementation mechanisms reflect integration of these cross-cutting priorities? What steps have the projects taken to ensure that staff have adequate capacity for addressing these cross-cutting issues? (Objectives 7) 6. Based on the findings from 1-5, above, how could the projects be modified to improve acceptability among targeted communities or the efficiency and effectiveness of project design and implementation? How should the projects’ Theories of Change and results frameworks be refined or modified? (Objectives 1,4,8) ii. Methods and Information Sources Given the focus of the JMTR on assessing project designs and implementation processes, the team will employ qualitative methods for analyzing both secondary and primary data. Prior to and during field work, the team will review available documentation of project activities to gain an understanding of the nature of interventions (training, capacity building exercises, food distribution, etc.), the outputs achieved, geographic location, timing of implementation and numbers of participants reached. Team members will review and analyze project data from the following sources:  Approved Project Technical Narratives  Results Frameworks  Indicator Performance Reference Sheets  Baseline Study Report  Annual Results Reports  Gender Analysis Reports  Barrier Analyses Reports  Site Activity Records and Training Summaries JMTR Methodology 5 June 2017 19  Participant registration data  Environmental Mitigation and Monitoring Plan (EMMP)  Data Quality Assessments  Trip reports, other available and relevant project documentation The principal means of collecting primary qualitative data for the JMTR will be semi-structured interviews and direct observation of sites and activities (e.g. training, growth monitoring, food distribution, cash-for-work projects, etc.).1 At a minimum participants in key informant interviews (KII) and focus group discussions (FGD) will include:  Project managers, technical advisors, field staff, implementing partner representatives  Government officials (Ministry of Health, Ministry of Agriculture, etc.), health service providers, HSAs, agricultural extensionists, VDC members, village leaders, other NGOs operational in the area etc.  Key personnel from USAID Malawi Mission, Feed the Future and other USG programs operating in the area  Individuals and groups (project participants and non-participants) in target communities  Observation at sites of program activities, farmer field school and other technology transfer models, health services, growth monitoring, training, food distribution, food storage, etc. Team members will interview awardee staff members, external stakeholders, project participants and individuals who live in targeted communities but have not directly participated in program activities. Non-participants’ perspectives are important to assess activity coverage (looking both for evidence of inclusion and exclusion errors) and to get “outsiders” impressions of the appropriateness, effectiveness and potential for sustainable impact of project activities. USAID/FFP will work directly with implementing partners to ensure that, where needed, the JMTR team has access to translators to help facilitate interviews with participants in the local language. iii. Review Team Composition and Roles The core JMTR team will consist of approximately 8 or 10 members from USAID/FFP technical team, technical specialists from implementing partner organizations’ headquarters/ regional office/ country office who are experts on specific technical sectors and who do not or did not have significant responsibility in the design or implementation of the projects technical experts from other USAID bureaus or offices, USAID Regional Office and Mission. The team members will be selected based on the technical sectors to be reviewed. The team as a whole will comprise expertise in each of the following areas: 1 Project staff will be asked to categorize target communities in terms of uptake of project-promoted practices. This will help purposive sampling of activity sites for qualitative investigation. JMTR Methodology 5 June 2017 20 a) Food security and livelihoods b) Social and behavior change communication c) Market, value chain and livelihoods d) Off farm livelihoods e) Maternal and Child Health, Nutrition f) Agriculture and Food Security g) Monitoring and Evaluation h) Gender i) WASH j) Asset Creation and Maintenance k) Institutional capacity development and community participation All team members should be skilled in qualitative methods of inquiry and analysis, willing to travel to remote areas, willing to work during the weekends, able to perform under pressure and long hours, and have demonstrated ability to meet deadlines. The team members will be under the direction of a team leader from USAID/FFP. The team leader will work with the external consultants in coordinating all evaluation activities, supervising the team, achieving the specified objectives, cooperating with all parties, and presenting JMTR results in a format and timeline that are useful for the improvement of program implementation. The team leader will also supervise development and presentation of JMTR recommendations. D. CRS and PCI Responsibilities UBALE and Njira project staff (including CRS, PCI and partner representatives) will serve as informants to the JMTR and support the review process by supplying lists of project sites, sharing project documents, advising about local protocols, making presentations, arranging meetings with the stakeholders, and making logistical arrangements. CRS and PCI will provide the following materials and documents to the JMTR team members, assuming the information is readily available and will not require significant preparation time from program staff: 1. Lists and locations of all target communities by activity or group of activities, implementation start dates and numbers of direct and indirect beneficiaries for all activities; 2. Updated theories of change with highlighting the changes 3. Maps showing program area, food distribution points, and community-level activity locations and coverage by SO 4. Program Proposal Narrative and relevant Attachments JMTR Methodology 5 June 2017 21 5. Annual Reports and PIRS defining the calculations and sources of annual indicators 6. Pipeline and Resource Estimate Proposal narratives 7. Program routine monitoring records and reports 8. Program monitoring tools, including the IPTT and Performance Monitoring Plan 9. M&E plan and lists of recipients of routine monitoring reports and description of data bases and contents 10. Organogram of program staff showing supervision/management roles and partnerships for each sector 11. Reports from formative research and barrier analyses 12. Lists and descriptions of program capacity building activities, identifying specific committees, task forces, working groups, etc., those are the focus of intervention 13. 2015/2016 National Food Insecurity Response Plan 14. Malawi Vulnerability Assessment Committee (MVAC) 15. A map of all other projects operating in the area with the list of key activities that they implement 16. Trip reports from visiting technical staff and feedback from the USAID Malawi Mission 17. Technical guidance on key processes – all manuals, handbooks, and guidance – A list of all different approaches with brief description In addition, CRS and PCI will provide professional interpreters (as required) for team members and facilitate in-country travel and logistical arrangements for the JMTR team members including vehicles, printing/copying and access to data bases. Members from USAID will pay for their own accommodation and food, however they would need help with the hotel booking. E. Reporting Observation and Analyses: Members of the JMTR team will record their observations and analyze their findings. Each member will produce a brief informal report that will include key observations analysis and conclusions that are directly linked to individual recommendations. Ground truthing of the observations and interpretation: At the end of the field work in each of the project areas, the review team will organize a two-day workshop with the key staff (i.e. selected field staff, technical staff, and project management) to present and discuss the observations. This will allow the project staff to validate the observations of the JMTR team. In case of major disagreements, the project staff should provide evidence in support of the argument and the review team may revisit the field. JMTR Methodology 5 June 2017 22 F. Deliverables The following products will be developed by the JMTR team as a means of informing follow-up on the part of FFP and implementing partners: a) An informal brief report for each technical and cross cutting areas including key observations and findings, analyses and key recommendations; b) A prioritized list of actionable recommendations; c) A learning workshop to present key learning from the JMTR, share key prioritized recommendations, and formulate develop action plans. The two external consultants (to be contracted by CRS and PCI) will lead the field work and analysis of MTR findings and will be responsible for preparing the final JMTR Report. G. Timeframe The collection of primary qualitative data will commence in the beginning of March 2017. An illustrative list of Activities, Responsibilities and Timeline is Annex 1. The timeframe for the JMTR allocates sufficient time for validation by CRS and PCI staff ensures adequate time to capture and reconcile negotiations between FFP and field staff aimed at forming actionable and feasible JMTR recommendations. JMTR Methodology 5 June 2017 23 H. Ethical Guidelines Every member of the JMTR team must adhere to ethical guidelines as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below. A more detailed description can be found at http://www.eval.org/p/cm/ld/fid=51 1. Systematic Inquiry: Reviewers conduct systematic, data-based inquiries. 2. Competence: The review team possesses the education, abilities, skills and experience appropriate to undertake the tasks proposed in the midterm review. Team members work within the limits of their professional training and competence, and decline to review activities and plans that fall substantially outside those limits. The JMTR team collectively demonstrates cultural competence. 3. Integrity/Honesty: Individual reviewers display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire review process. 4. Respect for People: JMTR members respect the security, dignity and self-worth of respondents, project participants, clients, and other stakeholders. Reviewers regard informed consent for participation in interviews and inform participants and clients about the scope and limits of confidentiality. 5. Responsibilities for General and Public Welfare: Reviewers articulate and take into account the diversity of general and public interests and values that may be related to the midterm review. JMTR Methodology 17 May 2017 24 Annex 1: Malawi Joint Midterm Review (JMTR) Activities, Responsibilities and Timeline Activities Respon sible Time Comments Initiate discussions with Partners FFP (AoR) Thursday, June 16 Initial discussion of JMTR including Ubale and Njira COP, PCI HQ, FFP Malawi and DC. Set prospective timeline for draft SOW Draft SoW and submit to AOR CRS, PCI Anticipated date: July 28 SOW should be based on SOW for Guatemala JMTR and in alignment with FFP M&E and Reporting Policy and Guidance. Review draft SoW and forward comments to partners FFP August 15 FFP DC will review SOW and share with Malawi Mission and Regional M&E Specialist for comment Revised SoW CRS, PCI August 26 CRS and PCI make revisions to draft SoW based on FFP comments Finalize SoW FFP, CRS, PCI October 7 All parties will coordinate to finalize the SoW based on agreed JMTR questions and processes Define composition of JMTR team (# of members, technical expertise, office affiliation, involvement in projects, evaluation experience, etc.) FFP September 29 FFP DC will provide specific criteria for JMTR team that Partners can use to identify and nominate potential members JMTR pre- planning meeting (as needed) FFP, CRS, PCI September 9 Purpose of meeting would be to jointly discuss team composition, SoW, logistics, expectations, and field work schedule Identification / nomination of Partner JMTR team members CRS, PCI November 1 Partners provide FFP with preferred JMTR team members Finalize composition of JMTR team FFP, CRS, PCI November 15 Confirmation of JMTR members Development of draft JMTR protocol and share with the partners FFP, Mission December 1 FFP to share draft JMTR with Partners for review, comment Finalize JMTR protocol FFP, Mission, CRS, PCI December 15 FFP DC will finalize JMTR protocol based on comments, clarifications requested by Malawi Mission and Partners Meeting/ call among JMTR members to identify the document list for review FFP January 15 The JMTR team leader will arrange a meeting/ conference call among the team to identify a list of documents need Provision of background documentation relevant to JMTR CRS, PCI, Mission February 1 Relevant documentation will include (but not be limited to): approved project proposals; updated ToC, LogFrame, IPTT, M&E Plan; assessment reports, Annual Results Reports (ARR), thematic studies, etc. JMTR Methodology 17 May 2017 25 Activities Respon sible Time Comments An initial presentation by the AoR and the Mission to the JMTR team FFP February 15 The AoR in coordination with the Mission will make an initial presentation to the JMTR team. Identification and categorization of target communities CRS, PCI February 20 Partner field staff provide a comprehensive list (Excel sheet) of target communities (with all relevant administrative information – district, CHV, etc) and maps with community location for use in qualitative sampling. Information provided to include: early vs. slow adopters; activities implemented, remoteness, inception / duration of project participation, local partner participation. Review relevant project documentation literatures and develop a list of questions for partners (in case of information gaps and/ or clarifications needed) JMTR Team February 20 JMTR team members review available project documentation and identify existing information gaps / needs for clarification Development of qualitative JMTR tools JMTR Team February 20 FFP, CRS, PCI team member jointly develop interview guides (topical outlines) with prospective questions for Key Informant Interviews (KII) and Focus Group Discussions (FGDs) Selection of JMTR qualitative sample FFP February 25 Based on information provided, the JMTR team leader will select the qualitative sample to enable collection of representative data and completion of field work in the time allowed Logistical arrangements made for JMTR field work CRS, PCI, Mission March 1 Partners to work with Mission to finalize logistical arrangements for JMTR field work including: vehicles, local translators, meeting and workshop venues, security briefing, mapping of selected JMTR communities. Both of the partners will designate a JMTR point person who will be the principal contact for JMTR team members. Briefing meeting FFP, Mission, CRS, PCI March 6 FFP DC and Mission to review/clarify objectives, methods, deliverables and proposed timeline of JMTR. Partners will make brief presentations on project objectives, activities, strengths, challenges, lessons learned. Field work JMTR Team 10 working days (5 days for each project) Anticipated that field work for individual projects will be conducted sequentially (e.g. beginning with Ubale, then Njira) JMTR Methodology 17 May 2017 26 Activities Respon sible Time Comments Ground Truthing Workshops FFP, Mission, CRS, PCI After completing field work for each project In order corresponding with field work. Purpose of Ground Truthing Workshops is to validate / solicit Partner response to preliminary JMTR findings and address existing information gaps. Analysis and synthesis of the findings JMTR Team One month JMTR team members analyze their individual data and that shared by other team members, as well as clarifications provided during Ground Truthing Workshop Development of draft JMTR recommendations JMTR Team April 15 Based on analysis of qualitative data and project documentation, team members develop draft JMTR recommendations within their respective technical areas In-country JMTR Workshop Recommendation, learning and follow Up Plan workshop FFP, Mission, CRS, PCI May 1-3 Final meetings held with Mission and Partners (over three-day period) to finalize recommendations, discuss lessons learned, and formulate plans for follow-up De-briefing to DC based stakeholders FFP, CRS, PCI May 15 Present final recommendations and debrief headquarters personnel on conclusions/next steps resulting from in-country workshop Draft an action plan and submit to FFP CRS and PCI June 1 Partners will submit the action plan to AoR Review / revise action plan with CRS HQ and key CRS Malawi staff FFP, CRS June 15 Meet in Washington DC (with Malawi staff joining remotely) to finalize action plan Review / revise action plan with PCI HQ and key PCI Malawi staff FFP, PCI June 15 Meet in Washington DC (with Malawi staff joining remotely) to finalize action plan JMTR Methodology 17 May 2017 27 II. UPDATED JMTR PROTOCOL WITH INFORMATION ON ACTUAL PROCESS USED JMTR Methodology 17 May 2017 28 Protocol for a Joint Mid-Term Review of the FFP-funded Development Food Assistance Projects in Malawi I. INTRODUCTION In Fiscal Year 2014, the U.S. Agency for International Development (USAID) Office of Food for Peace (FFP) awarded funding to private voluntary organizations (PVOs) to implement multi-year Development Food Assistance Projects (DFAPs) in the most food-insecure regions of Malawi. FFP issued awards to Catholic Relief Services (CRS) for the UBALE Project to be implemented in Nsanje, Chikwawa and Blantyre Districts and to Project Concern International (PCI) for the Njira Project to be implemented in Balaka and Machinga Districts. Both projects began implementation on early October, 20142 , and are approaching the mid-points of their planned Life of Activity (LoA). A Joint Mid-Term Review (JMTR) is being organized for a team of twelve core evaluators including two independent consultants, representatives from FFP and the USAID Bureau of Food Security, and a representatives from CRS, PCI or their partners to work with project implementing staff to review the projects and formulate realistic and achievable recommendations for improving the effectiveness and efficiency of the two projects over the remainder of their implementation. This protocol describes the plans and procedures that will be used to implement the JMTR. II. JMTR PURPOSE & OBJECTIVES A. Purpose The JMTR is a formative evaluation exercise intended to review the progress of the two projects in producing planned outputs, to assess the intended and unintended effects of these outputs, and to examine the quality of various processes being used to implement the program to formulate recommendations to be implemented in the remaining life of the program. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) piloting new interventions relevant for targeted impact groups, (e) improving the effectiveness of implementation systems, or (f) improving efficiency in use of resources. The evaluation process will tailor and prioritize recommendations for each project and ensure that they are implementable within the time frame remaining and with the resources available to the two projects. The process will also facilitate the sharing of ideas on good practices between the two projects. B. Specific Objectives The objectives of the JMTR are to: 9. Assess the overall strategy of each project in terms of its relevance for addressing food insecurity with targeted impact groups, taking into account contextual changes that may have occurred since the projects began implementation. This will entail reviewing the strategies that ensure that targeted impact groups are reached by the projects, reviewing the theories of change, and assessing the hypotheses, risks, and assumptions made during the design of the projects. Key Overarching Questions. What are the strengths and challenges of the project design so far? Are the assumptions made in the theory of change that informs program design still valid? What changes have occurred in the context since the project began implementation that have resulted in new or changed impact groups or the need for new types of assistance to address food insecurity. To what 2 The start date for UBALE was 28 September 2014 and for Njira was 1 October 2014. JMTR Methodology 17 May 2017 29 extent have program design processes incorporated measures to improve sustainability of program outputs and outcomes? How have the project strategies been designed to put in place the elements needed to contribute to higher-order social change? How should the projects’ Theories of Change and results frameworks be refined or modified? 10. Assess the quality of project inputs, implementation and outputs to identify factors that enhance or detract from the efficiency, quality, acceptability, and effectiveness of the activities’ implemented and the likelihood that they will contribute to sustained achievement of project goals. Key Overarching Questions. In each technical sector, to what extent has the project adhered to the initial technical approach, implementation plan, outputs, and participant targets included in the initial technical narrative? What are the strengths and challenges to the program inputs, implementation of activities and processes, the quality of outputs and the sustainability of the outcomes achieved? How have problems and deterrents been managed? How well do implementation processes adhere to internationally acceptable technical standards (for sustainable social and behavior change communication, experiential learning or learning-by-doing) and proven approaches, methodology and processes? Are the messages conveyed consistent, adequately contextualized, adequately disseminated and communicated? What commercial opportunities have been created to enable beneficiaries to expand engagement in value chains? How could they be improved? 11. Present, through qualitative methods and available quantitative data, evidence of changes3 (intended and unintended outcomes) associated with project activities and outputs, assess how well the observed changes support the theory of change and logic of the results frameworks, and identify factors (both internal and external) in the implementation or context that impede or promote the achievement of targeted results. Key Overarching Questions. What changes do community members and other stakeholders associate with project activities? Which factors appear to promote the apparent changes, and which have deterred intended changes? How do the changes correspond to those hypothesized by the projects’ results framework? 12. Review the level and effectiveness of coordination and collaboration with external organizations that are critical to achieve the project goals and purposes. This includes actors that provide complementary services necessary to achieve the project outcomes, actors that will provide essential services to sustain the outcomes after the end of the two projects, actors that influence people’s access to goods and services, and organizations that promote or impede an “enabling environment”. Key Overarching Questions. What has been the impact of the various collaborative relationships cultivated by the project toward enhancing the effectiveness of the project or efficiency in use of resources? How aligned are the strategies of the projects toward the development strategies of USAID and the GoM? What changes can be made in these collaborative relations to further enhance effectiveness and efficiency? 13. Related to collaborative learning and action, review systems for capturing and documenting lessons learned and assess the extent to which they are used in project implementation and refining project design, including feedback from the perspective of stakeholders and participants. Assess processes and use of monitoring and evaluation evidence for adjusting project strategies. Assess how well the project is seeking out, testing and adapting new ideas and approaches to enhance project effectiveness or efficiency. 3 These changes can occur at the individual, household, community and higher levels, including systemic changes. JMTR Methodology 17 May 2017 30 Key Overarching Questions. How have UBALE and Njira management and technical specialists used data to inform programmatic decisions, referral and follow up? What processes have been instituted to improve data collection and data quality? How has the project improved effectiveness or efficiency as a result of new ideas or approaches brought int the project? How is information generated by the project used to inform decision-making? How can this be made more effective? 14. Related to sustaining project impact, determine the extent to which outcomes, systems, and services are designed and being implemented to continue after the project ends and assess progress made on implementing sustainability strategies. The review will identify the organizations, services and relationships that need to be maintained after the project ends in order to sustain the outcomes and will analyze the threats to these to formulate recommendations for strengthening the sustainability strategies of the project. Key Overarching Questions. With the changes that have been observed as a result of project activities, what organizations, services or relationships are required to sustain these changes? What resources and capacities do organizations or service providers need to sustain themselves and these relationships after the project ends? What has the project done to ensure that these capacities and resources will be available after the project ends? What will motivate these organizations and service providers to continue to support the changes? What has the project done to ensure that this motivation does not diminish after the project ends? 15. Relative to the major cross-cutting themes in both projects, determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from project activities. Assess the extent to which project interventions target youth (UBALE only), support greater capacities for local governance and address sources of environmental risk. Key Overarching Questions. How effective are program design and implementation mechanisms in addressing the cross-cutting issues of gender, governance, the environment and targeting of youth (UBALE)? What (if any) challenges have projects encountered in these areas that may not have been anticipated in the project design, and how have the projects responded? To what extent do project activities and implementation mechanisms reflect integration of these cross-cutting priorities? What steps have the projects taken to ensure that staff have adequate capacity for addressing these cross￾cutting issues? 16. Review implementation systems, including management systems, partner relations, commodity management systems, financial management systems, human resource management systems and materials management systems to identify recommendations for improving the effectiveness of implementation and/or the efficiency in use of project resources. Key Overarching Questions. What key factors of implementation systems are associated with positively and negatively influencing the quality and sustainability of planned outputs? How could the projects be modified to improve the efficiency and effectiveness of implementation? III. PROJECT DESCRIPTIONS Brief descriptions of the two projects are provided below. The information that is provided focuses on information about the projects relevant for planning the JMTR, e.g., the types of outputs being produced by each project, the types of participants that will need to be interviewed, and the key partners that will also need to be interviewed. Prior to undertaking the JMTR, team members will review key background documents to understand the theories of change for each project, the approaches that are being used in producing outputs, and the outcomes and impact that are expected from the achievement of outputs. JMTR Methodology 17 May 2017 31 A. The United in Building and Advancing Life Expectations (UBALE) Project Under the leadership of Catholic Relief Services (CRS), the UBALE Project is being implemented in all 26 rural Traditional Authorities in the three districts of Rural Blantyre, Chikwawa and Nsanje by a consortium composed of CRS, CARE, the Chikwawa Catholic Diocese, the National Cooperative Business Association/CLUSA International (NCBA CLUSA), the National Smallholder Farmers Association of Malawi (NASFAM), and Save the Children. Table 1 shows the current Goal, Purposes and Sub-Purposes for the project4 . The total number of direct beneficiaries targeted by the project is 248,200 households (over 1.2 million people). The total proposed cash budget for the project is USD 61,838,910, and 10,600 MT of commodities are planned to be distributed over the LoA in Food for Assets activities and conditional supplemental rations for pregnant or lactating women, children under two, and their families. The approaches used and expected outcomes in the UBALE Project are described in the original project document, and the performance indicators at each level are described in the project's Indicator Performance Tracking Table (IPTT). The sections which follow list the major types of outputs for each of the three purposes along with the major types of participants. The last section provides an overview of the partners in the UBALE Project. Table 1. Goal, Purposes and Sub-Purposes for the UBALE Project GOAL: Reduced chronic malnutrition and food insecurity, and increased resilience among vulnerable populations in Rural Blantyre, Chikwawa, and Nsanje Districts in Malawi. Purpose 1: Vulnerable households' income increased Purpose 2: Nutritional status of pregnant and lactating women and children under the age of two improved Purpose 3: Communities are empowered to contribute to their own sustainable development Sub-Purpose 1.1 Farm management and business planning practices of vulnerable farming households improved Sub-Purpose 2.1 GoM MCHN/WASH systems management practices are improved Sub-Purpose 3.1 Improved governance, risk reduction and environmental practices Sub-Purpose 1.2 Marketing club, cluster and association members' agricultural sales increased Sub-Purpose 3.2 Community have productive assets Sub-Purpose 1.3 Members of marketing clubs, clusters and associations increased use of sustainable financial services Sub-Purpose 2.2 Targeted households adopt evidence￾based behaviors that reduce the prevalence of malnutrition. Sub-Purpose 3.3 Women participate in decision-making structures Sub-Purpose 1.4 Selected individuals establish/strengthen businesses Sub-Purpose 2.3 Women of reproductive age have improved agency Sub-Purpose 1.5 Joint decision-making at household-level improved Purpose 1 Income. Under this component, the UBALE Project plans to reach a total of 134,000 households, including 31,000 households with direct agricultural asset transfers and 103,000 households through Lead Farmer services. At least 59,000 households drawn from these two groups will be targeted for improved farm management capacity building and 29,000 households will be engaged in marketing capacity building. In addition, a total of 175 men and 94 women employed by the GoM in critical roles to support project activities are targeted for capacity building by the project. Major Outputs. Following are the major clusters of activities designed to produce outputs in the UBALE Project under Purpose 1. 4 These differ from the original proposal as a result of changes made following the M&E Workshop held at the beginning of the project. JMTR Methodology 17 May 2017 32  Capacity building of farmers on risk reduction practices and actions to improve resilience to climate change  Capacity building of farmers on sustainable agriculture and improved storage practices or technologies  Capacity building of farmers for improved farm management or business planning  Organization and implementation of Diversity and Nutrition for Enhanced Resilience (DiNER) Fairs  Formation of new or strengthening of existing Savings and Internal Lending Community (SILC) Groups  Capacity building of community-based Private Service Providers (PSPs) to promote and support SILC  Formation and capacity building of PSP networks  Promotion of small and micro-enterprises to support targeted value chain development  Capacity building of Agriculture Extension Development Officers (AEDOs) and Lead Farmers  Formation and capacity building of seed multiplication groups for orange fleshed sweet potato (OFSP), nutrition-improved Andean (NUA) beans, fruit trees or cassava  Capacity building of community-based paravets  Formation and capacity building of Stakeholder Panels at the District and Area levels  Creation of Learning Centers  Formation of new or strengthening of existing marketing clubs and marketing clusters  Formation and capacity building of marketing associations  Capacity building of marketing club members on improved marketing planning and practices  Identification and capacity building of Market PSPs  Capacity building of selected participants for business development  Capacity building of marketing clubs/clusters/associations for developing business plans for obtaining financial assistance  Facilitation of linkages between marketing clubs/clusters/associations and MFIs or insurance providers  Capacity building of women and men to promote joint household decision-making Types of Participants. Table 2 outlines the different types of participants engaged and sites developed in the UBALE Project. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 2. Participants and Sites in the UBALE Project Under the Income Component (Purpose 1) Type of Participant/Site Male Female New Continuing from WALA LOA Total SILC Groups 3000 970 3970 SILC PSPs 85 75 160 PSP Networks 3 3 Farm Demonstration Plots (Mama) 206 206 Farm Demonstration Plots (Baby) 2064 2064 GoM AEDOs 65 60 125 Lead Farmers 1030 1030 2060 JMTR Methodology 17 May 2017 33 DiNER Fairs 62 62 Seed Multiplication Groups 275 275 Paravets 60 30 90 District and Area Stakeholder Panels 15 15 Learning Centers 9 9 Marketing Clubs 1200 350 1550 Marketing Clusters 240 66 306 Marketing Associations 16 16 Market PSPs 64 43 107 Beneficiary-owned Businesses 350 350 Basic Literacy Participants (groups assisted to register with GoM services) No Target Agro-Dealers (in proposal) No Target Buyers with Marketing Linkages to Participants No Target Service Providers - Agro-Input Suppliers, MFIs, Micro-Insurance Firms, BDS Service Providers No Target Purpose 2 MCHN. Under this component, the UBALE Project uses a Care Group Approach to reach over the LoA a total of 124,000 households with a pregnant or lactating woman. UBALE plans to distribute conditional supplementary food rations to a total of 158,800 mother-child pairs, including those targeted for participation in Care Groups. Major Outputs. Following are the major clusters of activities designed to produce outputs in the UBALE Project under Purpose 2.  Capacity building of District and TA level MCHN structures on improved supervision, communication, reporting and good governance.  Formation of new and strengthening of existing Water User Committees  Formation of a Care Group structure, including identification of Lead Parents  Capacity building of Lead Parents on essential nutrition actions and health decision-making  Capacity building of care givers on essential nutrition actions through Care Groups and Community Complementary Feeding and Learning Sessions (CCFLS)  Formation and capacity building of Community Leaders for Action on Nutrition (CLAN)  Formation and capacity building of Grandmother Groups  Capacity building of caregivers in Community-Based Childcare Centers (CBCC)  Distribution of conditional supplementary food rations to mother-child pairs  Capacity building of women and men to promote joint household decision-making on MCHN￾related decisions  Capacity building of project staff, health facility staff and health surveillance assistants (HSAs) on youth-friendly approaches  Capacity building of youth club members on good pre-conception health behaviors Types of Participants. Table 3 lists the different types of participants engaged and sites developed in the UBALE Project. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. JMTR Methodology 17 May 2017 34 Table 3. Participants and Sites in the UBALE Project Under the MCHN Component (Purpose 2) Type of Participant/Site Male Female New Continuing from WALA LOA Total District and TA MCHN/WASH Structures for the GoM 20 20 Water User Committees 1200 1200 Care Groups 718 718 Lead Parents 710 6470 7180 CLANS 166 332 Grandmother Groups ??? Nutrition Fairs 1022 1022 Caregivers in CBCCs 340 1360 1700 Food Recipients 158,000 Youth Club Members (Pre-Conception Health) 560 840 1400 Participants in Gender Champion Dialogue Sessions 1000 8600 9600 Purpose 3 Community Resilience. The UBALE Project will target 186,100 households to benefit directly or indirectly from disaster preparedness, risk mitigation and risk reduction activities, and 23,000 households will receive rations through Food for Assets activities designed to build community resilience. Major Outputs. Following are the major clusters of activities designed to produce outputs in the UBALE Project under Purpose 3.  Capacity building of Village Civil Protection Committees (VCPCs), Area Civil Protection Committees (ACPCs), Village Development Committees (VDCs), Area Development Committees (ADCs), and Youth Clubs on disaster early warning and response planning, disaster risk reduction, resource mobilization, good governance, environmental protection and participatory planning, monitoring and evaluation.  Capacity building of Natural Resource Management (NRM) Committees on natural resource management, environmental protection, and good governance.  Capacity building of District Executive Committee (DEC) Sub-Committees on good governance and disaster risk reduction.  Distribution of early warning monitoring and reporting toolkits to VCPCs and NRM Committees  Facilitation of Participatory Vulnerability Capacity Assessments  Use of Food for Assets for construction or rehabilitation of community productive assets, including latrines at health facilities, evacuation centers, river bank protection structures, river dykes, watershed development schemes or dams.  Promotion of women's participation in decision-making  Capacity building of Youth Club members, Gender Champions, and community leaders on gender equity and Theater for Development (TfD). JMTR Methodology 17 May 2017 35 Types of Participants. Table 4 lists the different types of groups, participants and sites in the UBALE Project under Purpose 3. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 4. Participants and Sites in the UBALE Project Targeted Under Community Resilience Type of Participant/Site Male Female New Continuing from WALA LOA Total VCPCs 156 64 220 ACPCs 26 26 VDCs 284 284 ADCs 26 26 Youth Clubs 284 284 NRM Committees 125 125 DEC Sub-Committees 6 6 Latrines at Health Facilities FFA Sites 29 29 Evacuation Center FFA Sites 50 50 River Bank Protection FFA Sites 140 140 River Dyke FFA Sites 10 10 Watershed Development Scheme FFA Sites 50 50 Dam FFA Sites 12 12 FFA Participants 2115 2115 4230 Youth Club Members, Gender Champions, and Community Leaders (Gender Equity) 2430 2425 4855 Cross-cutting Themes. The UBALE Project has four major cross-cutting themes, including governance, gender, targeting of youth and promoting resilience to climate change. The first three cut across all three purposes in the project, while the last is concentrated in Purposes 1 and 3. Partner Roles and Responsibilities. The UBALE Project has the following partners. Each of these will be consulted during the JMTR through interviews or remotely depending on time and location. Consortium Partners. At the consortium level, CRS is the lead agency responsible for overall programmatic and financial oversight. CRS also manages (1) the commodity pipeline, (2) program-level knowledge management systems, (3) technical support for implementation in Chikwawa District by the Chikwawa Diocese, and (4) technical leadership for Purposes 1 (Income) and 2 (MCHN). CARE implements UBALE in Nsanje District and provides overall technical leadership on Purpose 3 (Community Resilience) and gender. The Chikwawa Diocese is responsible for implementation in Chikwawa District with technical support from CRS, and Save the Children is responsible for implementation in Rural Blantyre District and facilitates linkages to the SSDI and the Integrating Nutrition in Value Chains (INVC) Projects. NASFAM provides technical support for the formation and capacity building of marketing associations, and NCBA CLUSA oversees development of UBALE's agri-business strategy and provides technical support to NASFAM. Technical Partners. In addition to consortium partners, the UBALE Project also planned to work with a number of technical partners, including the World Agroforestry Center (ICRAF), the International Potato Center (CIP), the International Center for Tropical Agriculture (CIAT), the GoM Department of Agricultural Research Services (DARS) and Digital Green. JMTR Methodology 17 May 2017 36 Research Partners. In defining and implementing a research and learning agenda, the UBALE Project proposed to work with the University of Notre Dame, Tufts University, the Lilongwe University of Agriculture and Natural Resources (LUANAR), the International Crops Research Institute for the Semi￾Arid Tropics (ICRISAT) and Purdue University. B. The Njira Project: Pathways to Sustainable Food Security Project Concern International (PCI), in collaboration with Emmanuel International and Agricane, is leading implementation of the Njira Project in 10 Traditional Authorities in the two districts of Balaka and Machinga. Table 5 shows the current Goal, Purposes and Sub-Purposes for the project5 . Table 5. Goal, Purposes and Sub-Purposes for the Njira Project GOAL: To improve food insecurity among vulnerable populations in Balaka and Machinga Districts in Malawi Purpose 1: Increased income Purpose 2: Improved health and nutrition of pregnant and lactating women and children under five Purpose 3: Improved capacity to prepare for, manage and respond to shocks Sub-Purpose 1.1 Increased sustainable and nutrition￾friendly agricultural production Sub-Purpose 2.1 Improved nutrition and health practices Sub-Purpose 3.1 Improved community and household assets for disaster mitigation Sub-Purpose 1.2 Increased agricultural and non￾agricultural Sales Sub-Purpose 2.2 Increased utilization of reproductive, maternal, new-born and child health prevention and treatment services Sub-Purpose 3.2 Improved institutional support for disaster risk management structures at all levels Sub-Purpose 2.3 Increased utilization of hygiene, sanitation and water facilities Sub-Purpose 3.3 Enhanced community empowerment in managing disasters The total number of direct beneficiaries targeted by the project is 244,248 persons. The total proposed cash budget for the project is USD 30,000,000, and 4,866 MT of commodities are planned to be distributed over the LoA in Food for Assets activities and supplemental rations for pregnant or lactating women, children under two, and their families. The approaches used and expected outcomes in the Njira Project are described in the original project document, and the performance indicators at each level are described in the project's Indicator Performance Tracking Table (IPTT). The sections which follow list the major types of outputs for each of the three purposes and major types of participants. The last section provides an overview of the partners engaged in the Njira Project. Purpose 1 Income. Under this component, the Njira Project plans to reach a total of 36,929 beneficiaries who are members of producer groups organized by the project. Major Outputs. Following are the major clusters of activities designed to produce outputs in the Njira Project under Purpose 1.  Capacity building of producers and others on improved technologies or management practices related to crop genetics, cultivation practices, livestock management, fodder production, crop pest and disease management, conservation agriculture, , irrigation, post-harvest storage and handling, and marketing 5 These differ from the original proposal as a result of changes made following the M&E Workshop held at the beginning of the project. JMTR Methodology 17 May 2017 37  Capacity building of farmers on risk reduction practices and actions to improve resilience to climate change  Promotion of fuel efficient stoves  Capacity building of Community Animal Health Workers (CAHWs)  Capacity building of Producers Associations, Water User Associations, Women's Groups, Other Community-Based Organizations, and Trade and Business organizations for marketing and business development  Implementation of End Market Analyses  Implementation of Rapid Market Analyses  Promotion of joint household decision-making on the use of cash income  Promotion of women in leadership positions in community-based groups  Assistance for communities to identify, demarcate and better manage land  Organization and training of farmers in collective marketing activities  Implementation of fodder production and management training  Creation of community-managed tree nurseries  Creation of kitchen gardens  Promotion of conservation agriculture techniques  Implementation of Doer/Non-Doer (DND) analyses of conservation agriculture uptake  Creation of marketing learning alliances  Implementation of climate change adoption training  Promotion of poultry and small ruminants value chains  Cultivation of linkages between marketing groups and buyers  Formation and capacity building of Women's Empowerment/Village Savings and Loan (WE/VSL) groups  Promotion of bamboo production  Establishment of market-oriented demonstration plots  Distribution of high quality and disease free seed and planting material  Facilitation of access to commercial marketing systems  Establishment of small irrigation schemes  Establishment of micro-watershed management sites  Promotion of crop value chains  Promotion of pigeon production  Development of conservation agriculture plans  Mobilization of communities in micro-enterprise development Types of Participants. Table 6 outlines the different types of participants engaged and sites developed in the Njira Project. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 6. Participants and Sites in the Njira Project Under the Income Component (Purpose 1) Type of Participant/Site Male Female New Continuing from WALA LOA Total Producers Associations 2,535 2,535 JMTR Methodology 17 May 2017 38 Water Users Associations 78 78 Women's Groups 82 82 Kitchen Garden Producers 9,540 9,540 Community-Managed Tree Nurseries 973 973 Irrigation and Watershed Sites Planting Multipurpose Bamboo 60 60 Hectares under Agreement with Communities for Demarcation 18,027 18,027 Households with Fuel-Efficient Stoves 30,000 30,000 Micro-Watershed Management Sites 20 20 Watershed Development Committees 20 20 Small-Scale Irrigation Schemes 78 78 Recipients of seed and other planting Material 13,438 14,558 27,996 Recipients of pigeon distributions 6,400 6,400 12,800 CAHWS 84 84 Individual Farmers Implementing Micro￾enterprises in Promoted Value Chains 200 200 WE/VSL Groups 1,820 --- 1,820 Demonstration Plots 252 252 Village Agents for VSL 200 200 Lead Farmers 84 84 Purpose 2 MCHN. Under this component, the Njira Project plans to reach a total of 240,225 beneficiaries including 103,165 beneficiaries with MCHN interventions, 51,497 beneficiaries with water supply and sanitation interventions, and 50,701 beneficiaries with family planning and reproductive health interventions. Major Outputs. Following are the major clusters of activities designed to produce outputs in the Njira Project under Purpose 2.  Capacity building through a care group structure on essential nutrition actions and essential hygiene actions  Promotion of consumption of targeted nutrient-rich value chain commodities  Distribution of conditional supplementary food rations to pregnant or lactating women, children under two and their families  Capacity building of women and men through couples workshops and community dialogues to promote joint household decision-making on MCHN-related decisions  Strengthening of Reproductive, Maternal, Newborn and Child Health (RMNCH) institutional services, including GMP, Essential Newborn Care (ENC), ANC and Reproductive Health/Family Planning  Support for improved information flows between health facilities and community management of acute malnutrition (CMAM) service providers  Capacity building of Community Leaders for Action on Nutrition (CLAN)  Implementation of participatory nutrition analyses JMTR Methodology 17 May 2017 39  Promotion of and capacity building on Youth Friendly Health Services (YFHS) for health personnel  Promotion of integrated reproductive health and family planning services  Promotion of improved reproductive, maternal, newborn and child health (RMNCH) care￾seeking behavior  Provision of improved equipment and supplies to health facilities  Strengthening of early identification and referral services for growth faltering children  Capacity building of Health Surveillance Assistants (HSAs) on RMNCH  Creation and capacity building of water point committees  Implementation of WASH Campaigns  Implementation of community dialogues  Promotion of Community-Led Total Sanitation  Construction or rehabilitation of sanitation and hygiene facilities  Implementation of hygiene and sanitation education Types of Participants. Table 7 lists the different types of participants engaged and sites developed in the Njira Project. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 7. Participants and Sites in the Njira Project Under the MCHN Component (Purpose 2) Type of Participant/Site Male Female New Continuing from WALA LOA Total Care Groups 200 65 265 Father Groups 199 --- 199 CLANS 631 --- 631 Health Surveillance Assistants (HSAs) 267 --- 267 Water Point Committees 84 --- 84 Village Health Committees 69 15 84 GMP Volunteers 3,180 --- 3,180 CMAM Volunteers 3,180 --- 3,180 District Hospitals --- 2 2 Health Centers 9 6 15 CBCCs 69 15 84 Purpose 3 Community Resilience. The Njira Project will target a total of 32,432 beneficiaries in this component of the project which intends to build resilience capacities of households and communities. This number includes 27,125 individuals participating in natural resource and biodiversity capacity building or disaster preparedness and planning capacity building and 9,310 food for assets participants. Major Outputs. Following are the major clusters of activities designed to produce outputs in the Njira Project under Purpose 3.  Capacity building of Village Civil Protection Committees (VCPCs), Area Civil Protection Committees (ACPCs), and District Civil Protection Committees (DCPCs) on preparing for, managing and responding to shocks, developing Disaster Risk Management Plans, and developing disaster contingency plans.  Support for community implementation of disaster risk management plans. JMTR Methodology 17 May 2017 40  Cultivation of functional linkages between the Department of Disaster Management Affairs (DoDMA), DCPCs, ACPCs, and VCPCs for early warning and disaster response.  Implementation of safety nets (using cash for assets and food for assets) to increase household resilience and develop productive assets, such as feeder roads, dams, river bank reinforcement, or afforestation  Facilitation of inclusion and representation on VCPCs, ACPCs abd DCPCs in DRM structures  Capacity building of Civil Protection Committees on good governance and gender equity.  Provision of rain gauges and capacity building of communities to collect and analyze rainfall data.  Facilitating access of Civil Protection Committees to seasonal and short-term climate and weather forecasts  Development and consolidation of geo-reference maps for use by Civil Protection Committees  Implementation of large-scale Disaster Risk Reduction awareness campaigns  Provision of support for for district contingency plans  Mapping of risks, hazards, assets and resources  Promotion of validated early waring systems and technologies Types of Participants. Table 8 lists the different types of participants engaged and sites developed in the Njira Project under Purpose 3. The JMTR will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 8. Participants and Sites in the Njira Project Targeted Under Community Resilience (Purpose 3) Type of Participant/Site Male Female New Continuing from WALA LOA Total VCPCs 1035 225 1,260 ACPCs 7 4 11 DCPCs --- 2 2 Ubwino Centers 65 15 80 Department of Disaster Management Affairs (DoDMA) 1 1 Roads Improved or Constructed through FFA (KM) 100 100 FFA Participants 3,723 5,587 9,310 Persons Trained on Gender and Good Governance 6.372 9,557 15,929 Cross-cutting Themes. The Njira Project has three major cross-cutting themes, including gender equity, governance, and environmental management. Partner Roles and Responsibilities. The Njira Project has the following partners. Consortium Partners. The Njira Project has three consortium-level partners, PCI, Emmanuel International and Agricane. PCI is the lead agency responsible for overall program management, technical leadership, and financial oversight for the project. In addition, PCI also manages the commodity pipeline, program-level knowledge management systems, and implementation in Balaka District. Emmanuel International implements Njira in Machinga District. Agricane is responsible for developing and implementing conservation agriculture and irrigation interventions in both districts. JMTR Methodology 17 May 2017 41 Demonstration & Replication Partners. The Njira Project has six demonstration and replication partners who provide technical support for design and implementation of various interventions. These include the Department of Disaster and Management Affairs (DODMA)6 for disaster risk management interventions; CNFA for agri-business interventions; Total Land Care on fuel efficient stoves, climate smart agriculture, conservation agriculture, and agronomy services; Sun Mountain International for environmental and climate change-related interventions; and the Watershed Organization Trust for watershed management interventions. In addition, local NGOs are being engaged for mobilizing communities to support interventions on gender, environment and youth. Research Partners. In defining and implementing a research and learning agenda, the Njira Project planned to work with the University of Notre Dame, Colorado State University, the Lilongwe University of Agriculture and Natural Resources (LUANAR), and the University of Malawi Comparison of the UBALE and Njira Projects Table 8 provides a comparison of the two projects on key features Table 8. Comparison of UBALE and Njira Projects FEATURE UBALE (CRS) Njira (PCI) Goal Reduced chronic malnutrition and food insecurity, and increased resilience among vulnerable populations Improved food security among vulnerable households Purposes & Sub￾Purposes P1: Increased Income  SP1.1 Farm Management & Business Plannng  SP1.2 Agricultural Sales  SP1.3 Financial Services  SP1.4 Business Development  SP1.5. Joint HH Decision-Making P2: Improved Nutritional Status of PLW and Children Under Two  SP2.1 GoM MCHN Systems  SP2.2 Household-level Behavioral Change  SP2.3 Agency of Women of Reproductive Age P3: Empowered Communities  SP 3.1 Governance, Risk Reduction and Environment  SP 3.2 Community Productive Assets  SP 3.3 Women's Participation in Decision-Making Structures P1: Increased Income from Agricultural and Non-agricultural Activities  SP1.1 Agricultural production  SP1.2 Agricultural and Non-Agricultural Sales P2: Improved health and nutrition of pregnant and lactating women and children under five  SP2.1 Nutrition and Health Practices  SP2.2 Health and Nutrition Services  SP2.3 Hygiene, Sanitation and Water Facilities P3: Improve capacity to prepare for, manage and respond to shocks  SP 3.1 Community & Household Disaster Mitigation Assets  SP 3.2 Institutional Support for Disaster Risk Management  SP 3.3 Community Empowerment for Managing Disasters Geographic Target Areas Rural Blantyre 69 GVHs in 8 TAs Chikwawa 104 GVH in 11 TAs Nsanje 91 GVHs in 8 TAs Balaka 35 GVHs in 5 TAs Machinga 45 GVHs in 6 TAs Total Direct Beneficiaries 248,200 households 244,248 beneficiaries Budget $65,000,000 $30,000,000 Commodities 10,600 MT (CSB+ and vegetable oil) 4,866 MT (beans, CSB, vegetable oil) 6 In the original proposal, CADECOM was identified to fill this role but at project start-up, it was determined that they did not sufficient technical expertise. JMTR Methodology 17 May 2017 42 IV. EVALUATION TEAM The JMTR will be implemented by a team of development professionals, including two independent consultants and representatives from FFP, the USAID Bureau of Food Security, USAID Bureau of Democracy, Conflict and Humanitarian Assistance, CRS, and PCI. The members of the JMTR Team are are expected to participate in the full review process for both projects, or as much of the process as possible, and will have responsibility for leading investigations in assigned areas. These persons will develop data collection tools in their areas of responsibility, prepare presentations for Verification Workshops and draft the content for the JMTR Summary Report. For the three program components in each project (corresponding to Purposes), one of the JMTR Team members will be designated as the lead for that component and will have the final say on the formulation of recommendations in the assigned areas and on the content that will go into the JMTR Summary Report. In addition to the JMTR team, a number of other persons may join the exercise as Observers. These persons are technical specialists who will be available to provide ideas and input to the JMTR Team members in their areas of expertise, but they are not expected to participate directly in conducting interviews or focus group discussions since they are not participating in the whole exercise from planning through presentations. To clarify the functions of observers, a set of ground rules for observers is provided in Annex A. A. Team Members Following are the Core Team members of the JMTR team. Mike DeVries (JMTR Team Leader, Independent Consultant). A Design, Monitoring and Evaluation Specialist, Mike D will be responsible for coordinating implementation of the JMTR, including facilitation of meetings/workshops/debriefings, working with FFP, CRS and PCI to develop implementation schedules, facilitating the sharing of information between team members, providing support to the project implementers in developing recommendations, and completion of the final report. In addition, he will have the lead on Purpose 3 (DRM) and responsibilities associated with investigations on Management Systems, Partnerships, and Financial Management. Rupert Best (Independent Consultant). A Value Chain Specialist, Rupert will be responsible for reviewing the implementation of value chain and agri-business development activities and processes, including analyzing the opportunities and challenges for targeted beneficiaries to participate in value chains and how the projects are meaningfully linking with Feed the Future value chain promotion activities. Rupert will have the lead on Purpose 1 (Income) and will also coordinate investigations on integration and complementarity in both projects and on governance in the Njira Project. Marit Wilkerson (USAID Bureau of Food Security). Marit is a Climate Smart Agriculture Specialist working in the USAID Bureau of Food Security. She will be working with Rupert on the Pupose 1 (Income) investigations and will have specific responsibilities for reviewing the implementation of farm production activities for the two projects. She will also coordinate investigations on environmental compliance in the Njira Project with the support of Charlotte. Mike Manske (FFP HQ). Mike M is a Nutrition Adviser in FFP headquarters. He will have the lead on Purpose 2 (MCHN) and will be responsible for coordinating investigations on social and behavioral change activities in each project as well human resource management systems in both projects. Mary Pat Kieffer (PCI HQ). Mary Pat is the Senior Director for Health for PCI and is a specialist in strategic program design and implementation of health programs. She will be responsible for working with Mike M on investigations related to Purpose 2 (MCHN) and for specifically leading investigations on JMTR Methodology 17 May 2017 43 health systems capacity building activities in each project. She will also coordinate investigations on administrative support systems (logistics, procurement, communications and inventory management) in each project. Amos Chigwenembe (CRS Malawi). Amos is the Senior Program Manager for WASH for CRS Malawi. He will be responsible for working with Mike M on investigations related to WASH under Purpose 2 (MCHN) in the UBALE Project. Lucy Mungoni (USAID Malawi). Lucy is a WASH Specialist working with the USAID Malawi mission. She will be responsible for working with Mike M on investigations related to the WASH under Purpose 2 (MCHN) in the Njira Project. She will join the team at the end of the UBALE phase. Killian Mutiro (FFP Southern Africa Region). Killian is the Monitoring & Evaluation Adviser for FFP for South Africa Region. He will lead the investigations on collaborative learning and action in both projects and will also take the lead in coordinating investigations on targeting and the Theory of Change in the two projects, youth targeting as a cross-cutting theme in the UBALE Project, and commodity management systems in both projects. Mark Kumbukani Black (CARE Malawi). Kumbukani is the Monitoring & Evaluation Coordinator for CARE Malawi, a consortium partner in the UBALE Project. In addition to being an M&E specialist, Kumbukani is also a gender and women's empowerment specialist. He will be responsible for leading the women's empowerment and gender equity investigations in the UBALE Project, including investigations on the gender champions interventions, and for working with Amy on the women's empowerment and gender equity investigations in the Njira Project. Kumbukani will also be responsible for investigations related to the Ubwino Centers under Purpose 3 in the Njira project. Aaron Chassy (CRS HQ). Aaron is the Director of the Equity, Inclusion and Peacebuilding (EQUIP) Unit in CRS HQ. He will be responsible for developing data collection tools and leading investigations around the cross cutting theme of governance and will be with the JMTR team through the completion of the the Verification Workshop for UBALE. Aaron will also provide technical backstopping for Rupert who will lead the investigations on governance in the Njira Project. Amy Sunseri (PCI HQ). Amy is a Gender and Capacity Development Specialist working in PCI HQ. She will be working remotely with Kumbukani on the gender investigations in the UBALE project and will join the team in mid-March for the review of the Njira Project. Charlotte Germain-Aubrey (USAID DCHA). Charlotte is an Environment and Climate Change Advisor working in USAID Washington. She will be responsible for coordinating investigations on environmental compliance on the two projects. She will join the team from the beginning through the UBALE phase and will provide support remotely for Marit who will continue the investigations in the Njira Phase. Table XX summarizes the organization of the JMTR team into the two teams that conducted the UBALE and Njira Reviews, respectively. Table XX. JMTR Team Members Team Member Organization Responsibilities UBALE REVIEW TEAM Mike DeVries Independent JMTR Team Leader, Purpose 3 (Team Leader), Management, Partnership and Finance Rupert Best Independent Purpose 1 (Team Leader), Integration & Complementarity Marit Wilkerson USAID Bureau of Food Purpose 1 JMTR Methodology 17 May 2017 44 Security HQ Mike Manske USAID Food for Peace HQ Purpose 2 (Team Leader) and Human Resources Mary Pat Kieffer PCI HQ Purpose 2 and Administrative Support Amos Chigwenembe CRS Malawi P2 WASH Killiam Mutiro USAID Food for Peace Southern Africa Theory of Change, Targeting, Collaborative Learning & Action, Youth Targeting, and Commodities Mark Kumbukani Black CARE Malawi Purpose 3 and Gender Aaron Chassy CRS HQ Purpose 3 and Governance Charlotte Germain￾Aubrey USAID Bureau of Democracy, Conflict & Humanitarian Assistance HQ Environment NJIRA REVIEW TEAM Mike DeVries Independent JMTR Team Leader, Purpose 3 (Team Leader), Management, Partnership and Finance Rupert Best Independent Purpose 1 (Team Leader), Integration & Complementarity, and Governance Marit Wilkerson USAID Bureau of Food Security HQ Purpose 1 and Environment Mike Manske USAID Food for Peace HQ Purpose 2 (Team Leader) and Human Resources Mary Pat Kieffer PCI HQ Purpose 2 and Administrative Support Lucy Mungoni USAID Malawi P2 WASH Killiam Mutiro USAID Food for Peace Southern Africa Theory of Change, Targeting, Collaborative Learning & Action, and Commodities Mark Kumbukani Black CARE Malawi Purpose 3 and Gender Amy Sunseri PCI HQ Purpose 3 and Gender During the initial planning period for each review before data collection begins, the team leader for each purpose (Rupert, Mike M and Mike D) will ensure that team members conducting investigations are clear on the outputs which have been assigned to them for investigation within the purpose. V. INFORMATION TO BE OBTAINED The following sections outline the information that will be obtained to achieve each of the eight objectives of the JMTR while meeting the purpose of the JMTR to generate recommendations for the remainder of the lives of the UBALE and Njira Projects. A. Objective 1: Relevance of Project Strategies To accomplish Objective 1, the JMTR will review the theories of change for the two projects. These have been developed around a vision and pathways of change for particular types of targeted impact groups, so the JMTR will review the targeting systems for the two projects to identify strengths and weaknesses in how the projects identify participants compatible with the Theories of Change that have been developed. The JMTR team will also examine the history of the projects, particularly how they have evolved since inception and the critical features of the operating environment that have affected, positively or negatively, project implementation and the impact that has been achieved. The JMTR will review the 2015/2016 National Food Insecurity Response Plan along with other policies and priorities of the Government of Malawi, and recent analysis of the Malawi Vulnerability Assessment Committee (MVAC) to determine what changes may be needed to project activities and implementation mechanisms to ensure that the strategies remain valid for the local context. The analysis will inform the formulation of JMTR Methodology 17 May 2017 45 recommendations associated with the overall strategy of the program toward achieving the goals of the two projects. Information for understanding the history of the program and contextual factors affecting implementation will be obtained from project monitoring reports, as well as through interviews with project implementation staff and partner leadership representatives. B. Objectives 2 (Outputs) & 3 (Outcomes): Project Inputs, Approaches, Outputs and Outcomes The JMTR will examine the outputs produced under each Sub-Purpose to identify what the projects have accomplished on the ground. The inputs and processes used to produce outputs as well as the quality of the outputs in terms of achieving change will be investigated. Both qualitative and available quantitative information will be used to assess the effects and impact of the outputs produced by the project at different levels, including effects at the immediate outcome level , effects at the Sub-Purpose Level for clusters of outcomes, effects at the Purpose level for the set of Sub-Purposes and impact collectively at the project Goal level. JMTR team members will look for evidence on how members of target groups have changed their ideas, attitudes, and practices as a result of project activities and will identify potential reasons for why some beneficiaries have started applying project promoted practices while others have not. At every level, the evaluation will assess the sustainability and replicability of the changes that have been observed. The review team will consider staffing and activity resources, community participation, participant targeting, asset transfers, the extent to which activities and outputs demonstrate a commitment to the cross-cutting issues of gender, governance and the environment, and sustainability strategies. The review will investigate the expected change as per the project documents, but will also be observant for new areas of unexpected effects and impact, both positive and negative, that are occurring as a result of project activities. Recommendations may be formulated to include these in the projects' monitoring and evaluation tools. Information on processes, outputs and outcomes will be obtained through reviews of project monitoring reports, including quarterly and annual reports, reviews of key secondary data, interviews/focus group discussions with project participants, and interviews with project implementation staff. JMTR team members will analyze performance monitoring data collected by the projects, as well as review the performance management plans and IPTTs. JMTR team members will review technical guidance, including implementation manuals, on key processes and approaches used by the projects and observe learning/training sessions that are planned by the projects when the JMTR team is in the area. C. Objective 4: Coordination and Collaboration The JMTR will obtain information on external actors, i.e., other projects and service providers that are relevant to the strategies of the UBALE and Njira Projects. These include external actors who provide complementary services necessary to achieve and sustain project outcomes, external actors that influence people’s access to goods and services, and external actors that promote or impede an “enabling environment”. The JMTR team will work with project implementation staff to identify the most important of these external actors for each component of the project and will then analyze the quality of the coordination and collaboration with the relevant project. Particular attention will be given to interaction with other USG-funded activities as well as coordination with GoM services. Information on the quality of coordination and collaboration with key external actors will be obtained from project monitoring reports, interviews with project implementation staff, and interviews with representatives of these key external actors. D. Objective 5: Collaborative Learning and Action JMTR Methodology 17 May 2017 46 Collaborative Learning and Action refers to (a) how projects bring in and test new ideas and approaches, (b) how monitoring and evaluation systems provide information for decision-making and reporting, and (c) how best practices and lessons learned emerging from project implementation are documented and shared outside the project. The JMTR will obtain information to understand the systems being used for capturing, documenting and disseminating lessons learned and best practices. The M&E systems for both projects will be examined to understand how information generated from the systems is being used to enhance the effectiveness or efficiency of the project in achieving impact. Emphasis will be placed on the collection, analysis and management of data to enable iterative learning and evidence￾based improvements to project design and implementation. The JMTR will identify changes that have already occurred in the project strategy or implementation as a result of new ideas or approaches brought into the project from outside sources and will formulate recommendations for how to seek out appropriate new ideas and approaches in the remaining life of the projects. The research agendas for the project will be reviewed for relevance and timeliness in providing information for decision-makers. Information for collaborative learning and action investigations will be obtained from a review the projects' PMPs, monitoring reports and other reports, interviews with project management and representatives of project steering and advisory committees, interviews with project implementation and technical staff, interviews with the leadership of partner organizations, and interviews with knowledge management staff. E. Objective 6: Sustaining Project Outcomes and Impact The JMTR will review the sustainability strategies, interview staff and management, interview project participants, and interview private and public stakeholders to determine how likely project-generated outcomes and impact are to be sustained after the projects end. The review will identify the organizations, services and relationships that need to be maintained after the project ends in order to sustain each of the outcomes planned by the projects and will analyze the threats to these that could affect likely sustainability. Specific attention will be paid to the appropriateness, efficiency and efficacy of capacity building activities targeted to participants and local partners to enable them to sustain project impact. F. Objective 7: Cross-Cutting Themes (Gender, Governance, Targeting Youth and Environment) For each of the four major cross-cutting themes in each project, including gender, governance, targeting of youth, and environment, the JMTR team will assess how well the projects are operationalizing the cross-cutting theme and achieving the proposed impact. Information on cross-cutting themes will be obtained through interviews with project management, project implementation staff, leadership of partners and stakeholders, and project participants. Project monitoring reports will be reviewed, and specific strategy documents related to each cross-cutting theme, including assessment reports and barrier analyses, will be reviewed. G. Objective 8: Implementation Systems Project implementation systems include management systems, partner relations, knowledge management systems (covered under Objective 5), financial management systems, commodity management systems, human resource management systems and materials and equipment procurement management systems. The JMTR will review these systems to identify recommendations for improving the effectiveness of implementation and/or the efficiency in use of project resources. Information on project implementation systems will be obtained from directly reviewing the systems being used, project monitoring reports including quarterly reports, annual results reports, financial reports, and commodity management reports. Interviews will be held with project management staff, JMTR Methodology 17 May 2017 47 implementation staff, and leadership of consortium and technical partner organizations, and relevant technical staff from each of the partner organizations. VI. EVALUATION PROCESS The JMTR will be undertaken over a period of approximately five months from mid-January through mid-June of 2017 and will be implemented in four phases:  Phase 1: Evaluation Preparation (Mid-January through March 5)  Phase 2 (in Malawi): Review of the UBALE Project (March 6 - 23)  Phase 3 (in Malawi): Review of the Njira Project (March 24 - April 7)  Phase 4: Evaluation Recommendations Finalization and Processing (April 10 - June 21) The current schedule for the JMTR is provided in Annex B. The sections which follow describe the major activities planned in each phase. A. JMTR Preparation (January 19 - March 6) During the JMTR preparation period, the composition of the review team will be finalized (as indicated in section V.A. above), a draft protocol including a basic operational plan (this document) will be drafted, the JMTR team will review the background documents (listed below), the data collection tools will be drafted, and the sites for field visits will be selected. The JMTR Team will assemble in Malawi by March 5, if possible. On Monday, March 6, the team will hold an initial meeting to meet each other and to discuss the JMTR plan. Also on that day, an orientation meeting will be held with USAID/FFP Malawi7 . The following sections describe these JMTR outputs and processes in more detail. 1. JMTR Protocol. This document is the protocol that guides implementation of the JMTR. The Zero Draft of the protocol will be released for review by organizers of the JMTR and members of JMTR team 31 January. Comments, questions or suggestions on the zero draft should be sent to Mike DeVries (mdevries02@yahoo.com). On or about February 17, a conference call will be organized to discuss the protocol toward finalizing it into a working draft that will be widely circulated by February 20, as the final planning guide going into the implementation phase of the JMTR. Further changes will be made once the JMTR arrives in-country and has a chance to discuss the projects with project implementers. 2. Background Document Review. There are a many documents that could be reviewed to prepare for the JMTR. However, given time limitations for team members, the available documents have been divided into two sets. One set represents required reading for all team members in order to be able to understand the full strategy for each project as well as the range of activities being undertaken. The second set of documents should be reviewed by individual team members when possible. Required Reading. The following documents are required reading for all team members and must have been reviewed before the JMTR Team assembles in-country on March 6.  Approved Project Technical Narratives  Theories of Change and Logical Frameworks (from the Performance Monitoring Plans)  Indicator Performance Tracking Tables (from the Performance Monitoring Plans unless there has been a subsequent revision)  Annual Results Reports for 2015 and 2016 7 Due to other commitments, no one from the USAID Mission was able to meet with any of the JMTR Team members from the time the team leader arrived on March 1 through March 5. This initial meeting, however, should still be planned in future reviews. JMTR Methodology 17 May 2017 48  All Quarterly Monitoring Reports Other Reference Documents. The following documents are also available and should be reviewed by team members when possible.  Baseline Study Report  Full Performance Monitoring Plans  Gender Analysis Reports  Barrier Analysis Reports  Site Activity Records and Training Summaries  Project Implementation Manuals (or training curricula and training materials in lieu of a PIM)  Participant Registration Data  Environmental Mitigation and Monitoring Plan  Data Quality Assessments  The WALA Final Evaluation report Obtaining Documents. A directory has been established on the USG Google Platform for access by USG members of the JMTR8 . The key point of contact for any questions about this directory and documents for the two projects is Shahina Mailk, the Agreement Officer's Representative (AOR, shmalik@usaid.gov). For others who do not have access to the USG platform, the documents are available on the DropBox (Malawi JMTR 2017). The key point of contact about this directory is Mike DeVries (mdevries02@yahoo.com). 3. Data Collection Tools. Section VII below provides guidance on the data collection tools, primarily topical outlines, that will be needed for the JMTR. JMTR Team Members with support from Observers will be responsible for developing the initial draft of the data collection tools. The deadline for completing these is February 28 so that team members will have time to review background documents and this protocol. These drafts will be sent to the JMTR team leader who will review them, provide feedback and then incorporate working versions of the tools in this protocol in Annex B. 4. Site Selection. Section VII provides guidance on how sites will be selected for field visits for the JMTR. During this preparation phase, CRS and PCI should assemble the lists of sites that have been requested in Section VII. The JMTR Team Leader will arrive in-country by March 1 and will meet with UBALE and Njira management teams shortly after arriving to finalize the preliminary selection of sites by March 3. 5. Orientation Meetings on March 6. Two meetings are planned for March as described below. Initial JMTR Team Meeting, March 6. The initial meeting of the JMTR Team will occur immediately after core team members have assembled in Malawi. In this meeting, team members will be introduced to each other, and the JMTR team leader will provide an overview of the evaluation process and answer any questions that team members may have. Orientation Meeting with USAID/FFP Malawi, March 69 . Following the initial team meeting, an orientation meeting will be held with representatives of USAID/FFP Malawi at which the JMTR Team will be introduced, an overview of the review process will be provided by the team leader, and the JMTR 8 At the end of the day, only the Drop Box was used for documentation since not all team members had access to the USG Google platform and all USG members on the JMTR Team had secondary, non-USG email addresses which allowed them access to the Drop Box. 9 This meeting did not happen since key USAID Mission staff had other commitments that precluded them from having a meeting with the JMTR team. Future reviews should still schedule this meeting, however. JMTR Methodology 17 May 2017 49 team will answer any questions that USAID/FFP may have about the review. In addition, the JMTR team will seek to understand the expectations of USAID/FFP and any specific interests that they would like to see covered by the review. B. Phases 2 & 3: Reviews of UBALE and Njira in Malawi On Tuesday, March 7, the JMTR Team will travel to Blantyre. From March 8 through March 23, the team will review the UBALE Project and from March 24 through April 5, the team will review the Njira Project. A Stakeholder Debriefing will be held on April 6 before the team travels back to Lilongwe, and a debriefing will be held in Lilongwe with USAID/FFP before the team departs the country on April 7 or shortly thereafter10. The reviews of the two projects will follow the same process as described in the next section. Descriptions of the two debriefings to be held before the team departs the country are described at the end of this section. 1. Project Review Process for UBALE and Njira. Over the 16 days with UBALE and the 13 days with Njira, the JMTR Team will undertake the following process. Project Orientation Meetings, March 8 (UBALE) & March 24 (Njira). In these half-day meetings, project management staff will present an orientation to the project for the JMTR Team. The key content of these presentations should include an overview of the strategy of the project, an overview of the resources (money, commodities and staff) available for the project, a map showing geographic locations, a description of the outputs and activities under each component, a description of the roles and responsibilities of partners, and a description of the key challenges affecting the project. The purpose of these meetings is to obtain clarity on the types of outputs produced by the project, the stakeholders that need to be interviewed to understand the impact of these outputs, and additional data sources for information to support the review. Discussions will also be held around the site selection for the qualitative interviews to determine what the sites represent in terms of outputs and quality. Attendees to this meeting include the project management and technical team, selected representatives from implementing partners, and anyone else from the project likely to be involved in implementing or supporting the JMTR. Meetings with Project Stakeholders in Blantyre, March 9 (UBALE) & March 25 (Njira). On this day, JMTR Team members will conduct interviews with key stakeholders that are based in Blantyre for UBALE and in Blantyre or Zomba for Njira. These could include interviews with representatives of implementing partners, technical partners, government stakeholders, and private sector stakeholders. Given the limited time, the JMTR will need to be strategic on the use of this time and will consult with the project management teams for each project to determine with whom meetings should be arranged and for which members of the JMTR Team. Field Data Collection, March 10-18 (UBALE) & March 26-April 1 (Njira). Over this period, the JMTR team will conduct key informant interviews, hold focus group discussions and observe project activities to obtain qualitative data. The JMTR will plan to spend three days in each of the five districts covered by the two projects. The specific data collection process in each district is described in section VII.C below. A detailed itinerary for the field work will be developed in collaboration with project management staff as described above. 10 Because key participants for the USAID Debriefing were scheduled to travel in the afternoon of April 7, the USAID Debriefing had to be scheduled for early in the day on April 7. In order to be available for this meeting, the JMTR team traveled to Lilongwe on April 5 and held both of the project Stakeholder Debriefings on April 6, with UBALE in the morning and Njira in the afternoon. JMTR Methodology 17 May 2017 50 Information Processing & Preparation for the Verification Workshop, March 19-21 (UBALE) & April 2-3 (Njira). During the field work, JMTR team members will process data as it is obtained, and in this period the team will continue analyzing information to identify major findings and recommendations that will be presented for discussion at the Verification Workshop. Each JMTR team member will use this time to prepare the Powerpoint presentation(s) that she/he plans to present in the Verification Workshop. Verification Workshops, March 21-22 (UBALE) & April 3-4 (Njira). During and following the field work, the evaluation team will have started formulating preliminary observations on outputs produced, the impact being achieved, lessons learned and recommendations for the remaining life. This is all content that will be included in the JMTR Summary Report. Given the time available for this workshop, however, only the major observations and the recommendations resulting from the JMTR investigations will be shared in the Verification Workshop with project implementation staff (2 days for UBALE and 1.5 days for Njira). These will be discussed to ensure that they reflect reality and are described appropriately. Two major outputs are targeted for the workshop. These are (1) agreement on the validity of key observations assembled so far from the review and (2) preliminary agreement on major recommendations for the remaining life of each project. Participants in each workshop will be project implementation staff and the JMTR team. This is an in-house event, only for those participants who are fully engaged in the project. An illustrative Workshop Plan is provided in Annex D. Post-Workshop Processing, March 23 (UBALE) & April 5 (Njira). This one day period is required to ensure that results of the discussions in the Verification Workshop are captured before the JMTR team moves on from UBALE to Njira in the first case and from Njira to stakeholder debriefings in the second case. JMTR Team members who have made presentations in the Verification Workshop will revise their presentations to incorporate changes as a result of discussions. Each JMTR Team memberwill also prepare a condensed version of her/his presentation that can be consolidated with other team member presentations for the stakeholder and USAID/FFP debriefings. 2. Pre-Exit Debriefings. Two debriefings will be conducted by the JMTR team before they leave Malawi, as described below. Stakeholder Debriefing, April 6. A draft summary of the JMTR product following the Verification Workshops, i.e., major observations and recommendations, will be presented to the leadership of each project's implementing partners and representatives from other project stakeholders, such as government agencies or technical partners, for final discussions. This is an opportunity for implementation stakeholders to provide input into the review before the final product is produced. Debriefings (at a set location to which representatives will be invited) will be held with each project on this day, with UBALE stakeholders in the morning and Njira stakeholders in the afternoon. Participants are expected to be project CoPs, the leadership from consortium and technical partners, and representatives from other agencies having active roles in each project. USAID/FFP Debriefing, April 7. Following the Stakeholder Debriefings, final changes will be made to the JMTR product, and a summary of the final product will be presented to USAID/FFP Malawi. Participants in this meeting will be the leadership of CRS and PCI, the CoPs for each project and representatives from USAID. The JMTR team leader will facilitate the presentation, and all JMTR core team members, if they are still in-country, will attend. C. Phase 4: Recommendation Finalization and Processing After departing the country, the JMTR will work remotely to assemble a draft report by May 5. This report which fully documents recommendations and supporting observations will be used as the key document for a JMTR Recommendation Processing and Planning Workshop. Following this workshop, JMTR Methodology 17 May 2017 51 the Review Report will be finalized, a debriefing provided to FFP HQ-based stakeholders, and action plans will be developed by each project that will be incorporated in the PREP for IY 4 for each project. 1. Draft Report, May 5. JMTR team members will continue reviewing and analyzing background documentation and information collected while in Malawi toward assembling content for the draft report. The JMTR team leader will consolidate the writing into a draft report to be finalized by May 5. 2. JMTR Recommendation Processing and Planning Workshop, May 22-27. Two separate workshops will be held with Njira (May 22-24) and UBALE (May 25-27) at which the recommendations from the JMTR will be reviewed and finalized. Since the project management staff will have seen most, if not all, of the recommendations in the Verification Workshop, they will have already begun strategizing on how to respond to the recommendations. The JMTR team leader and other members of the JMTR team who can be available for the workshop will be present to facilitate the processing and planning. By the end of the workshop, each project will have developed provisional implementation plans to be incorporated in the coming PREP that is due by the end of June 30. 3. Final Report, June 1. Following the processing and planning workshop, final changes will be made by the JMTR team leader to the draft report to produce a final report on the JMTR by May 22. 4. Debriefing at FFP HQ, TBD. The JMTR final recommendations and preliminary plans for operationalizing recommendations developed in the workshop will be presented to HQ-based stakeholders on a date after the JMTR Summary Report has been completed. 5. Finalization of Recommendation Implementation Plans, May 29-June 21. Following the processing and planning workshop, project management teams for each project will complete the drafting of operational plans by June 6 for review by FFP. By June 14, a meeting will be held in FFP HQ with Malawi staff joining remotely to to discuss and finalize the plans for each project. The final plans incorporating feedback from FFP will be drafted by June 21 and included in the PREP submission to be made by June 30. VII. QUALITATIVE DATA COLLECTION Qualitative data will be collected primarily through key informant interviews and focus group discussions with representatives of project implementation staff, project participants, and project partners including consortium, technical and GoM partners. Topical outlines or checklists will be used to guide interviews and focus group discussions. Annex C (to be completed by JMTR team members by March 5) contains the working draft tools that will be used to start the field work. Ideally, the JMTR will gather information from all of the partners and participants listed in Section III above describing the two projects. JMTR Team members will provide project management staff with a list of the meetings, interviews and FGDs that they would like to have arranged, and the JMTR team will work with project management staff to develop schedules for meetings. A. Composition of Key Informant Interviews and Focus Group Discussions Key informant interviews are normally held with between 1 to 4 persons, ideally no more than two, and these interviews typically last no more than one hour. Focus Group Discussions (FGDs) are organized with from five to no more than ten persons. They are called focus groups because all of the members of the group have a common feature, so FGD facilitators must ensure that participants meet the desired common criterion, i.e., members of a VSL or SILC group, farmers who received seed from the project, lead mothers, lead fathers, etc. FGDs should normally not extend beyond two hours. In both cases, interviews and discussions should be held in secluded locations so that bystanders or passersby cannot influence the discussions. JMTR Methodology 17 May 2017 52 B. Topical Outlines or Checklists JMTR Team members will develop topical outlines or checklists which include key thematic questions to be used in key informant interviews or focus group discussions. These tools will be drafted by the individual Core Team Member assigned to specific investigations and tailored to the types of participants and approaches being used by each project. The questions in a topical outline are fairly general and used to stimulate discussion. Reviewers will keep in mind at all times that the purpose of the information gathering is to understand what the project has done, what changes have occurred as a result, what has helped or hindered achievement of these changes, and how likely are the changes to be sustained after the project ends. The discussion facilitator will be free to explore in more depth any interesting topics that may come up during each discussion, related to these objectives. A suggested sequencing of questions in a topical outline, drawn from a Program Constraints Assessment (PCA) approach, would be as follows. 1. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar.) 2. How would you describe what this project seeks to accomplish? 3. How have you or other members of your household participated in this project? For how long have you/they been involved? 4. Please describe how you or your family have benefitted from the project. 5. Please describe how you or your family have been negatively affected by the project. 6. Who, in your opinion, has benefitted most from the project? 7. Are there other people who should be benefitting from the project but are not? Please describe them for us. 8. What constraints do you believe inhibit the project from fully accomplishing its purposes? 9. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? For each question, the interviewer/facilitator should have an idea on what kind of response to expect, based on a review of the background documents, but should avoid leading the respondent to make these responses. After a respondent has completed answering a question and an expected topic has not come up, the interviewer/facilitator can then ask…what about this?…. noting that the respondent did not spontaneously report on the topic. Before beginning an interview or discussion, an introduction and explanation of the purpose of the evaluation will be provided with stress put on the importance of obtaining useful information that reflects reality. The evaluation team will seek consent from the participants, and no names will be recorded in interviews and group discussions. If portable recording devised are used, the device should be shown to respondents and not activated until after respondents have introduced themselves. C. Observation of Project Activities In addition to conducting interviews and focus group discussions, JMTR team members will also observe implementation of project activities and physical sites where project investments have been made. These should be regularly scheduled project activities, not activities organized only for the JMTR. Section III, describing each of the projects, lists the different types of sites that the members of the JMTR team should visit. D. District-Level Data Collection JMTR Methodology 17 May 2017 53 A three day process is envisioned for data collection in each of the five districts in which the UBALE and Njira Projects are being implemented, as described below11 . 1. Day 1 (Half Day) Project Orientation and Interviews with Project Staff. A different project partner is implementing activities in each district. In order for the JMTR to begin to understand similarities and differences in implementation between different partners, the first activity in each district will be to work with project managers and staff to understand implementation in the district. Once the team arrives in the district office, the following activities will be undertaken: 1. General meeting between project staff and JMTR team with introductions of all participants and a quick overview of the JMTR process by the JMTR team leader. 2. Review of the schedule developed by the district team for interviews with district-level stakeholders on Day 2 (see Step 3 below) to be sure that JMTR team members know who they are meeting and who from project staff will be accompanying them to make introductions to stakeholders. In addition, any changes that the JMTR team members would like to make in the schedule are discussed. 3. Interviews and focus group discussions with project implementing staff by sector, i.e., JMTR team members will meet separately with project staff implementing activities in their assigned purpose, cross-cutting theme or implementation system. 4. Plenary meeting for one hour at which the project manager in the district can present (1) the unique features of the project in that district and (2) major challenges that have affected implementation. A handout of statistical facts about the project should be presented to the JMTR team members. 2. Day 1 (Half Day) First Field Visits to Communities and Sites. In this half day, the JMTR Team will conduct interviews and focus group discussions with representatives of project participants, including both targeted beneficiaries and intermediaries in sites that have been selected for the review (see Section F below). In addition to conducting interviews and focus group discussions, JMTR team members will also observe, where possible, implementation of project activities and physical sites where project investments have been made. The reason for starting the field data collection before meeting with project stakeholders on Day 2 is that it will give JMTR team members the opportunity to see at least some project field activities before interviewing stakeholders. 3. Day 2 (Half Day) Interviews with District-Based Stakeholders. JMTR team members will conduct interviews with representatives of project stakeholders based at the district-level. Team members investigating the MCHN components of the project, for example, will meet with district-level MoH staff, MoH facilities and other relevant technical partners. Similarly, other JMTR team members will meet with the relevant stakeholders n there areas of responsibility. At least five days before these meetings, the JMTR team members will have sent project management a list of the stakeholders with whom they would like to meet. Project staff should also make suggestions to the JMTR team members on other key stakeholders not listed. The project management in the district will then work on developing a schedule for these meetings, which will be discussed and finalized in Step 1 above. 2. Days 2 (Half Day) & 3 (Full Day) Continuing Field Visits to Communities and Sites. On the second and third days in each district, the JMTR team will continue to conduct interviews and focus group discussions with representatives of project participants, including both targeted beneficiaries and intermediaries in sites selected for the review. Given the size of the review team, including translators, 11 Over the course of implementing the JMTR, the district-level process was reviewed and revised before going to the next district. The process described is that used in the last district, Balaka, and reflects the best process used in the Malawi JMTR. JMTR Methodology 17 May 2017 54 the reviewers will be organized into two teams who will travel to separate sites for data collection. Each of the two teams will target visiting a site for a half day conducting between 2 and three interviews or FGDs in each location, so the JMTR team will conduct interviews and discussions in an estimated eight communities in each district12. Site selection, following the parameters described in Section D below, and development of a field work schedule will be done in collaboration with project management staff for, at least, the UBALE field visits by March 6. In addition to conducting interviews and focus group discussions, JMTR team members will also observe, where possible, implementation of project activities and physical sites where project investments have been made. Section III, describing each of the projects, lists the different sites and activities that should be targeted for site visits. 3. Team Analysis and Sharing of Information Between JMTR Members. In order to be able to identify synergies and interaction between different interventions in each project and to share information that is relevant for investigations being conducted by other JMTR team, JMTR team members will meet together before leaving a district at the end of the second day of data collection in the district. The purpose of this meeting is to share insights and observations13 . 4. Translation. It is expected that some meetings with local staff and project partners will be carried out in English. Meetings especially at the community-level, however, will require translation assistance. This assistance in Chichewa will be arranged by CRS and PCI and logistics plans for travel and lodging should include interpreters. One interpreter will be recruited for each member of the team who does not speak Chichewa to ensure that there are enough interpreters. Before commencing field work, the JMTR team leader will meet with interpreters to explain their function and answer any questions they may have. Unlike a quantitative survey in which strict interpretation of questions and answers is required, the qualitative methodology used in the Malawi JMTR will require interpreters to be sure that reviewers are understanding clearly what is being said by respondents. 5. Documentation. Team member notes will be taken by individual team members during the data collection process. Wherever possible, same sex interviewer and participant will be utilized. Team members will also use photography as a data collection tool. E. Other Data Collection Because of time constraints and the locations of key informants, especially for some technical partners for the projects who do not have a presence in the project areas, some interviews will need to be organized remotely either by telephone, by skype or by email using a condensed version of the topical outline for the topic to be covered14 . F. Sampling and Participant Selection 12 The initial protocol for the JMTR had suggested that the JMTR team would visit a "common" village, in other words, there would be one site in each district which all JMTR team members will have visited to consduct interviews and FGDs. The purpose behind this was to be able to capture more accurately overlaps and synergies between project activities. Ultimately, given the size of he team, this "common" village approach proved logistically infeasible. 13 A second mechanism for facilitating the sharing of observations between team members using manila envelopes was tested during the UBALE review and proved to be ineffective, basically because team members barely had enough time to compile their own notes, with little or no time to compile notes on observations relevant for other team members. 14 In the Malawi JMTR, there was insufficient time to conduct any interviews by phone or by skype. Some data was collected, however, by email. JMTR Methodology 17 May 2017 55 Project Interest in Selecting Sites Organizers of mid-term evaluations are often reluctant to rely too much on projects to select the sites for field visits in an evaluation. The perception is that project implementers will want the reviewers to produce the best view possible about the project. This, however, is not in the best interests of development, the project or organizations implementing projects. Just as all projects are achieving at least some good impact, all projects also have implementation challenges and problems. Mid-term evaluations are opportunities to fix these problems and address the challenges. Moreover, if they do not get addressed by the time of the final evaluation, final evaluators will find them and criticize the project for not addressing them. The project that has used the Mid-term Evaluation to really improve the effectiveness and efficiency, will ultimately have greater impact and be viewed more favorably in the the final assessment. Since this is a Mid-Term Review, the sample of participants and sites for field visits will not be chosen randomly. These will be selected strategically, so that the review team can observe what is working and what is not working in each project, as well as any particularly innovative approaches. The recommendations to be generated by the JMTR will propose scaling up interventions that have good impact, modifying interventions to have more impact, suspending interventions that do not have enough impact relative to investment, piloting new interventions relevant for targeted impact groups, improving the effectiveness of implementation systems, or improving efficiency in use of resources. Reviewers need to see the problems in order to be able to propose recommendations to fix them. The JMTR will collect data from each of the districts being covered by each project. For each district, UBALE and Njira will classify GVHs according to the following criteria. 1. Participated in WALA or new in UBALE/Njira 2. Accessibility in terms of proximity to major infrastructure (markets, main roads, basic services) 3. Demonstrating good impact (e.g., early adopters) or demonstrating poor impact - This criterion is difficult to assess at this stage in the project life. However, front-line staff usually have a pretty good idea on villages which have been adopting the project-promoted messages more readily than other villages. The objective of the JMTR is to see examples of both. 4. Intensity of intervention, i.e., activities under only one purpose being implemented, activities for two purposes, or activities for all three purposes Working with project management staff, the JMTR team leader will identify the initial set of GVHs to be visited, seeking to ensure a representative balance on types of sites and wide representation on interventions, taking into account logistical considerations. After the selection of eight GVHs has been made for the district, project management will then list the types of project activities/sites in each GVH. That information will then be shared with the JMTR team members who will indicate which interviews/FGDs she/he would like to have organized in the GVH. For interviews and FGDs in each site to be visited, project staff, volunteers and village leaders will identify individuals for interviews and discussions to ensure that households selected represent the economic and geographic diversity of a village. G. Data Analysis The information gathered by the JMTR team will be analyzed at multiple points during the review process. As the reviewers are holding interviews and focus group discussions, they will probe and explore topics in more depth with respondents to ensure clear understanding. This represents the first level of analysis. A second level of analysis occurs when reviewers cross-reference responses from interviews and discussions with existing data bases. In addition, the evaluator will query project implementation staff as a third source of information. The result of this level of analysis is that each evaluator will formulate his/her own key observations relative to the assigned topics. JMTR Methodology 17 May 2017 56 A third level of analyses occurs at the JMTR team meeting scheduled for the second day in each district, through the sharing of notes on observations and immediately before the Verification Workshop when the evaluation team is processing information individually and sharing information with other team members. The advantage of qualitative methods is to have the ability to conduct real time analysis of the information. As individual team members encounter information from other team members that they may not have had a chance to discuss with respondents, they will have the opportunity to do so when they go again to conduct interviews/discussions. A fourth level of analysis occurs in the Verification Workshop in which preliminary observations and findings are presented to project implementation staff. If project implementation staff seriously question a particular finding that has been presented by the evaluation team, the opportunity exists in the workshop to further discuss and analyze the finding to reach the truth. Finally, a fifth level of analysis occurs after the JMTR team has dispersed and begun assembling the content that will go into the draft report. Team members will have the opportunity to look at the information that has been gathered in more depth as they are writing. VIII. IMPLEMENTATION SCHEDULES The complete schedule for implementing the JMTR is provided in Annex A. The schedule for site visits with UBALE and Njira team will also be included in Annex A, but will be added to the protocol in early March after the JMTR Team Leader has worked with project management teams to develop preliminary plans that can be reviewed by core team members. IX. CRS AND PCI RESPONSIBILITIES CRS and PCI will ensure that the necessary information has been passed to the GoM to allow the team to implement the review in the communities that will be selected. UBALE and Njira project staff (including CRS, PCI and partner representatives) will serve as informants to the JMTR and support the review process by supplying lists of project sites, sharing project documents, advising about local protocols, making orientation presentations to the JMTR, arranging meetings with the stakeholders, and making logistical arrangements. In addition, CRS and PCI will ensure that their project partners are informed of the process and that implementing partners will need to make staff available as possible for interviews and other consultations with the the JMTR team. CRS and PCI will also provide professional interpreters (as required) for team members and facilitate in-country travel and logistical arrangements for the JMTR team members including vehicles, printing/copying and access to data bases. Members from USAID will pay for their own accommodation and food. They will need assistance, however, with hotel bookings. JMTR Working Group. CRS and PCI have formed a JMTR Working Group in Malawi to facilitate coordination of logistical arrangements for the JMTR . The key point of contact for the JMTR Working Group is Dane Fredenburg (dane.fredenburg@crs.org). Questions about in-country logistical arrangements can be sent to Dane, and JMTR team members should also ensure that he receives communications on arrival and departure dates. Given the size of the JMTR team with interpreters and the complexity of the schedule, iIf at all possible, for each review, the project should provide a full-time logistical support person to accompany the team to the field to provide logistical support15 . 15 This was done for the Njira Project, but not for the UBALE project and the effectiveness of the logistical support for the JMTR was much better in the former. JMTR Methodology 17 May 2017 57 X. JMTR REPORT16 The JMTR Repor t will be submitted in two Volumes. Volume 1 called the JMTR Summary will not exceed 50 pages excluding annexes, and will contain the following:  An executive summary providing an overview of the JMTR  A short section providing a background summary for the JMTR including the purpose, the team composition, and an overview of the process.  A short section describing the history of the projects and the contextual factors affecting the projects.  Two separate sections, of approximately twenty pages each representing the review of UBALE and the review of Njira, respectively. Each review should have an overview of the project strategy, a summary of project progress, and prioritized recommendations organized by theme. The recommendations should be specific, concise, and feasible given the time and resources available to the projects.  A short section summarizing opportunities for shared learning between the two projects.  A short summary of relevant higher level observations that go beyond the scope of either project but need to be brought to the attention of project stakeholders, including USAID.  The annexes to Volume I should include one annex for each project providing more detail on observations from the JMTR and additional recommendations that did not make the priority list of recommendations that are included in the main text. Volume II of the JMTR Report is called the JMTR Methodology and contains the following:  The Initial Scope Of Work for the Malawi JMTR that was developed to initiate the planning for the exercise  The final JMTR Protocol  The data collection tools used  The final schedules for the JMTR  List all persons interviewed and sites visited  Final Workshop Plans for the Verification Workshops  And a summary of information obtained from a review of the JMTR Process, including review limitations An illustrative format for the two volumes of the the report are provided in Annex E ANNEXES A. Ground Rules for JMTR Observers B. JMTR Schedules (Overall and Field Work) C. Data Collection Tools D. Illustrative Verification Workshop Plan E. Illustrative JMTR Summary Report Format 16 This section describes the actual formats used for the JMTR Summary report and reflects fairly extensive discussions on the format with the projects and FFP on the preferred format. JMTR Methodology 17 May 2017 58 ANNEX A MALAWI JMTR GROUND RULES FOR OBSERVERS Following are some ground rules that we would like Observers for the Malawi JMTR to follow as they provide support to the team implementing the JMTR.  Because the JMTR Team is so large (12 persons) and the logistics will be challenging over a short period, Dane (CRS) and Michael (PCI) will focus on logistics (hotel and transport) for the JMTR Team. Observers are asked to utilize the services of their own organization to arrange logistics for them for the JMTR. They can ask if there is space in the JMTR vehicles for them, but Dane and Michael are not obligated to ensure that space is available for them.  During the field work, observers are asked not to engage core team members while they are conducting interviews or focus group discussions, nor should they ask questions directly of respondents during interviews or FGDs without obtaining permission from the core team member doing the interviews to do so. After the interview or discussion has been completed, observers are free to hold discussions with the core team member.  Observers are free to attend JMTR team meetings and evening discussions held on the second day in each district.  Observers will not be allowed to attend the Verification Workshops.  Observers who are NGO staff are welcome to attend the Stakeholder Debriefing on April 6, and Observers who are USAID stakeholders are welcome to attend the USAID Debriefing on April 7. JMTR Methodology 17 May 2017 59 ANNEX B JMTR Schedules (Overall and Field Work) The schedules for the Malawi JMTR can be found in section V beginning on page 118. JMTR Methodology 17 May 2017 60 ANNEX C JMTR DATA COLLECTION TOOLS The data collection tools used in the Malawi JMTR can be found in Section IV beginning on page 64. JMTR Methodology 5 June 2017 61 ANNEX D Malawi Joint Mid-Term Review (JMTR) ILLUSTRATIVE VERIFICATION WORKSHOP PLAN The actual Verification Workshop Plans for the UBALE and Njira Verification Workshops are provided in Section VII beginning on page 150. JMTR Methodology 5 June 2017 62 ANNEX E Malawi Joint Mid-Term Review (JMTR) REPORT FORMATS (ACTUAL) SUMMARY REPORT (JMTR Report, Volume I) (The length of this report should not exceed 50 pages, excluding Annexes. Target page limits for each section are shown in parentheses.) I. EXECUTIVE SUMMARY (3 pages) II. BACKGROUND FOR THE JMTR (3 Pages) A. Purpose of the JMTR B. JMTR Team C. JMTR Schedule and Process III. HISTORY AND CONTEXTUAL FACTORS AFFECTING THE PROJECTS (2 Pages) A. Factors Occurring at Project Start-up B. Post-Start-up Issues C. General Contextual Issues IV. REVIEW OF THE UBALE PROJECT (20 Pages) A. Overview of the UBALE Project B. Summary of Project Progress C. Recommendations for the Remaining Life of UBALE 1. Theme A: Life-Changing Impact 2. Theme B: Governance 3. Theme C: Collaborative Learning and Action for Adaptive Management 4. Theme D: Cross-Purpose Integration 5. Theme E: Using Food Strategically V. REVIEW OF THE NJIRA PROJECT (20 pages) A. Overview of the Njira Project B. Summary of Project Progress C. Recommendations for the Remaining Life of Njira 1. Theme A: Sustainable Impact 2. Theme B: Program Quality 3. Theme C: Monitoring, Evaluation and Learning 4. Theme D: Implementation Systems 5. Theme E: Using Food Strategically VI. JOINT LEARNING (1 page) VII. HIGHER LEVEL OBSERVATIONS (1 Page) VIII. CONCLUDING REMARKS (1/2 page) ANNEXES Annex A: JMTR Team Members Annex B: Detailed Observations and Additional Recommendations for UBALE Annex C: Detailed Observations and Additional recommendations for Njira JMTR Methodology 5 June 2017 63 JMTR METHODOLOGY (JMTR Report, Volume II) (There is no limit on the length of the length of this report) I. EXECUTIVE SUMMARY FOR VOLUME II II. INITIAL SCOPE OF WORK FOR THE MALAWI JMTR III. UPDATED JMTR PROTOCOL WITH INFORMATION ON ACTUAL PROCESS USED IV. DATA COLLECTION TOOLS V. DATA COLLECTION SCHEDULES VI. PERSONS INTERVIEWED AND SITES VISITED VII. FINAL WORKSHOP PLANS FOR THE VERIFICATION WORKSHOPS VIII. JMTR PROCESS REVIEW JMTR Methodology 5 June 2017 64 IV. DATA COLLECTION TOOLS JMTR Methodology 5 June 2017 65 PURPOSE 1: INCOME PURPOSE 1: INCOME Topical Outline: Purpose 1 – Production, Marketing, Agroenterprise, Finance (Marit and Rupert) UBALE Purpose 1: Vulnerable households increase incomes • Sub-Purpose 1.1: Farm management and business planning practices of vulnerable farming households improved • Sub-Purpose 1.2: Marketing club, cluster and association members’ agricultural sales increased • Sub-Purpose 1.3: Members of marketing clubs, clusters and associations increased use of sustainable financial services • Sub-Purpose 1.4: Selected individuals establish/strengthen businesses • Sub-Purpose 1.5: Joint decision making at household-level improved Njira Purpose 1: Increased food and income from agricultural and non-agricultural activities • Sub-Purpose 1.1: Increased sustainable nutrition friendly and market oriented agriculture production • Sub-Purpose 1.2: Increased agricultural and non-agricultural sales The review in this area is to formulate recommendations for: (1) scaling up interventions that are proving to be effective in achieving outputs and outcomes (2) modifying interventions to improve effectiveness in achieving outputs and outcomes (3) suspending interventions that are not proving to be effective in achieving outputs or outcomes (4) piloting new interventions that are relevant and useful for achieving outputs and outcomes (5) improving the efficiency in the use of resources Information will be gathered from project participants, implementing staff, program managers, and key informants connected to the project. The Objectives of review questions are: 1. Assess overall strategy Key Overarching Questions. What are the strengths and challenges of the project design so far? Are the assumptions made in the theory of change that informs program design still valid? What changes have occurred in the context since the project began implementation that have resulted in new or changed impact groups or the need for new types of assistance to address food insecurity. To what extent have program design processes incorporated measures to improve sustainability of program outputs and outcomes? How have the project strategies been designed to put in place the elements needed to contribute to higher-order social change? How should the projects’ Theories of Change and results frameworks be refined or modified? 2. Assess the quality of project inputs, implementation and outputs Key Overarching Questions: In each technical sector, to what extent has the project adhered to the initial technical approach, implementation plan, outputs, and participant targets included in the initial JMTR Methodology 5 June 2017 66 technical narrative? What are the strengths and challenges to the program inputs, implementation of activities and processes, the quality of outputs and the sustainability of the outcomes achieved? How have problems and deterrents been managed? How well do implementation processes adhere to internationally acceptable technical standards (for sustainable social and behavior change communication, experiential learning or learning-by-doing) and proven approaches, methodology and processes? Are the messages conveyed consistent, adequately contextualized, adequately disseminated and communicated? What commercial opportunities have been created to enable beneficiaries to expand engagement in value chains? How could they be improved? 3. Find evidence of and assess impact of changes (intended and unintended outcomes) Key Overarching Questions: What changes do community members and other stakeholders associate with project activities? Which factors appear to promote the apparent changes, and which have deterred intended changes? How do the changes correspond to those hypothesized by the projects’ results framework? 4. Review the level and effectiveness of coordination and collaboration with external organizations Key Overarching Questions. What has been the impact of the various collaborative relationships cultivated by the project toward enhancing the effectiveness of the project or efficiency in use of resources? How aligned are the strategies of the projects toward the development strategies of USAID and the GoM? What changes can be made in these collaborative relations to further enhance effectiveness and efficiency? 5. Related to collaborative learning and action Key Overarching Questions. How have UBALE and Njira management and technical specialists used data to inform programmatic decisions, referral and follow up? What processes have been instituted to improve data collection and data quality? How has the project improved effectiveness or efficiency as a result of new ideas or approaches brought into the project? How is information generated by the project used to inform decision-making? How can this be made more effective? 6. Related to sustaining project impact Key Overarching Questions. With the changes that have been observed as a result of project activities, what organizations, services or relationships are required to sustain these changes? What resources and capacities do organizations or service providers need to sustain themselves and these relationships after the project ends? What has the project done to ensure that these capacities and resources will be available after the project ends? What will motivate these organizations and service providers to continue to support the changes? What has the project done to ensure that this motivation does not diminish after the project ends? 7. Relative to the major cross-cutting themes Key Overarching Questions. How effective are program design and implementation mechanisms in addressing the cross-cutting issues of gender, governance, the environment and targeting of youth (UBALE)? What (if any) challenges have projects encountered in these areas that may not have been anticipated in the project design, and how have the projects responded? To what extent do project activities and implementation mechanisms reflect integration of these cross-cutting priorities? What steps have the projects taken to ensure that staff have adequate capacity for addressing these cross￾cutting issues? 8. Review implementation systems Key Overarching Questions. What key factors of implementation systems are associated with positively and negatively influencing the quality and sustainability of planned outputs? How could the projects be modified to improve the efficiency and effectiveness of implementation? JMTR Methodology 5 June 2017 67 Production-related topics UBALE Sub-Purpose 1.1: Farm management and business planning practices of vulnerable farming households improved Njira Sub-Purpose 1.1: Increased sustainable nutrition friendly and market oriented agriculture production • Differences between male & female application • Surprises about number of households applying improved technology • Engagement of Savings and Internal Lending Communities, WE/VSL groups – membership trends (how many, demographics, retainment), loans trends (how much, to what end, in total), leadership & governance, trading & business engagement trends • PSP’s – graduation potentials/trends, • Experience of training and equipping AEDOs and Lead Farmers – demonstration plots (numbers, practices), understanding of differing techniques & portfolio of options (climate smartness understood? What is “improved”?), training of AEDO’s & lead farmers (topics, internalization, usefulness), quality/quantity/need/demand/access of inputs, changes in input access and/or training due to dry spells of past year, EPA Resource Centers (functioning, utilization, offerings) • Video Extension methods- dissemination of video, further video plans, role of DAES • Diversity for Nutrition and Enhanced Resilience – DiNER fairs (attendance, demographics, prep), decision to split between rain-fed & winter seasons • Seed Multiplication Groups – membership trends, success of differing seeds (including maize seed bank), uptake, potential for sale and intersection with other parts of project, • Paravets- training trends, experience w MoAIWD, use of paravets in communities • Stakeholder Panels – participation, access, partnership with SANE (UBALE) • Farmer Learning Center Research - progress, initial results, partnership with other groups conducting this research Interviewee profiles: Type of Participant/Site SILC Groups Marketing Clubs SILC PSPs Marketing Clusters PSP Networks Marketing Associations Farm Demonstration Plots (Mama) Market PSPs Farm Demonstration Plots (Baby) Beneficiary-owned Businesses GoM AEDOs Basic Literacy Participants (groups assisted to register with GoM services) Lead Farmers Agro-Dealers (in proposal) DiNER Fairs (UBALE), Seed Fairs (Njira) Buyers with Marketing Linkages to Participants Seed Multiplication Groups Service Providers - Agro-Input Suppliers, MFIs, Micro￾Insurance Firms, BDS Service Providers Paravets District and Area Stakeholder Panels Learning Centers (UBALE) Questions Topics to be covered with project beneficiaries from targeted impact groups (FFA recipients, SILC groups) JMTR Methodology 5 June 2017 68 UBALE Activities Njira Food distributions Work standards Social safety net food distributions Work standards 1. Since you have participated in the UBALE or Njira project, please tell me what the project seeks to accomplish? 2. How have you participated in this project? How did you learn about it? How were you selected? For how long have you been involved? 3. Please describe how you or your family have benefitted from the project. 4. Please describe how you or your family have been negatively affected by the project. 5. Who, in your opinion, has benefitted most from the project? 6. Are there other people who should be benefitting from the project but are not? Please describe them for us. 7. What constraints do you believe inhibit the project from fully accomplishing its purposes? 8. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 9. What else would you like the reviewers to know about the project? Additional questions 1. What have you learned since the start of the project? How did you acquire that learning? (Who did you learn from or through what medium?) 2. What has changed with your farming? WHY? (Possibly dig into diversity and/or climate change) What has changed with your household finances? WHY? 3. What has changed in your life since the start of the project? Were those changes related to the project? 4. Are your neighbors engaged with the project? 5. Are there more women or men that you interact with through this project? Does this project benefit men and women? 6. Are you involved with SILC, DiNER, Lead Farmer, marketing clubs, seed multiplication groups, other groups of the project? Why? 7. In the last two years, have you bought any inputs such as seed, pesticide, fertilizer, farm materials? How did you pay? Who did you buy from? Was this a change from previous years? 8. Have you received training pesticide or fertilizer use? 9. How do you store your food? How well and how long does it last? 10. How are you ensuring the safety and efficacy of any pesticide storage, if any? 11. Do you sell your produce at the market? How often do you go? Is it profitable? Has that changed in the two years? Why, do you think? 12. What changes would you like to see next year in your life that this you think this project could provide? 13. What else would you like to eat, either buy or grow? Do you think that’s possible next year? Topics to be covered with project-created service providers/leaders (SILC, WE/VSL, PSPs, PSP Networks, Farm demo plots, Lead Farmers, Seed Multiplication Groups, Paravets, District and Area Stakeholder Panels) 1. Since you have participated in the UBALE or Njira project, please tell me what the project seeks to JMTR Methodology 5 June 2017 69 accomplish? 2. How have you participated in this project? How did you learn about it? How were you selected? For how long have you been involved? 3. Please describe how you or your group have benefitted from the project. 4. Please describe how you or your group have been negatively affected by the project. 5. Who, in your opinion, has benefitted most from the project? 6. Are there other people who should be benefitting from the project but are not? Please describe them for us. 7. What constraints do you believe inhibit the project from fully accomplishing its purposes? 8. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 9. What else would you like the reviewers to know about the project? Additional questions: 1. If SILC group: What are you saving for? Are your neighbors involved in the SILC? Who leads it? Why? How often do you meet and how easy or hard is it to participate? Will you participate next year? And the year after? 2. If SILC PSP: Why did you choose to become a PSP? How has your standing in the community changed? What have you learned? 3. If LEAD Farmer: Why did you choose to be a lead farmer? How has your standing in the community changed? What have you learned? Who did you learn from? Did you know them before? Will you do this next year/season? And the year after? What kind of support do you receive for the safety and effectiveness concerning use of pesticides and pharmaceuticals? 4. If Seed Multiplication Group: What seeds are you growing? What is the easiest to multiply? What is the hardest? What else would you like to grow in this group? Do you have the resources you need? Are you selling the seeds? 5. If paravet: Why did you choose to become a paravet? Who is teaching you and what are you learning? How has your standing in the community changed? Are people hiring you? 6. For stakeholder panels: Who else is on the panel? What is the purpose of the panel? How often do you meet and how easy or hard is it to participate? 7. For all, what was the impact of the recent drought? How is your family and farm/business doing now? 8. For all, have you ever been to a DiNER/Seed/Livestock fair? What did you learn or do there? Would you go again? Topics to be covered with project intermediaries (GoM AEDOs, Agro-input Suppliers, Agro-dealers) 1. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar) 2. How would you describe what this project seeks to accomplish? 3. How have you or your organization participated in this project? For how long have you been involved? 4. What has changed as a result of project activities? 5. How have people benefitted with respect to food security and resilience because of these changes? Which kind of people have benefitted most from the project? 6. Are there other people who should be benefitting from the project but are not? Please describe them for us. 7. In terms of what the project has done with you, what has been most useful? What has been least useful? 8. What constraints do you believe inhibit the project from fully accomplishing its purposes? JMTR Methodology 5 June 2017 70 9. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 10. What else would you like the reviewers to know about the project? UBALE Activities Njira Activities Governance training Developing constitutions and by-laws Participatory planning, monitoring an evaluation training Developing and implementing Action Plans Gender equity and diversity trainings Resource mobilization training FFA proposals Community score cards Female participation on committees Early warning toolkits Disaster risk management training Participatory Vulnerability Capacity Assessments, including gender responsive vulnerability assessment Establishing VNRMCs Planning and implementation of FFA projects Organization functionality assessments Training on management and maintenance of community assets Gender champion selection and training Disaster Rick Management Manual development and training Mapping key resources and hazards Updating district contingency plans Development of community and area contingency plans Support for developing DRM plans Installation of river line gauges and rain gauges Social safety net (FFA) planning and implementation Training on resilience design principles and techniques for FFA projects Development of water shed development plans Establish UBWINO Centers with early warning equipment Training on rainfall data collection Facilitating linkages between Civil protection Committees at different levels DRM workshops Good governance, gender and environmental training Community-level DRM campaigns D-RISK MIS Additional questions: 1. If AEDO: What techniques do you teach? Why? How often do you contact this community? Who do you interact with the most? (Women or men?) Do you have farmers requesting your services? If yes, are you able to meet the demand? If no, why not, do you think? How did the drought impact your work? What is your EPA Resource Center like? 2. If supplier/dealer: How has your business changed in the last year? Why? 3. For all, what sort of communications do you use? Radio, word-of-mouth, in-person meetings, video, phone, pamphlets? 4. For all, do you go to any of the project events like DiNER fairs/Seed Fairs/Livestock Fairs? 5. For all, do you interact with projects groups such as SILC, Seed Multiplication, Lead Farmers, paravets, etc.? Topics to be covered with partners (DiNER Fair Organizers, Learning Center partners, Comms partners) 1. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar) 2. How would you describe what this project seeks to accomplish? 3. How have you or your organization participated in this project? For how long have you been involved? Why did you get involved? 4. What has changed as a result of project activities? Has your organization changed in any way because of it? 5. How have people benefitted with respect to food security and resilience because of these changes? Which kind of people have benefitted most from the project? JMTR Methodology 5 June 2017 71 6. Are there other people who should be benefitting from the project but are not? Please describe them for us. 7. In terms of what the project has done with you, what has been most useful? What has been least useful? 8. What constraints do you believe inhibit the project from fully accomplishing its purposes? 9. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 10. What else would you like the reviewers to know about the project? UBALE Activities Njira Activities Governance training Developing constitutions and by-laws Participatory planning, monitoring an evaluation training Developing and implementing Action Plans Gender equity and diversity trainings Resource mobilization training FFA proposals Community score cards Female participation on committees Early warning toolkits Disaster risk management training Participatory Vulnerability Capacity Assessments, including gender responsive vulnerability assessment Establishing VNRMCs Planning and implementation of FFA projects Organization functionality assessments Training on management and maintenance of community assets Gender champion selection and training Disaster Rick Management Manual development and training Mapping key resources and hazards Updating district contingency plans Development of community and area contingency plans Support for developing DRM plans Installation of river line gauges and rain gauges Social safety net (FFA) planning and implementation Training on resilience design principles and techniques for FFA projects Development of water shed development plans Establish UBWINO Centers with early warning equipment Training on rainfall data collection Facilitating linkages between civil protection Committees at different levels DRM workshops Good governance, gender and environmental training Community-level DRM campaigns D-RISK MIS Additional questions: 1. For all, are you engaged with SILC, PSP’s, Lead Farmers, seed multiplication events, marketing clubs, stakeholder panels, and other groups of this project? In what way? 2. Do you attend any of the project’s events such as DiNER fairs? Who goes to these events? 3. For Farmer Learning Center Research partners, what has been the progress of the research? Your take on findings? What matters about the research (why do it)? Topics to be covered with project technical and implementation staff 1. In a Focus Group: What excites you about the project? What motivates you to get up in the morning to come to work? 2. In Focus Group: What are the challenges that the project faces? What are the challenges that you face in doing your job? 3. How would you describe what this project seeks to accomplish? How does what you do appear in the project’s Theory of Change? 4. Please list for us all the different types of activities that you have implemented as part of your job in the project 5. Which of these activities has been most effective in achieving change on food security and resilience JMTR Methodology 5 June 2017 72 for targeted impact groups? 6. Which of these activities has been least effective in achieving change on food security and resilience for targeted impact groups? 7. Who, in your opinion, has benefitted most from the project under Purpose 1? 8. Are there other people who should be benefitting from the project but are not? Please describe them 9. In terms of improving food security and resilience, what other activities could the project undertake? How important are these relative to what the project is already doing, i.e., the project budget is limited, so if we want to do something new we have to stop doing something that we are already doing in order to release resources? 10. What constraints do you face in doing your job? What suggestions do you have for addressing these constraints? 11. What else would you like the reviewers to know about the project? UBALE Activities Njira Activities Governance training Developing constitutions and by-laws Participatory planning, monitoring an evaluation training Developing and implementing Action Plans Gender equity and diversity trainings Resource mobilization training FFA proposals Community score cards Female participation on committees Early warning toolkits Disaster risk management training Participatory Vulnerability Capacity Assessments, including gender responsive vulnerability assessment Establishing VNRMCs Planning and implementation of FFA projects Organization functionality assessments Training on management and maintenance of community assets Gender champion selection and training FFA Projects Disaster Rick Management Manual development and training Mapping key resources and hazards Updating district contingency plans Development of community and area contingency plans Support for developing DRM plans Installation of river line gauges and rain gauges Social safety net (FFA) planning and implementation Training on resilience design principles and techniques for FFA projects Development of water shed development plans Establish UBWINO Centers with early warning equipment Training on rainfall data collection Facilitating linkages between civil protection Committees at different levels DRM workshops Good governance, gender and environmental training Community-level DRM campaigns D-RISK MIS FFA Projects Additional questions: 1. Do you see any difference between male and female participation? Application of technology? Ways the project engages with the two demographics? How about between younger men and older men? Older women and younger women? 2. What has surprised you? Have any of your assumptions, your own or the projects, been supported or disputed? 3. What are the major membership trends of the target groups you work with, esp. SILC, PSP’s, paravets, Lead Farmers, seed multiplication groups, and stakeholder panels? Do you see the same people joining the groups year after year? 4. What are the demographics and attendance rates at project events like DiNER fairs? Who is there? Men, women, young, old, poorest, less poor, etc? JMTR Methodology 5 June 2017 73 5. How is the project publicized? 6. Have you learned anything from this project? What is it and how did you learn? 7. What is your interaction with implementers from the other parts of the project, esp. Purpose 2 & 3? Topics to be covered on site visits (FFA SITES) UBALE FFA Sites Njira Watersheds and dams (20) Riverbanks (55) Others planned? Feeder road rehabilitation (52.7 kilometers) Dam construction River bank reinforcement Afforestation 1. How was location identified? Who owns the land on which the asset has been developed? 2. In implementing the FFA activity, what did the project provide? What did the local community provide? 3. Who benefits directly from the asset? How do they benefit? 4. Who benefits indirectly from the asset? How do they benefit? 5. What has changed as a result of the asset? 6. Who is responsible for maintaining and repairing the asset? Where do the resources come from to do this? Market, agroenterprise and finance-related topics UBALE Sub-Purpose 1.2: Marketing club, cluster and association members’ agricultural sales increased Sub-Purpose 1.3: Members of marketing clubs, clusters and associations increased use of sustainable financial services Sub-Purpose 1.4: Selected individuals establish/strengthen businesses Njira Sub-Purpose 1.2: Increased agricultural and non-agricultural sales Topics for Program Managers and Implementation Staff of the Market, Agroenterprise and Finance components General • Strategy for achieving Purpose 1: original and changes introduced during execution • Specific contribution of sub-purpose 1.2, 1.3, 1.4 (UBALE) and 1.2 (Njira) to project goal (Theory of Change) • Principal activities being implemented to achieve outputs and outcomes For each principal activity or group of activities targeted at particular participant groups (e.g. Savings and Lending groups, marketing clubs, marketing clusters, marketing associations, small and micro￾enterprises, private service providers for SILC and marketing, paravets, value chain service providers, other stakeholders) • Strengths of the approach: what is working well and why? • Challenges to the approach: what is not working well and why? How are the challenges being addressed? • Changes observed in attitudes and behavior and evidence of these. What is making these changes happen? Particular attention to use of income to improve nutritional wellbeing. • Where changes are slow or not evident, what are the obstacles to change? JMTR Methodology 5 June 2017 74 • What change do you expect to see in each of the major actors (farmers, farmer organizations, buyers, service providers, local government) by the end of the project and what will motivate these actors to sustain these changes? • Which of the changes mentioned can be replicated without further external resources? Which changes will require additional resources for them to be replicated? • What changes in the way the project is being implemented would make it more effective in achieving its outputs and outcomes? • Are the environmental impacts of the micro-enterprises activities being considered? How? Program Participants from Targeted Impact Groups Savings groups, small and microenterprises, market clubs, market clusters, marketing associations, Stakeholder Panels a) General information  Why did you come together as a group? When was the group formed? What are the principle activities that you undertake together?  How many members (by sex and age)? How are group members selected?  What are the benefits of being a member of the group? What are the ‘costs’ of being a member?  What is the food security status of most group members (no. of months when food consumption must be reduced)?  How many people in the community are worse off than the members in your group?  Do these people, or others, want to participate in your group? Can they join? If not, why not? b) Project-related  How have you participated in the project?  What do you like most about the project and why?  What do like least about the project and why?  What changes are you observing as a result of the project’s interventions? Why do you think that these are occurring?  If there have been few changes, what do you think is preventing the change?  Can you calculate costs of production? Can you prepare a plan for your business?  Does your group, or do your members, employ day laborers? If so, about how many are employed?  Are the environmental impacts of the micro-enterprises activities being considered? How?  Is there some training pertaining to workers’ safety (pesticide application protocol, safe waste disposal, etc…)  What changes would you like to see in how the activities that you are involved in are undertaken? What additional activities would make the project better and help to achieve the goals?  What will happen when the project comes to an end? How will the changes that have been occurring be sustained? Traders and Buyers of Crops and Livestock from Targeted Impact Groups Agricultural Commodity Exchange, Export Trading, Rab Processors, Afrisian, etc. • What do you buy and who do you buy from? In what volumes? • How do you interact with the farmers, market clubs, market associations created by UBALE/Njira • What support have received from UBALE/Njira? • What is working well and why? JMTR Methodology 5 June 2017 75 • What is not working so well and how can it be improved? • What changes do you observe as a result of the project’s interventions? • What buying policies or practices have you needed to change to be able to include the project’s target groups? • How can the changes being brought about by the project be sustained once the project has terminated? • Do you engage in food storage? How are you ensuring the safety and efficacy of any pesticides for commodity storage, if any? Program-created Service Providers Lead farmers, village agents, private service providers for SILC and marketing, paravets. • What services do you offer and how are they paid for? • What types of activity do you undertake? • How were you selected to become a service provider for the project? • What types of support have you received from UBALE/Njira • What kind of support do you receive for the safety and effectiveness concerning use of pesticides and pharmaceuticals? • What do you expect to gain from this experience? • What is working well? And why? • What is not working so well and can it be made to work better? • When the project ends, what will motivate you to continue providing the services to the community? Existing Public and Private Sector Agricultural and Value Chain Service Providers Agricultural Extension Development Officers, Opportunity Bank, United General Insurance, Star Insurance Brokers and Advisory Services, agri-dealers ….. • What is your role in the project? • What activities do you undertake? • What types of support have you received from UBALE/Njira • What is working well? And why? • What is not working so well and can it be made to work better? • What policies or practices have you needed to change – or will need to change - to be able to provide services to the project’s target groups? • What needs to be done to ensure that what you achieve in the project can be sustained once the project terminates? Local government The general gist will be around how farmers and their organizations are or can participate in decision￾making about the allocation of resources and the formulation and approval of by-laws that provide an enabling environment for value chain development. ===================================================================================== JMTR Methodology 5 June 2017 76 PURPOSE 2: MCHN Malawi JMTR - UBALE and Njira Projects Purpose 2: Improved Health and Nutrition of Pregnant and Lactating Women and Children under Five TOPICAL OUTLINE The purpose of the investigation using the following questions is to obtain information to be able to formulate recommendations for the remaining life of the project. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) assess the quality of the outputs produced by the project under this purpose (2) identify the outcomes and impact being generated by these outputs (3) assess the quality of the approaches used by the project (4) judge the potential sustainability of the impact that has been achieved Information will be gathered through key informant interviews, focus group discussions or large group discussions with program participants, project implementation staff, program managers and observation of project activities and outputs. For Purpose 2, the following persons or groups will be engaged for each project as indicated in interviews or focus group discussions (targeted number in each district shown in parentheses): UBALE  GOM at District Level: District Core Team /DNCC (1) including District HMT; Director of Planning and Development (1); District Environmental Officer (1) and District Water Officer (1)  Participants and Intermediaries: ACLAN and CLAN participants (2), TA MCHN structures (2); ANCC and VNCC (2); Care Group/Ration participants – FGD (2), CG promoters and lead parents (2), grandmother groups -FGD (1); CBCC caregivers (1); Water User Committees – FGD (2)  Site Visits: FFA Latrine, Hand Washing Facilities and Water Point visits (3); CCFLS/Care Group BCC activity (2); MCHN Ration Post-Distribution activity (HH visit) (2); Nutrition Fairs (1); Homestead Gardening Activity (1)-  Partners: Care and CADECOM  Project implementing staff: District MCHN Coordinator and Gender and Sustainability Field Officer (1) WASH Coordinator (1); Health Field Officers - Group (1);  Project Technical staff: TIL Health and Nutrition and Capacity Strengthening Advisor NJIRA  District Level: DHO (1); District Hospital- Machinga (others?) (1); Director of Planning and Development (1); District Environmental Officer (1) and District Water Officer (1)  Participants/Intermediaries at Area/Community Level: HSAs and CMAM volunteers (2), Health care workers at health facilities including YFHS providers (2), Cluster leaders (2), neighborhood women/PM2A participants (2 FGD), father groups (1 FGD), health promoters (2); CBDAs (2);; JMTR Methodology 5 June 2017 77 Community Leaders: TA, GVH and village heads (2); Ag Extension Officers (1); Water Point Committees –FGD (2); Area Mechanics (2)  Site Visits/Observations: GMP/Care Group BCC session (2); MCHN Ration Post-Distribution activity (HH visit) (2): Community Health Center /CBCC visits (2); Latrine, Hand Washing Facilities and Water Point visits (3); Keyhole Gardens and Cookstoves (1)  Partners – Dignitas, SSDI, ActionAid  Project implementing staff: Purpose 2 Coordinator, Sector Technicians (WASH and MCHN)  Project Technical staff: MCHN Advisor and Capacity Strengthening Advisor PROJECT BENEFICIARIES FROM TARGETED IMPACT GROUPS (especially CG participants, water management committees) 10. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar) 11. Since you have participated in the UBALE or Njira project, please tell me what the project seeks to accomplish? 12. How have you participated in this project? How were you selected? For how long have you been involved? 13. Please describe how you or your family have benefitted from the project. 14. Please describe how you or your family have been negatively affected by the project. 15. Who, in your opinion, has benefitted most from the project? 16. Are there other people who should be benefitting from the project but are not? Please describe them for us. 17. What constraints do you believe inhibit the project from fully accomplishing its purposes? 18. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 19. What else would you like the reviewers to know about the project? PROJECT INTERMEDIARIES (District Health Management Teams, District Health Officer, Clinic/Hospital In-Charge, DNHA structures, DNCC/ANCC/VNCC, Water Officer, etc.) UBALE Njira Care Groups Grandmother groups CCFLS Gardens, tippy taps Care groups and father groups CCFLS/GMP Keyhole gardens, tippy-taps, stoves UBALE Activities Njira Activities Strengthen capacity of DNHA structures (TA MCHN) Water User Committees CG Promoters and Lead Parents CBCC caregivers CLAN, ACLAN Facility assessments CBDA training on FP AEDO training on CCFLS HSA on CCM Clinics trained on Youth Friendly Health Services JMTR Methodology 5 June 2017 78 1. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar) 2. How would you describe what this project seeks to accomplish? 3. How have you or your organization participated in this project? For how long have you been involved? 4. What has changed as a result of project activities? 5. How have people benefitted with respect to health and nutrition because of these changes? Which kind of people have benefitted most from the project? 6. To what extent have you perceived changes in the people in your area as a result UBALE/Njira activities? 7. Are there other people who should be benefitting from the project but are not? Please describe them for us. 8. In terms of what the project has done with you, what has been most useful? What has been least useful? 9. What constraints do you believe inhibit the project from fully accomplishing its purposes? 10. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 11. What else would you like the reviewers to know about the project? PROJECT TECHNICAL AND IMPLEMENTATION STAFF UBALE Staff Njira Staff District MCHN Coordinator WASH Coordinator Health Field Officers TIL Health and Nutrition Capacity Strengthening Advisor Purpose 2 Coordinator Sector Technicians (WASH and MCHN) MCHN Advisor Capacity Strengthening Advisor 1. How would you describe what this project seeks to accomplish? How does what you do appear in the project’s Theory of Change? 2. Please list for us the different types of activities that you have implemented as part of your job in the project? 3. Which of these activities has been most effective in achieving change on health and nutrition for targeted impact groups? 4. Which of these activities has been least effective in achieving change on health and nutrition for targeted impact groups? 5. Who, in your opinion, has benefitted most from the project under Purpose 2? 6. Are there other people who should be benefitting from the project but are not? Please describe them 7. In terms of improving health and nutrition, what other activities could the project undertake? How important are these relative to what the project is already doing, i.e., the project budget is limited, so if we want to do something new we have to stop doing something that we are already doing in order to release resources? Water Point Committees and Area Mechanics JMTR Methodology 5 June 2017 79 8. What constraints do you face in doing your job? What suggestions do you have for addressing these constraints? 9. What else would you like the reviewers to know about the project? SITE VISITS: Health Facility Example (also have others being finalized) UBALE Site Visits Njira Site Visits Latrine, Hand Washing Facilities and Water Points; CCFLS/Care Group BCC activity MCHN Ration Post-Distribution activity Nutrition Fairs Homestead Gardening Activity Latrine, Hand Washing Facilities and Water Points CCFLS/Care Group BCC activity MCHN Ration Post-distribution activity Facility Visit: o Clinic/Hospital assessments for MNCH o Training on YFHS, Family Planning o Strengthening referral systems 1. Are you in charge of the facility /department? 2. How many years have you been qualified? 3. How long have you been at this facility? 4. How long have you been in your current position? 5. Are any other organizations providing support to this facility / district? a) Who are they and what support are they providing? 6. Are you familiar with the UBALE or Njira Project? (Describe the project, if not familiar) 7. How would you describe what the Njira project seeks to accomplish? 8. How have you or your organization participated in this project? For how long have you been involved? 9. Did you participate in any of the following activities with UBALE or Njira? a) (Baseline) assessment b) Gender assessment c) Development of action plans d) Training e) Self assessment f) Supportive Supervision g) Development of tools or activities 10. Have the below subject areas been supported (at any time) by UBALE or Njira in your area of responsibility? a) MNH human resources b) Health education c) Antenatal care d) Normal labor & delivery and essential newborn care e) Postnatal care JMTR Methodology 5 June 2017 80 f) Emergency obstetric care g) Emergency neonatal care h) KMC i) Maternal or neonate death audit j) Infection prevention k) Immunization l) IMNCI for 0 – 2 months m) IMNCI for 2 months – 5 years n) Maternal and child nutrition o) Youth Friendly Health Services p) Referrals and community follow up 11. How have people benefitted with respect to health and nutrition because of these changes? Which kind of people have benefitted most from the project? 12. To what extent have you perceived client level changes as a result of their participation in Njira activities? ============================================================================== Area Development Committee Focus Group Discussion 1. What do you know about UBALE/Njira as a program in your area? How do you work with the project/organization implementing the project? 2. What are the main issues relating to water and sanitation that concern you in your area? 3. Where do you present your water and sanitation concerns to when you have them? 4. How responsive to your water needs are the ones you take your concerns to? 5. Are there any measures that you have put in place to ensure that water sources are protected from pollution in your area? 6. Are you involved in discussions that concern water supply in your area and in the District as a whole? 7. How are you involved? 8. What specific support do you get from local government for your water points in your area? 9. What improvement would you want to see made in the way the District Council and other actors working in water supply service delivery provide support to you? 10. Which other organizations are working in your area in water supply service delivery? 11. How do you ensure that water points in your area remain functional? JMTR Methodology 5 June 2017 81 12. On the existing boreholes and other improved community water sources in your area, how were you involved (a) in selecting the site (b) in choosing the type of technology being used? ===================================================================================== Key Informant Interviews Director of Planning 1. What do you know about UBALE/Njira as a program in your area? How do you work with the project/organization implementing the project? 2. How many organizations in the district are providing WASH services? 3. Duplication of efforts are bound to happen where there are no coordination mechanisms. How wo you coordinate all the agencies and government departments implementing WASH interventions in your district? 4. What coordination challenges do you face in your district 5. What compliance measures have you put in place to ensure that WASH sector actors comply with government policies, regulations and guidelines? 6. How does water feature in your socio-economic profile in terms of priority rating? 7. How does water financing compare with other sectors in the District? 8. Does the District have a WASH planning and review forum? 9. If yes, how are community members involved? 10. Who else participates in these forums? 11. How is the implementation of WASH monitored in this district? 12. Considering that there are many government agencies and NGO partners, who leads monitoring of WASH in the district? 13. How could programs like UBALE/Njira help improve WASH service delivery at different levels in your district? ============================================================== Water Point Committee Focus Group Discussion 13. What type of water source exists in the community? Piped water, drilled well, protected dug well with hand-pump, Protected spring, Unprotected well, Unprotected spring/River/stream 14. Who constructed the borehole/water point in your area? JMTR Methodology 5 June 2017 82 15. Is the water source functional? 16. If it is not functional, how long has it been non-functional? 17. When it is broken, who is repairs it? 18. What is the longest period the water point has been broken? 19. What is the primary reason for taking that period of time to repair the water point? 20. How often does your water point/borehole break down? 21. As a water point committee, who else supports you in your work? 22. What specific support do you get from local government for your water point? 23. What should the District Council do to support you to ensure that your water points function in your area? 24. What is the alternative source of drinking water when your borehole is not functional? 25. How long has your committee been existent? 26. Has your water point committee been trained in community-based management (CBM) training? 27. If yes, has there been any difference in how you are managing your water point? 28. Can you give practical examples of what you have done since being trained? 29. Who trained you? How long was your training? 30. How differently would you have liked the training to be organized? 31. Does your committee have funds for operations and maintenance for your water point? How much do you have? 32. How did you raise the funds and how do you do that? 33. Which parts frequently wear out on your borehole and where do you buy spare parts for your borehole/water point? 34. Are there any water quality monitoring exercises taking place at your water source? 35. Who carries out the water quality monitoring of your water points? 36. Did they give you results of their water quality test results? ================================================================ Key Informant Interviews District Environmental Health Office 14. What do you know about UBALE/Njira as a program in your district? How do you work with the project/organization implementing the project? 15. What are the WASH challenges that you face in your district? JMTR Methodology 5 June 2017 83 16. Is your district running any Community-led Total Sanitation programs? 17. If yes, where are these being implemented? Who is implementing them? 18. What success has CLTS programs registered in the District? (Number of villages triggered for CLTS, Verified and Declared ODF). 19. How are the programs implemented to ensure sustainability of CLTS gains? 20. What are the general challenges that CLTS programs are facing in the districts? 21. What are CLTS management challenges that the District faces? 22. What can projects such as UBALE that are not directly implementing CLTS do to improve performance and sustainability of CLTS in your district? 23. Do you have a WASH database? How do you update it? 24. Who manages it and how are sectors involved in data generation? 25. Does the District Council have a water quality monitoring team? Who are members? 26. What water quality monitoring programs does the District Council run in the district? 27. What are the regimes and testing kits does the district use? 28. Can you give an example of how the District Council ensures that water quality testing is done on all water points before they are handed over to communities? 29. What are some of the challenges that you face in this regard? 30. Do you take water quality results back to communities for their information? 31. In places where the quality of water is beyond human consumption, how 32. When did the district last carry out the last round of water quality testing? 33. What can projects such as UBALE that are not directly implementing CLTS do to improve performance and sustainability of CLTS in your district? ================================================================== Key Informant Interviews – Guiding questions District Water Office 34. What do you know of UBALE/Njira as a program in your district? How do you work with the project? 35. What challenges do you face in water service provision in your district? 36. How do the District Council support communities to sustain their water sources? What services do you offer? 37. What other structures/arrangements are in your district that provide further support sustainability and operations and maintenance of community water systems? JMTR Methodology 5 June 2017 84 38. How do these structures collaborate with you? 39. Who regulates and certifies their operations in the district? 40. What challenges do you face with other support structures that support community water supply? 41. What do you think needs to happen to improve this form of system? 42. Does the District have a WASH planning and review forum? 43. If yes, how are community members involved? 44. Who else participates in these forums? 45. How could programs like UBALE/Njira help improve service communities at (a) District level. (b) ADC and community levels? ============================================================== Malawi JMTR - UBALE and Njira Projects Purpose 3: Community Empowerment (UBALE Project) & Capacities to Prepare for, Manage and Respond to Shocks (Njira Project) TOPICAL OUTLINE The purpose of the investigation using the following questions is to obtain information to be able to formulate recommendations for the remaining life of the project. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) assess the quality of the outputs produced by the project under this purpose (2) identify the outcomes and impact being generated by these outputs (3) assess the quality of the approaches used by the project (4) judge the potential sustainability of the impact that has been achieved Information will be gathered through key informant interviews, focus group discussions or large group discussions with program participants, project implementation staff, program managers and observation of project activities and outputs. The following persons or groups will be engaged for each project as indicated in interviews or focus group discussions (targeted number in each district shown in parentheses): UBALE Project Village Civil Protection Committees (3) Area Civil Protection Committees (1) Village Development Committees (3) Area Development Committees (1) Youth Clubs (2) JMTR Methodology 5 June 2017 85 Village Natural Resource Management Committees (3) DEC Sub-Committees (1) Food for Assets Sites (3 site visits) Groups of Food for Assets Participants (3) Gender Champions (3) Project Technical Coordinators - TIL Resilience (CARE) and FFA Adviser (CARE) Project Implementing Staff - District Capacity Strengthening Manager, Resilience District Coordinator, Resilience Field Officers & FFA Coordinator Gender and Sustainability Field Officers Njira Project Village Civil Protection Committees (3) Area Civil Protection Committees (1) District Civil Protection Committees (1) District Councils (2) Ubwino Centers (3) Department of Disaster Management Affairs in Lilongwe (1) Food for Assets Sites (3 site visits) Groups of Food for Assets Participants (3) Project Technical Coordinators - DRR Advisor & Local Capacity Strengthening Advisor Project Implementing Staff - District Program Manager, District DRR/Purpose 3 Coordinator, Sector Coordinator (Machinga), DRR Officers (Balaka), and DRR Facilitators/Sector Technicians TOPICS TO BE COVERED WITH PROJECT BENEFICIARIES FROM TARGETED IMPACT GROUPS (FFA PARTICIPANTS) 20. Since you have participated in the UBALE or Njira project, please tell me what the project seeks to accomplish? 21. How have you participated in this project? How were you selected? For how long have you been involved? UBALE Activities Njira Food distributions Work standards Social safety net food distributions Work standards 22. Please describe how you or your family have benefitted from the project. 23. Please describe how you or your family have been negatively affected by the project. 24. Who, in your opinion, has benefitted most from the project? 25. Are there other people who should be benefitting from the project but are not? Please describe them for us. 26. What constraints do you believe inhibit the project from fully accomplishing its purposes? 27. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 28. What else would you like the reviewers to know about the project? 29. ENVIRONMENT: Please describe any concerns you may have about changes in environmental hygiene (piles of waste accumulating?) or natural resource degradation (fuel or construction wood becoming harder to find). 30. What changes have you noticed in the weather/climate that affects your activities (water wells JMTR Methodology 5 June 2017 86 running dry? Floods more intense/frequent?) 31. What support have you received from the project related to adapting to environmental impact? 32. GENDER: How were women selected to participate in the FFA projects? 33. How have the needs of women been addressed in the FFA project? 34. How are women involved in making decisions about the FFA project? 35. GOVERNANCE: How well do you understand your entitlements for food distributions related to FFA projects? 36. How clear are the amounts of food you are entitled to receive and how much work is required to receive them? 37. YOUTH TARGETING & ENGAGEMENT: How are youth targeted and engaged in the FFA projects? TOPICS TO BE COVERED WITH PROJECT INTERMEDIARIES (CIVIL PROTECTION COMMITTEES, DEVELOPMENT COMMITTEES, NATURAL RESOURCE MANAGEMENT COMMITTEES, DODMA, GENDER CHAMPIONS, UBWINO CENTERS, YOUTH CLUBS, DISTRICT COUNCILS) 12. Are you familiar with the UBALE or Njira Project? (Describe the project, if the name is not familiar) 13. How would you describe what this project seeks to accomplish? 14. How have you or your organization participated in this project? For how long have you been involved? UBALE Activities Njira Activities Governance training Developing constitutions and by-laws Participatory planning, monitoring an evaluation training Developing and implementing Action Plans Gender equity and diversity trainings Resource mobilization training FFA proposals Community score cards Female participation on committees Early warning toolkits Disaster risk management training Participatory Vulnerability Capacity Assessments, including gender responsive vulnerability assessment Establishing VNRMCs Planning and implementation of FFA projects Organization functionality assessments Training on management and maintenance of community assets Gender champion selection and training Disaster Rick Management Manual development and training Mapping key resources and hazards Updating district contingency plans Development of community and area contingency plans Support for developing DRM plans Installation of river line gauges and rain gauges Social safety net (FFA) planning and implementation Training on resilience design principles and techniques for FFA projects Development of water shed development plans Establish UBWINO Centers with early warning equipment Training on rainfall data collection Facilitating linkages between Civil protection Committees at different levels DRM workshops Good governance, gender and environmental training Community-level DRM campaigns D-RISK MIS 15. What has changed as a result of project activities? JMTR Methodology 5 June 2017 87 16. How have people benefitted with respect to food security and resilience because of these changes? Which kind of people have benefitted most from the project? 17. Are there other people who should be benefitting from the project but are not? Please describe them for us. 18. In terms of what the project has done with you, what has been most useful? What has been least useful? 19. What constraints do you believe inhibit the project from fully accomplishing its purposes? 20. What suggestions do you have for addressing these constraints or otherwise enabling the project to have greater impact? 21. ENVIRONMENT: Please describe any concerns you may have about changes in environmental hygiene (piles of waste accumulating?) or natural resource degradation (fuel or construction wood becoming harder to find). 22. What changes have you noticed in the weather/climate that affects your activities (water wells running dry? Floods more intense/frequent?) 23. What support have you received from the project related to adapting to environmental impact? 24. GENDER: How are women involved your committee or organization and what are they roles? 25. How do women participate in decisions that are made by your committee? 26. What changes have you seen in gender roles as a result of project activities? 27. GOVERNANCE: How do you involve women, men and youth from different parts of the community in developing and implementing plans or proposals? 28. How are there interests represented in developing and implementing plans or proposals? 29. To whom and how do you report on implementation of plans and use of resources? 30. YOUTH TARGETING: How are youth targeted and engaged in your activities? 31. What else would you like the reviewers to know about the project? TOPICS TO BE COVERED WITH PROJECT TECHNICAL AND IMPLEMENTATION STAFF 10. How would you describe what this project seeks to accomplish? How does what you do appear in the projects Theory of Change? 11. Please list for us all of the different types of activities that you have implemented as part of your job in the project? UBALE Activities Njira Activities Governance training Developing constitutions and by-laws Participatory planning, monitoring an evaluation training Developing and implementing Action Plans Gender equity and diversity trainings Resource mobilization training FFA proposals Community score cards Female participation on committees Early warning toolkits Disaster risk management training Participatory Vulnerability Capacity Assessments, including gender responsive vulnerability assessment Establishing VNRMCs Disaster Rick Management Manual development and training Mapping key resources and hazards Updating district contingency plans Development of community and area contingency plans Support for developing DRM plans Installation of river line gauges and rain gauges Social safety net (FFA) planning and implementation Training on resilience design principles and techniques for FFA projects Development of water shed development plans Establish UBWINO Centers with early warning equipment Training on rainfall data collection JMTR Methodology 5 June 2017 88 Planning and implementation of FFA projects Organization functionality assessments Training on management and maintenance of community assets Gender champion selection and training FFA Projects Facilitating linkages between Civil protection Committees at different levels DRM workshops Good governance, gender and environmental training Community-level DRM campaigns D-RISK MIS FFA Projects 12. Which of these activities has been most effective in achieving change on food security and resilience for targeted impact groups? 13. Which of these activities has been least effective in achieving change on food security and resilience for targeted impact groups? 14. Who, in your opinion, has benefitted most from the project under Purpose 3? 15. Are there other people who should be benefitting from the project but are not? Please describe them 16. In terms of improving food security and resilience, what other activities could the project undertake? How important are these relative to what the project is already doing, i.e., the project budget is limited, so if we want to do something new we have to stop doing something that we are already doing in order to release resources? 17. What constraints do you face in doing your job? What suggestions do you have for addressing these constraints? 18. ENVIRONMENT: What do you know about the project's EMMP? 19. What are your responsibilities relative to the EMMP? 20. GENDER: What changes have you seen in gender roles as a result of project activities? 21. What could the project do differently or start doing to further facilitate positive changes on gender roles? 22. GOVERNANCE: What changes have you seen as a result of the governance interventions implemented by the project? 23. YOUTH TARGETING: which projects have specifically targeted youth under Purpose 3? 24. How have youth been engaged in these activities? 25. What else would you like the reviewers to know about the project? TOPICS TO BE COVERED ON SITE VISITS (FFA SITES) UBALE FFA Sites Njira Watersheds and dams (20) Riverbanks (55) Others planned? Feeder road rehabilitation (52.7 kilometers) Dam construction River bank reinforcement Afforestation 1. How was location identified? Who owns the land on which the asset has been developed? 2. In implementing the FFA activity, what did the project provide? What did the local community provide? 3. Who benefits directly from the asset? How do they benefit? 4. Who benefits indirectly from the asset? How do they benefit? 5. What has changed as a result of the asset? JMTR Methodology 5 June 2017 89 6. Who is responsible for maintaining and repairing the asset? Where do the resources come from to do this? 7. ENVIRONMENT: How was the environmental assessment for the FFA project conducted? 8. What environmental impacts were identified and how were these addressed? 9. GENDER: How do these FFA projects address the needs and interests of women? =============================================================== CROSS-CUTTING THEMES Malawi JMTR - UBALE and Njira Projects Gender Equity TOPICAL OUTLINES UBALE and Njira JMTR EVALUATION FGDs/ KI Topical Outlines Target: Community leaders /VDC/ADC Introduction  Who we are; why we are here; how long the process will take; What will be done with the results of our work  Participation is voluntary/ can leave at any time/ don’t have to answer anything they don’t want to. Gain consent to interview. Context setting: What activities from the Ubale / Njira project have you and your family members participated in? (Probe to understand which inventions the household may have received); How long have you been participating? Decision Making 1. Have there been any changes to how you make decisions in your household since participating in the project?  Who makes decisions in your household about what….(probe for medical treatment, education, taking out loans, what to buy for the JMTR Methodology 5 June 2017 90 home, bearing more children, etc.)? Has it always been this way? Probe for which decisions are considered “important” Probe for reasons/details for all questions. 2. Any differences in the types of important decisions that women or men have been making in the past 2 years? Probe for differences in “important” decisions for men and for women. 3. In your opinion who in a household should contribute to making decisions about crops and livestock? Who actually makes these decisions in the households? Has it always been this way? Other income-earning activities? Probe for reasons/details especially differences between women and men. 4. Are there certain types of households where women have a very strong influence in decision-making? What changes have occurred during the past 2 years? Probe for details. Types of households where women have very little influence? Probe for details. Gender roles, household dynamics and communication 5. Since you or your family began participating in the project, have your household responsibilities changed (for you personally and the community)? (domestic work, farming, income generation, and other activities). How? Why? 6. Has participating in the project activities of different households affected their relationship with their partner, whether positive or negatively? Whats your experience in your household? Probe why? 7. Are there any times when you believe it is appropriate to use violence in the household? Under what circumstances? Is it culturally acceptable for men to hit spouses? Daughters? Women to hit spouses? Daughters? Other HH members to hit females? Has there been any change on GBV? How? Why? Gender Equity 8. Have you noticed any changes among the women in your community since the project started? What are they? Probe for themes like, participation, voice, increase or changes in duties /burdens /responsibilities, well-being, access to resources, etc. 9. What is your definition of “women’s empowerment” – describe a woman whom you would consider to be “empowered” in this community. Are women more empowered now than 2 years ago? How and in what form? Probe to understand why women are not empowered. 10. Have there been any changes (in last 2 years) to the types of activities families do JMTR Methodology 5 June 2017 91 together? How couples treat each other? Describe. Why / how did change occur? a. Probe: How common is it in this community for men to accompany their wives to a hospital or clinic? Why/ why not? Examples. To public places such as church, meetings, social celebrations?) 11.What are the differences between men and women in accessing work, types of work, wages, and income generating activities? How has this changed over the last two years? Are there obstacles to women wanting to earn income Agency: Speaking up 12.In what types of settings are women/men comfortable speaking up? What subjects do people speak out about relating to community decisions/challenges? Differences between men and women speaking out? 13. Are there women in the community who have an easier time speaking up about public issues? What factors help them to speak up in public? Has this changed in the past 2 years? How? Why? Agency: Group participation/leadership positions 14.Has women holding leadership positions within the groups changed over the last 2 years? Are they involved with decision-making for the group? Has this changed over the last two years? Overall impression 15. What are the positive changes that the project has brought to this community? Are there any negative changes? Do you think there are changes that are not positive for the well-being of community members? Any changes over the last two years as a result of the Project? Probe. How would you recommend improving the project for the remaining years? Probe  Any questions for us; important information we are missing?  Repeat main objective of study and what will be done with shared information. Thanks to all for their time and active/honest participation ============================================================================== UBALE and Njira JMTR EVALUATION JMTR Methodology 5 June 2017 92 FGDs/ KI Topical Outlines Target: Gender champions/ VSL/FGD-KII Introduction  Who we are; why we are here; how long the process will take; What will be done with the results of our work  Participation is voluntary/ can leave at any time/ don’t have to answer anything they don’t want to. Gain consent to interview. Context setting: 1. What activities from the Ubale / Njira project have you and your family members participated in? (Probe to understand which inventions the household may have received); How long have you been participating? Are you or any of your family members’ part of a VSL group? Group structure 2. How long has the group been functioning? # Of members. Are certain types of men and women more likely to join than others? Explain. Trends in group number. Reasons for attrition. 3. Describe any benefits of belonging to group? Any tradeoffs/ challenges of belonging? Has this changed over the last 2 years? 4. What influenced you to be a gender champion/VSL? What issues do women face?--What issues do men face? What are you “championing”? What messages do you try to deliver? How? What have been your greatest successes so far? Greatest challenges? 5. Have you received any training from the project to support your role? If yes, to what extent have you applied the topics of the training? Gender equality /decision making 6. Have you noticed any changes among the women in your community since the project started? What are they? Probe for themes like, participation, voice, increase or changes in duties /burdens /responsibilities, well-being, access to resources, etc. 7. Have there been any changes to how you make decisions in your household since participating in the project?  Who makes decisions in your household about what….(probe for medical treatment, education, taking out loans, what to buy for the home, bearing more children, etc.)? Has it always been this way? Probe for which decisions are considered “important .Any differences in the types of important decisions that women or men have been making in the past 2 years? Probe for differences in “important” decisions for men and for women  Who make decisions about how to spend the money saved through your participation in the VSL/WE group? JMTR Methodology 5 June 2017 93 Gender roles, household dynamics and communication 8. Since you or your family began participating in the project, have your household responsibilities changed (for you personally)? (Domestic work, farming, income generation, and other activities). How? Why? Has it affected your relationship with your partner, whether positive or negatively? 9. How common is violence against women in this community? Under what conditions is it considered appropriate? Do men speak up publically about gender-based violence? Why/ why not? What type of men? Changes in last 2 years? Do women? Why/ why not? Changes in last 2 years? Gender Equity 10. What are the differences between men and women in accessing work, types of work, wages, and income generating activities? How has this changed over the last two years? Are there obstacles to women wanting to earn income 11. What is your definition of “women’s empowerment” – describe a woman whom you would consider to be “empowered” in this community. Are women more empowered now than 2 years ago? How and in what form? Probe to understand why women are not empowered. 12. Have there been any changes (in last 2 years) to the types of activities families do together? How couples treat each other? Describe. Why / how did change occur? a. Probe: How common is it in this community for men to accompany their wives to a hospital or clinic? Why/ why not? Examples. To public places such as church, meetings, social celebrations?) Agency: Speaking up 13.In what types of settings are women/men comfortable speaking up? What subjects do people speak out about relating to community decisions/challenges? Differences between men and women speaking out? 14.Are there women in the community who have an easier time speaking up about public issues? What factors help them to speak up in public? Has this changed in the past 2 years? How? Why? Agency: Group participation/leadership positions 15. Has women holding leadership positions within the groups and community changed over the last 2 years? How? Are they involved with decision-making for the group? Has this changed over the last two years? Overall impression & closing 16. Do you feel the well-being of the project participants have changed over the last two years? Better or worse or the same than 2 years ago? Probe for changes to food security, income levels, household relationships, etc.  How would you recommend improving the project for the remaining years? Probe  Any questions for us; important information we are missing? JMTR Methodology 5 June 2017 94  Repeat main objective of study and what will be done with shared information. Thanks to all for their time and active/honest participation ========================================================================== UBALE and Njira JMTR EVALUATION Target: Ubale staff for gender KI/FGD Topical Outline These questions will be asked of Ubale and Njira Staff and partner managers and staff and Local Government officials who know the project at the national and local levels. The questions are open-ended to encourage discussion. I. General Background  What is your current position? What is your association with the Ubale and Njira project? How long have you worked with Ubale and Njira?  What activities are you engaged in on gender? II. Activities A. Project Design and Implementation 1. What documents or plans currently guide your work on gender? To what extent is the original gender strategy guiding the day-to-day activities of the project? 2. Has the project had to drop any of the original planned activities? If yes, why? 3. In your opinion, were the right gender activities identified? Have the activities fit the needs of the communities? Are the activities appropriate to the local context? Why or why not? 4. Have you received any training related to gender to support you to do your work? If yes, have you applied any of what was covered? What / how? 5. How has the project scaled up efforts to engage gender champions on gender issues since the start of implementation? Has that strategy been effective? Why or why not? Changes to strategy to engage women and men? Key evolutions since project start up implementation? 6. How has the project ensured involvement of project participants for different genders, people with disability and other often-marginalized people throughout the project cycle? JMTR Methodology 5 June 2017 95 B. Collaboration with Government Extension Offices & Partners 7. Tell me a bit about your working relationship with the ADC/VDC. What other counterparts do you work with on gender-related priorities? 8. Has Ubale and Njira successfully integrated gender programme activities within the local development plans? What is the collaboration with other programs or projects? Factors affecting local collaboration. Success/ challenges? 9. Please describe the Ubale and Njira training. What types of gender training? How were participants selected to participate? What were the most successful training activities? What were less successful training activities? Why? What has been happening after the trainings? 10. What have been some of the greatest successes to date? The greatest challenges? 11. Who in a household makes important household decisions? Any changes for the past 2 years (Probe for which decisions are considered “important”; e.g., deciding medical treatment, education, taking out loans, bearing more children, etc.?) Probe for reasons/details for all questions. C. Overall impression 12. What is your understanding of empowerment? Do you think the project has been able to empower women and men in the project area? How? probe 13. What changes, positive or negative, have you noticed in the communities in which you work since the project began? Do you feel the well-being of the project participants have changed over the last two years? Probe for how it’s better/worse and why 14. In your opinion, which activities have been the most successful? For whom? What project activities should be included to promote gender and equity for the remaining period of the project? Why 15. What has been the key lessons learned from the implementation of the project as well as partnering with different stakeholders to achieve gender equality?  Any questions for us; important information we are missing?  Repeat main objective of study and what will be done with shared information. Thanks to all for their time and active/honest participation JMTR Methodology 5 June 2017 96 ========================================================================== Malawi JMTR - UBALE and Njira Projects Governance TOPICAL OUTLINES Households Questionnaire 1. Overall, how well is local government performing its core functions? Why do you think they are performing well or not so well? 2. How effective has local government been in delivering basic public services? 3. What role do civil society and community-based organizations play in delivering these basic services? How effective have they been in playing that role? 4. Do you know if your local government has a constitution? Does it have an action plan to improve delivery of these basic public services? 5. Can you describe three recent actions your local government has taken to implement this plan? What’s the difference you’ve seen in how well the local government delivers these basic public services? 6. How have you interacted with the Village Civilian Protection Committee or the Village Development Committee? Could you share with us three examples of how you or members of your family are participating with these committees – for example, in budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.? 7. How transparently are these committees managing public resources? Could you share with us three examples of how these committees are giving you greater access to information about how they are funding, managing and monitoring these projects? 8. How and how much have you or members of your family benefited from delivery of these basic services? Could you describe some specific examples? Please describe how you or members of your family could benefit more from these basic services? 9. Are there particular groups of women, men and youth in your community who should be benefitting from these basic services but are not? What constraints do you believe inhibit local government from delivering basic services in ways that benefit all women, men and youth? JMTR Methodology 5 June 2017 97 10. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities? 11. Besides elections, what kinds of accountability mechanisms do you use to hold local governments accountable to you and your communities? 12. What suggestions do you have that will help local government increase its accountability to you and your community? =============================================================== Stakeholder17 Questionnaire 1. Overall, how well is local government performing? Why do you think it is performing well or not so well? 2. How effective has local government been in delivering basic public services? 3. What role do you play in helping local government to deliver these basic public services? 4. How effective have you been in playing that role? How can you improve your performance? 5. Do you know if your local government has a constitution? Does it have an action plan to improve delivery of these basic public services? 6. Can you describe three recent actions the local government has taken to implement this plan? What’s the difference you’ve seen in how well the local government delivers these basic public services? 7. Could you share with us three examples of how local governments allow women, men and youth to participate in local governance – for example, budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.? 8. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities? 9. Are there particular groups of women, men or youth who should be benefitting from these basic public services but are not? 10. What constraints do you believe inhibit local government from delivering basic public services in ways that benefit all women, men and youth? 17 Stakeholders include CRS civil society partners, other civil society organizations, district-level officials. JMTR Methodology 5 June 2017 98 11. What suggestions do you have for addressing these constraints or otherwise for enabling local government to deliver basic public services more effectively and fairly? What specifically would you be willing to take responsibility for to address these constraints? 12. Besides elections, what kinds of accountability mechanisms do you use to increase local government’s accountability to the communities where they deliver these basic services? 13. What suggestions do you have to help local government increase its accountability to the communities and citizens it serves? ================================================================================ Government Officials Questionnaire 1. Overall, how well is local government performing? Why do you think they are performing well or not so well? 2. How effective has your particular institution been in delivering basic public services? 3. What role do civil society and community-based organizations play in helping you to deliver these basic public services? 4. How effective have civil society and community-based organizations been in playing that role? How can they improve their performance? 5. Have you participated in the UBALE good governance training? If so, does your institution now have a constitution? Do you have an action plan? 6. Could you share with us three examples of how women, men and youth are participating in helping to implement this action plan – for example., budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.? 7. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities? 8. Could you describe with us the results you have seen from implementing this action plan? In other words, what difference has it made in how well you deliver these basic public services? 9. Are there particular groups of women, men and youth who should be benefitting from these basic public services but are not? 10. What constraints do you believe inhibit your institution from delivering basic public services in ways that benefit all women, men and youth? 11. What suggestions do you have for addressing these constraints or otherwise for enabling your institution to deliver basic services more effectively and fairly? JMTR Methodology 5 June 2017 99 12. Besides elections, what kinds of accountability mechanisms are being used to increase your institution’s accountability to the communities where they deliver these basic public services? 13. What suggestions do you have to help your institution increase its accountability to the communities and women, men and youth you serve? ============================================================== Malawi JMTR - UBALE and Njira Projects Environmental Compliance and Climate Risk Management TOPICAL OUTLINE In general, we want to enquire on what potential environmental impacts have been identified and how they are dealt with. Did they plan this well in the project in terms of staffing, budgeting, monitoring. How do they develop the capacity of the project to deal with environmental and climate mitigation (training, consulting, monitoring of the activities to verify implementation at all levels). RESOURCES UBALE The more specific activities questions are related to the IEE and IEE amendment, and the BEO conditions on specific technical aspects of mitigation for certain activities. Also, the FY17 PREP ESR resulted in the BEO ETD, guiding the latest environmental concerns for the project specifically. Njira The more specific activities questions are related to the IEE and IEE amendment (with a memorandum), and the BEO conditions on specific technical aspects of mitigation for certain activities. Additionally, some Malawi-specific remarks on environmental concerns were recently gathered in a memo. The original M&E workshop for those project (2014) had a presentation on environmental compliance, and the DCHA BEO representative wrote a USAID report on how CRS and PCI were planning on implementing environmental compliance. Country-specific documentation and remarks. Malawi-specific remarks on environmental concerns were recently gathered in a memo. The original M&E workshop for those project (2014) had a presentation on environmental compliance, and the DCHA BEO representative wrote a USAID report on how CRS and PCI were planning on implementing environmental compliance. BACKGROUND Country level observations and concerns from M&E memo Major environmental management concerns: over-abstraction of water resources, degradation of water quality, protected area encroachment and indirect effects on public health, among others. JMTR Methodology 5 June 2017 100 Environment and Climate: Please note that all of the environmental impact concerns identified here in this memo also have a climate adaptation risk perspective, all water related. The only climate mitigation component may be related to cookstoves. Water for Irrigation: both ground and surface water Given the scale of these activities (PCI Njira implementing 220 ha of irrigation and constructing TBD amount of check dams; CRS UBALE constructing 12 dams), the major environmental concerns are with surface/ground water levels and quality; upstream/downstream effects; the use of “local stones” for construction, which risks riverbed damage; and whether or not qualified engineers are involved with project design. Protected Areas For CRS UBALE, the major concerns are that the successful support to crops such as coffee, tea and macadamia might drive conversion of protected or high-value forests around Blantyre and Nsanje, such as Mwabvi Wildlife Reserve, Lengwe National Park, Majete National Park, or Michiru Mountain Conservation area. For PCI Njira, sustainable water management near a Ramsar site. Njira is no longer pursuing activities within a Ramsar site, but outside of it. Still, the activities should be mindful of the capacity of wetlands for the large scale irrigation proposed (PCI Njira implementing 220 ha of irrigation and constructing TBD amount of check dams). Also, the farmer’s main concerns were about damage done to infrastructure by elephants. The BEO advised the project to coordinate with the National Park authorities to mitigate this problem. Pesticides: Concerns with driving the price of the land up with improved agricultural productivity, therefore threatening protected areas that are close-by. Also, counterfeit pesticides/fertilizers, potential for water pollution. Concerns about staff training. Permethrin is a pesticide used against mosquitoes, but has recently been shown to lose it efficiency as temperatures increase. NRM: For CRS UBALE, agroforestry, as well as tree nursery operations in support of agroforestry, perennial trees and fruit trees. They are also establishing one tree nursery per Group Village Headman (who oversees multiple villages) to help restore historically forested degraded lands. For PCI Njira, the project is using targeted reforestation to complement CSA, and is developing tree nurseries to complement project seed replication and planting material plots. They are also promoting on-farm agroforestry using trees that fix nitrogen, create fodder, or provide natural forms of pest controls. PCI Njira is also putting 10 sites (such as deforested ravines and hillside agriculture plots) under community-based watershed management which includes afforestation and reforestation in upper watersheds to increase filtration to groundwater and reduce runoff Cookstoves: production of fired-clay cook stoves Requires a high level of training and follow up to make cookstoves actually use less fuel wood. Climate Change concerns (from Kyle): JMTR Methodology 5 June 2017 101 Contour was not 90 degrees angle. Water retention pit: dug through rock for one. Better supervision was needed. Issue of the support from project ! ----------------------------------------------------------------------------------------- GUIDELINES FOR VISITS: Observations during interviews: connection between NGO and community, level of trust, interactions between different actors. HQ and COP: staffing for environmental compliance oversight Staffing for EMMP implementation Staffing for climate risk management Ask multiple parties to triangulate Processes in place/systems How emmp implemented as a cross cutting theme How are climate issues addressed FOR OFFICE VISITS: Primer: Environmental impact of activities need to be taken into consideration to increase resilience and ultimately the success of the activities and project, and the sustainability of their benefits to the community. As you know, to address this issue, The IEE identifies environmental vulnerabilities that can impede the success of your project. The EMMP is the implementation plan of the IEE. Mention that you are looking forward to see how they are implementing the EMMP in the field. 1. What systems are in place for the implementation of your EMMP ? a. Who is responsible for oversight? COP? DCOP? TL? b. How often is EMMP information collected by your field monitors? ¼ reports? 2. What is the system for your EMMP monitoring to be related to your performance monitoring system ? Primer: In order to comply with environmental safeguards, and climate risk management in a rigorous way, the project need to invest in capacity and budget to do so. Failing to plan ahead means that environmental compliance will either not be respected, or will have to be paid from activities budget, which is not desirable. Here, we are trying to see if the 3 main tools to mitigate environmental constraints CAN be achieved properly. 3. Staffing 4. Monitoring 5. Budget (only ask COP for budget) FIELD VISITS: Process and Technical a. PROCESS JMTR Methodology 5 June 2017 102 Primer: As you know, we have lots of good intentions with our projects, but sometimes they can have risks that we didn’t plan for. We want to Do No Harm. For example, when we cutting down trees, or using pesticides, planting a fast growing tree that is too water demanding and will drain underground water in a drought prone area…). It is important to ask these questions because as an interdisciplinary issue, environmental hygiene and natural resources will rarely come up when enquiring a community. However, these issues are tied to the success and sustainability of the project, we should be particularly attentive to issues of baseline environmental hygiene cleanliness (waste management, water quality, etc…) and natural resources landscapes (forests, ecosystem services, hydrology etc…) 1. How is your emmp being monitored to address cross cutting environmental constraints? 2. What environmental constraints have come up since the project started ? What suggestions do you have for addressing these constraints? b. TECHNICAL/SPECIFIC Primer: As you know, the following were ID’d --- Look at last ESR, draw from the ETD, and IEE/Amended IEE 2. Invasives 3. Tree species: fast growing for timber? Are they too water demanding for the region? 4. Protected Areas (National Parks, Ramsar sites, etc…) 5. Irrigation 6. Etc… see purpose 3 Topical Outline. COMMUNITY VISITS Community perception of the environment. 1. What does environment mean to you ? 2. How does the environment affect your daily life, and how have those changed over time? (less wood for fuel and fire, etc..) 3. How have weather patterns changed and affected your life ? (more floods, wells or rivers drying out, etc…) ===================================================================================== Malawi JMTR - UBALE and Njira Projects Cross-Cutting Theme: TARGETING OF YOUTH (UBALE Project) TOPICAL OUTLINE This topical outline provides key questions related to investigating the effectiveness of the UBALE Project in targeting out-of-school youth aged 14-24. The questions will be asked during focus group discussions held with different types of participant groups in the UBALE Project as well as specifically with representatives of Youth Clubs. Questions for Youth Clubs JMTR Methodology 5 June 2017 103 1. Please tell me about your club? How is it organized? Who are its members (age and sex)? What kinds of activities do you undertake? 2. Please tell me what you know about the UBALE Project (purpose is sustainable food security)? 3. How have you been engaged with or otherwise participated in the project? When did this participation begin? 4. Roughly what percentage of your members are out-of-school youth between the ages of 14 and 24? 5. What positive or negative changes have occurred as a result of project activities (changes in livelihoods activities, changes in knowledge on messages promoted by the project, changes in club activities, changes in the club's status, social capita or political capital), especially for out-of-school youth aged 14-24? 6. Which project activities have had the greatest impact in achieving this change? Why? 7. Which project activities are having the least amount of impact? Why? 8. Are there others in the community who should be participating in the project, especially youth between the ages of 14 and 24, but are not? Please describe them. What do they not participate? 9. What suggestions do you have for enabling the UBALE project to have greater sustained impact on out-of-school youth between the ages of 14 and 24? Questions for Other Project Participants (Producer Associations, SILC/VSL groups, Marketing Clubs, Care Groups, Village Civil Protection Committees, and FFA Participants) Note that these are questions asked in the context of FGDs or interviews being held with these groups, not in meetings organized specifically to discuss these opics) 1. How are out-of-school youth between the ages of 14 and 24 engaged in your group? How many members fall within this classification (by sex)? 2. How has the project benefitted these youth? 3. What suggestions do you have for enabling the UBALE project to have greater sustained impact on out-of-school youth between the ages of 14 and 24? =============================================================== IMPLEMENTATION SYSTEMS Malawi JMTR - UBALE and Njira Projects Context & Operating Environment TOPICAL OUTLINE (All Team Members) The following topics should be covered in interviews with representatives of consortium partners in each project, implementation staff, and participants. JMTR Methodology 5 June 2017 104  Since the UBALE or Njira Project began implementation, what changes have occurred in the operating context (e.g., major events like disasters, slow onset events like climate change, government policy changes, etc.) that have affected project implementation, either positively or negatively?  How did these specifically affect implementation? What did the project do to adapt to these changes?  How have these changes affected the food insecurity or resilience of targeted impact groups for each project? What other groups are becoming more food insecure or are having reduced resilience as a result of these contextual changes?  Given these changes in the operating environment, which project activities seem most irrelevant now in terms of having impact on the lives of targeted impact groups?  What suggestions do you have for adapting the project's strategy or implementation systems in order to be better able to respond to changes in the operating context? ================================================================================= Malawi JMTR - UBALE and Njira Projects Targeting TOPICAL OUTLINE (All Team Members) Topics for Project Managers and Implementation Staff ∞ Please explain your understanding of the Project's strategy for targeting communities and for identifying project participants from these communities. o Is the targeting criteria well understood by communities/participants? o To what extend did communities/participants participate in drawing the criteria? o What is the level of community/participants engagement in the targeting process? o Has there been any influence from Government on targeting and selection of participants? o Any political interference with targeting criteria and selection processes? o Any alignment of the targeting framework with the government of Malawi’s overall targeting framework for vulnerable households? ∞ How effective has this targeting strategy been, i.e., have the right people/communities been reached by the project or are their other communities/people that should have been targeted but were not? Who is not being reached well enough by the project? Why? ∞ Are there any mechanisms put in place to gather community feedback on targeting including inclusion and exclusion errors? ∞ In what ways did the project respond to problems with beneficiary selection that were observed? Were these responses effective, or could more have been done to resolve problems? JMTR Methodology 5 June 2017 105 ∞ How does the project monitor and manage targeting issues such as changes in the list of target communities, movement of program beneficiaries out of the target communities, non-participation of selected beneficiaries in program activities? ∞ How would you classify beneficiaries in terms of level of benefits? For example, there will be some beneficiaries in target communities that benefited from multiple program activities and others in communities benefiting from only one or a few activities. What difference, if any, do you think this made to the overall benefit obtained by the individual beneficiaries in the two different types of communities? Topics for Participants and Intermediaries ∞ What proportion of the population of the project’s target communities was selected as actual program beneficiaries? What proportion of the qualifying target group is currently getting assistance? ∞ Who else in the target communities, apart from these intended beneficiaries, has benefitted from the program and how have they benefitted? How should we describe these unintended beneficiaries in terms of food insecurity? ∞ Who else outside the target communities has benefitted from the program and how have they benefitted? Were these benefits intended by the program, or did they come about by chance? How should we describe unintended beneficiaries outside the target communities in terms of food insecurity? ∞ Some beneficiaries live in communities that benefitted from multiple program activities and others live in communities that benefitted from only one or a few program activities. Do you think this made a difference in the benefits obtained by the individual beneficiaries or not? Please give the reasons for your answer. ================================================================================== Malawi JMTR - UBALE and Njira Projects Management TOPICAL OUTLINE ∞ What structures are used to manage implementation of the UBALE/Njira Project across all the partners, including staffing of management teams and implementing partners, steering or advisory committees, meetings of managers, and so on? ∞ What is the history of these management structures, including turnover of personnel in key positions, restructuring or other changes? ∞ Who is responsible for the overall vision for the project? How well has the vision been articulated? How effectively has this vision been imparted to staff within the different agencies? ∞ Who is responsible for the vision within each implementing partner organization? Are the visions of the individual partners generally coherent with each other and with the vision at project level? If not, how are the differences managed? ∞ If there are differences, what impact has this had on project delivery? JMTR Methodology 5 June 2017 106 ∞ How are operational plans developed for the project? What has worked well in this process? What has not worked well? ∞ How are problems with implementation identified, analyzed and solved? How is information generated by the project's M&E systems used for decision-making? What has worked well in terms of problem-solving? What has not worked well? ∞ Please describe the working relationships between the UBALE/Njira CoP, UBALE/Njira project technical officers and senior leadership in your organization’s Headquarters. How much management responsibility is devolved and how much is exercised from HQ? ∞ Please describe the working relationships between the UBALE/Njira CoP and district-based managerial staff in the field. How much management responsibility is devolved to the field and how much is exercised by the CoP? ∞ What kinds of information are communicated (i) within CRS/PCI at different levels, (ii) among consortium partners at different levels, (iii) with USAID/FFP? What has worked well? What has not worked well? ∞ How well has the project communicated with external stakeholders, e.g., ministry officials (national and local), other donors and UN agencies, other NGOs with similar programs and projects in Liberia? What has worked well? What has not worked well? ∞ What have been the biggest challenges relative to administrative support for the program in the areas of financial management, commodity management, human resource management, procurement, transport or anything else related to project management? ∞ What solutions have been devised to address these challenges? What has worked well? What has not worked well? ∞ What changes would you propose to improve project management in the remaining life of the project? ==================================================================================== Malawi JMTR - UBALE and Njira Projects Partnership TOPICAL OUTLINE These questions should be asked of representatives of the following partner organizations for each Project, as shown in the table. If face to face interviews cannot be held, the questions will be conveyed remotely for a response. Type of Partner UBALE Njira Consortium Lead CRS PCI Consortium Partners CARE, Chikwawa Diocese, Save the Children, NASFAM and NCBA CLUSA Emmanuel International and Agricane Technical Partners (UBALE) or Demonstration & Replication Partners (Njira) World Agroforestry Center (ICRAF), International Potato Center (CIP), International Center for Tropical Agriculture (CIAT), GoM Department of Agricultural Research Services (DARS), and Digiral Green Department of Disaster and Management Affairs (DODMA), CNFA, Total Land Care, Sun Mountain International, and the Watershed Organization Trust Research Partners University of Notre Dame, Tufts University, University of Notre Dame, Colorado State JMTR Methodology 5 June 2017 107 the Lilongwe University of Agriculture and Natural Resources (LUANAR), the International Crops Research Institute for the Semi-Arid Tropics (ICRISAT) and Purdue University University, the Lilongwe University of Agriculture and Natural Resources (LUANAR), and the University of Malawi  In terms of working with CRS or PCI as the lead agency, what did you like about the relationship with them? What didn't you like and how would you like to see that changed in the remaining life of the project?  Relative to relationships with consortium partners, what has worked well in terms of working together to achieve impact or making more effective use of resources? What has not worked well? What would you like to see done differently in the remaining life of the project with regard to partnership relationships with consortium partners?  What other partnership relationships have been important for UBALE/Njira? What has worked well in these relationships? What has not worked well? What would you like to see done differently in the remaining life of the project with regard to partnership relationships with these other organizations?  How do you feel about how major strategic or problem-solving decisions were made in the project? How much influence did you have on the decisions? How informed were you on the reasoning behind the decisions that were made?  How would you describe your overall level of satisfaction with the UBALE/Njira Project? What would you suggest be done differently in future projects like UBALE/Njira? ==================================================================================== Malawi JMTR, March-April 2017 TOPICAL OUTLINE – Monitoring, Evaluation & Knowledge Management The purpose of the investigation is to;  Assess the adequacy and quality of the monitoring and evaluation processes used in UBALE and NJIRA projects  Review systems for capturing and documenting lessons learned and assess the extent to which they are used in project implementation and refining project design  Assess the use of monitoring and evaluation information in managing for results, adjusting project strategies and overall project decision making  Assess how well the project is seeking out, testing and adapting new ideas and approaches to enhance project implementation efficiency and effectiveness  Assess opportunities for improving efficiency and effectiveness of M&E systems Structure and Capacity of the M&E System  How does the M&E system work? Who is responsible for what? Who collects data? How often? How does the data/information flow from Partners to CRS/PCI?  Is the M&E Unit adequately staffed? What are the challenges with regards to human resources? Workload of the staff? JMTR Methodology 5 June 2017 108  What has been done so far to develop M&E capacity? Did CRS/PCI provide any M&E training to partner NGOs? What is the role of implementing partners in M&E? What is working well? What is not working? Where are the gaps? Utilization of M&E Products/Information  What is the contribution of M&E to the project?  How does the program monitor progress against the implementation plans? What has worked well? What has not worked so well? Why?  Do you have any mechanisms to know whether the outputs being achieved are turning to outcomes? Do you have evidence to show that this is happening?  How is M&E informing the programmatic decisions that have been made? Do you have any evidence of using M&E to make any programmatic adjustment?  How has the program strategy or approach changed as a result of information generated by the program's M&E systems?  How is feedback provided back down the reporting chain on program monitoring reports? What has worked well? What has not worked so well? Why? Beneficiary Accountability  How was the project staff oriented on the basic principles and practices of beneficiary accountability?  How did project staff consult with male and female beneficiaries in one or more communities to define indicators for project success following startup? How soon after startup?  What kind of feedback and response channel(s) did project staff establish with members of targeted communities? How soon was it established? How well does the channel reflect the preferences of the targeted communities?  How do project staff document and respond to community feedback received? Through what kind of media? In what language? And how frequently? Donor Accountability  Do project MEAL systems meet USAID MEAL requirements?  How do project staff communicate performance monitoring findings to key stakeholders? How frequently? Knowledge Management Knowledge In  Who is responsible for bringing new ideas and approaches into the program from outside sources? How is this responsibility formalized and monitored?  What new ideas and approaches have been brought into the program from outside the program? Who can be credited for bringing these into the program? Which of these have been most useful? Which have not been very useful? Why? Knowledge Out  Who is responsible for identifying, documenting and disseminating knowledge out of the program? JMTR Methodology 5 June 2017 109  How does the program identify best practices and lessons learned? Please provide some examples.  How do these get documented and disseminated?  What is working well as far as identifying, documenting and disseminating lessons learned and best practices from the UBALE/NJIRA experiences?  How can the program do a better job of capturing and disseminating lessons learned and best practices being generated by the project? M&E System Improvements  What lessons learned relative to M&E have emerged from the experience of UBALE/NJIRA up to now?  What is your recommendation to improve the system at various levels? At partner level? At CRS/PCI level?  Recommendations for improving M&E system efficiency?  Recommendations for improving M&E system effectiveness? Baseline Survey How have the Projects used the baseline results? Any feedback on the baseline survey? What worked? What did not work well? What can be improved upon? Meetings and People to Meet  M&E managers at CRS & PCI level  M&E Managers at Partner level  Communications Officers  Government Partners – utilization of findings;  District level M&E staff  Beneficiary database specialists  Focal persons for analysis Documents to Review: - IPTT - IR (outcome) monitoring - Activity tracking - Detailed Implementation Plans Short-term Implementation Plans Monitoring and Evaluation Plan Monitoring Tools Use of Monitoring Information Monitoring for Reporting Purposes Monitoring for Programmatic Decision-Making Collaborating, Learning and Adapting Schedule of Meetings JMTR Methodology 5 June 2017 110 Date District Institution Interviewees Method 9/3/17 Blantyre CRS MEAL Team Jenny –MEAL Head John – Senior MEAL Officer Monalisa – Knowledge Management & Learning Timbukani – Database & ICT4D Trovey – C-MIS Support KII 10/3/17 Blantyre Save the Children Project Manager MEAL Coordinator Data Entry/M7E Officer/Database focal person Post Distribution Monitoring Officers FGD/KII 13/3/17 Chikwawa CADECOM Project Manager MEAL Coordinator Data Entry/M7E Officer/Database focal person Post Distribution Monitoring Officers FGD/KII 16/3/17 Nsanje CARE Project Manager MEAL Coordinator Data Entry/M7E Officer/Database focal person Post Distribution Monitoring Officers FGD/KII ===================================================================== Malawi JMTR - UBALE and Njira Projects Integration & Complementarity TOPICAL OUTLINE With the Government’s Poverty Reduction Strategy and Other Government or Donor Strategies • How well does the strategy of the UBALE/Njira Project align with government strategies for poverty reduction, agricultural development, food and nutrition security, health and nutrition and disaster risk reduction and mitigation? • How does the UBALE/Njira Project coordinate with relevant government ministries at different levels? What works well? What has not worked well? • How does the UBALE/Njira Project coordinate with donors, UN agencies and NGOs that are funding and/or implementing similar activities in the target districts and elsewhere in Malawi? What works well? What has not worked well? Is there an intentional effort to organize the institutions’ respective efforts to avoid redundancy and increase complementarity? JMTR Methodology 5 June 2017 111 • What efforts do the UBALE/Njira Project and other NGOs make to ensure that they are utilizing similar, if not the same best practices associated with similar interventions in their respective implementation zones? What works well? What has not worked well? • What mechanisms has the UBALE/Njira Project used to resolve issues arising from conflicts between its own objectives and activities and those of other relevant actors in the Malawian context? What works well? What has not worked well? With USAID Country Strategy • Which activities in the UBALE/Njira Project have made the largest contributions to the USG strategy for Malawi? • Which activities were more marginal in terms of their contributions to the USG strategy? Across Purposes within the UBALE/Njira Project • The UBALE/Njira Project is designed to be implemented as an integrated set of activities for different beneficiary categories within each target community. To what extent do you think the project succeeded in this? • What potential synergies within the project were not sufficiently capitalized upon in terms of complementary activities that could produce benefits for all households within a target community, not only those of the beneficiaries of a specific program activity. Why? • What changes in project design and implementation methods might make it possible to realize a greater degree of synergy across different program components at community level in future programs? ================================================================================== Malawi JMTR - UBALE and Njira Projects Financial Management TOPICAL OUTLINE The following topics should be discussed with finance managers of the lead agencies and consortium partners in the project headquarters as well as in the district field offices. Budget Questions ∞ What is the current LOA approved budget? How has this been revised since project start-up? ∞ What are expenditures through the most recent reporting period in FY '17? ∞ What percentage of the approved budget has been spent through this period? ∞ How are budgets prepared and compiled? What works well? What does not work well? ∞ What suggestions do you have for improving the project's budgeting processes? ∞ The following table should be completed (the table line items can be modified as needed). Cash Expenditure Summary Projected Through 2nd Quarter FY 2017 (US$) Cost Center CDF 202e ITSH Cost Share Total Consortium Lead (CRS or PCI) JMTR Methodology 5 June 2017 112 Consortium Partners (listed individually) Total Direct Costs NICRA for CRS or PCI NICRA for Consortium Partners (listed individually) TOTAL Expenses through March 2017 TOTAL LOA Budget at Time of Approval Current Amended LOA Budget Percent of Current LOA Budget Spent by March 2017 Cash Flow ∞ How is cash for project implementation being provided to partners? ∞ How effective has cash flow been managed in the project? Have there been any significant delays in cash flow either from the donor to CRS or PCI or from CRS or PCI to the consortium partners? What was the cause of the delays? What changes were made in managing cash flow? ∞ What suggestions do you have for improving the cash flow systems in the project Reporting ∞ How do financial reports for the project get prepared? ∞ What problems have occurred with financial reporting and how have these been resolved? ∞ What suggestions do you have for improving the financial reporting systems in the project? Cost Share ∞ What is the cost-share commitment and how is this being met? ∞ What percentage of the cost-share commitment will be achieved by March 2017? ∞ What other forms of cost-share have arisen since the project was initiated? How are these being reported? Audits ∞ What audits have been completed on project funding since the project was initiated? ∞ What have been the audit findings? ∞ How have these been addressed? ============================================================================ JMTR Methodology 5 June 2017 113 Malawi JMTR - UBALE and Njira Projects Commodity Management TOPICAL OUTLINE The following topics will be covered in interviews with commodity management staff. Pipeline ∞ How does the food pipeline work? Who is responsible for call forwards, port off-loading and transport to the port warehouse, inland transport, and distribution to end users? ∞ What problems or changes have occurred with call forwards and how were these managed? ∞ What problems have occurred with port commodity management and how were these managed? ∞ What problems have occurred with inland transport and how were these managed? ∞ What problems have occurred with warehousing/final distribution and how were these managed? ∞ What lessons have we learned with commodity management in the UBALE/Njira? ∞ What losses have occurred and how did these occur? What was the impact of the losses on the project as well as on subsequent call forwards? What changes were made in commodity management systems in response to the losses? The following tables should be completed. LOA Commodity Summary (MT) Total FY 15 FY 16 FY 17 (through March 2017) Cumulative disbursements through March 2017 Original Proposed LOA Quantities Revised Projected LOA Quantities Percentage of LOA Projection Achieved Through March 2017 Commodity Loss Summary (MT) Fiscal Year Amount Purchased Ocean Losses Amount Received in Country Inland Losses Percent Lost FY 15 FY 16 FY 17 through Quarter 2 TOTAL Reporting ∞ How are commodity reports prepared? ∞ What problems have occurred in report preparation and how have these been managed? JMTR Methodology 5 June 2017 114 ∞ What has been the response from the donor on commodity reports? Rations ∞ How are ration composition and quantities calculated? ∞ How have they changed over the life of the program? ∞ How have rations been used by recipients? ∞ What changes should be considered relative to ration composition or quantities? Why should these changes be considered? ============================================================================= Malawi JMTR - UBALE and Njira Projects Human Resource Management TOPICAL OUTLINE The following topics should be covered in interviews with those responsible at different levels in the UBALE/Njira Project for human resource management. Staff Recruitment and Retention ∞ What significant challenges has the UBALE/Njira Project faced relative to human resources in the project? How did the UBALE/Njira Project deal with these challenges? ∞ How many positions total are there in the UBALE/Njira Project? How many vacancies are there at this point in time? ∞ How would you describe staff turnover in the project? How has this affected project implementation? ∞ In general, what has worked well and what has not worked well relative to hiring and retaining staff in the UBALE/Njira Project? Staff Capacity Building and Performance Management ∞ What types of activities has the UBALE/Njira Project undertaken to build staff capacities to be able to work effectively? ∞ How would you describe the systems in the UBALE/Njira Project for motivating staff and facilitating high levels of performance? What has worked well? What has not worked well? ==================================================================================== Malawi JMTR - UBALE and Njira Projects Administrative Support (Communications, Procurement, Logistics) TOPICAL OUTLINE The following topics should be covered, as relevant, in interviews with staff from all partners who are directly responsible for managing administrative support for the UBALE/Njira Project, particularly related to communications, procurement and logistics. 1. What systems are in place for ensuring effective communications related to UBALE/Njira within your organization and with other partners? What is working well? What problems have occurred JMTR Methodology 5 June 2017 115 relative to communications? How have these problems affected the project? What changes would you recommend be made to these systems to make communications more effective? 2. How is procurement done for the UBALE/Njira Project, including procurement of tools and equipment such as vehicles, as well as procurement of materials used for daily operations? What is working well? What problems have occurred relative to procurement? How have these problems affected the project? What changes would you recommend be made to the procurement procedures to make them more effective? 3. How is logistics done for the UBALE/Njira Project, especially logistics related to transportation of staff or materials? What is working well? What problems have occurred relative to logistics? How have these problems affected the program? What changes would you recommend be made to the logistics systems to make them more effective? JMTR Methodology 5 June 2017 116 V. DATA COLLECTION SCHEDULES JMTR Methodology 5 June 2017 117 FULL SCHEDULE FOR THE MALAWI JMTR Joint Mid-Term Review of FFP Projects - Malawi ACTIVITIES, RESPONSIBILITIES AND TIMELINE - January 2016 through June 2017 # Step Responsible Person Date(s) Comments Evaluation Preparation 1 Initial JMTR Team Meeting by Conference Call Call organized by FFP AoR January 19 (Thursday) The FFP M&E Adviser will introduce the team and the JMTR Team leader will facilitate the meeting to discuss the evaluation process and development of the evaluation protocol 2 Provision of initial background documentation required to develop the protocol CRS & PCI January 23 (Monday) Required background documents at this stage are the Approved Technical Narratives, Current Results Frameworks, Annual Results Reports and Project Staffing Structures 3 Completion of first draft of Evaluation Protocol JMTR Team Leader January 31 (Tuesday) The draft protocol will include the description of methodology and operational plan for the evaluation. 4 Provision of feedback on draft Evaluation Protocol FFP, PCI & CRS February 7 (Tuesday) Written feedback on the draft protocol sent to JMTR Team Leader 5 Completion of Working Draft Evaluation Protocol JMTR Team Leader February 14 (Tuesday) Draft protocol revised, incorporating feedback received 6 Second JMTR team meeting by Conference Call Call organized by FFP AoR February 17 (Friday) The FFP Senior M&E Adviser will facilitate a conference call with the JMTR Team, PCI, CRS and Mission to discuss the evaluation protocol and preparations that need to be undertaken 7 Provision of remaining background documents (via DropBox) CRS, PCI & FFP February 20 (Monday) These include the full Performance Monitoring Plan, Baseline Report, Gender Analysis, Reports on Other Formative Analyses, Quarterly Performance Reports, a list of available project databases, Environmental Mitigation and Monitoring Plan, Consultancy Reports, and Trip Reports. 8 Categorization of participating communities by criteria specified in the Evaluation Protocol CRS & PCI February 28 (Tuesday) Community lists sent to the JMTR team members JMTR Methodology 5 June 2017 118 9 Review background documentation and Development of Draft Data Collection Tools JMTR Team Members February 28 (Tuesday) Draft tools sent to the JMTR team leader to be reviewed and incorporated in the Evaluation Protocol 10 Preliminary selection of field work sites JMTR Team Leader with PCI & CRS March 3 (Friday) Field work sites will be selected jointly by JMTR Team Leader and PCI/CRS as per the Evaluation Protocol 11 Evaluation Team Arrives in Malawi JMTR Team March 5 (Sunday) Logistical arrangements by CRS/PCI JMTR Working Group. 12 JMTR Team Meeting JMTR Team March 6 (Monday) Team members meet face-to-face for the first time to get acquainted and the JMTR team leader answers any questions members have on the JMTR process New Selection of field work sites for UBALE & finalization of topical outlines JMTR Team March 7 (Tuesday) Team members finalize topical outlines and team leader works with UBALE team to finalize site selection for UBALE CRS - UBALE 15 Project Orientation UBALE CoP & Key Staff March 8 (Wednesday) Project orientation for evaluation team to further clarify project activities, stakeholders, best practices, and challenges. Individual interviews by JMTR team members with UBALE implementation staff 16 Meetings with UBALE Stakeholders in Blantyre JMTR Team Members March 9 (Thursday) Individual meetings by JMTR Team members with Implementing Partners, Technical Partners, Government Stakeholders, Private Sector Stakeholders and others based in Blantyre as specified in Evaluation Protocol 17 Field Visits to Nsanje, Chikwawa and Rural Blantyre Districts JMTR Team Members March 10 - 18 (Friday through Saturday) Field visits to three Implementing Partners in three Districts. Three days per district, with one day devoted to project orientation and District-based stakeholder interviews and two days for community visits per district. March 10-12 in Rural Blantyre March 13-15 in Chikwawa March 16-18 in Nsanje 18 Information Processing & Preparation for Verification Workshop JMTR Team Members March 19 - 20 (Sunday & Monday) Information processing, and preparation for Verification Workshop 19 Verification (Ground- JMTR Team March 21 - 22 Presentation of preliminary findings and recommendations to JMTR Methodology 5 June 2017 119 Truthing) Workshop Members with UBALE Implementing Staff (Tuesday & Wednesday) UBALE implementers for verification and refinement, identification of "Gray Areas" for further investigation 20 Post-Workshop Processing JMTR Team Members March 23 (Thursday) Finalization of key points to be put in the evaluation report PCI - NJIRA 21 Project Orientation Njira CoP & Key Staff March 24 (Friday) Project orientation for evaluation team to further clarify project activities, stakeholders, best practices, and challenges. Individual interviews by JMTR team members with Njira implementation staff 22 Meetings with non￾government Stakeholders for Njira in Blantyre and/or Zomba JMTR Team Members March 25 (Saturday) Individual meetings by JMTR Team members with Implementing Partners, Technical Partners, Private Sector Stakeholders and others based in Blantyre or Zomba as specified in Evaluation Protocol 23 Field Visits to Balaka and Machinga Districts JMTR Team Members March 26-Apr 1 (Sunday through Saturday) Field visits to Implementing Partners in two Districts. Three days per district, with one day devoted to project orientation and District-based stakeholder interviews and two days for community visits per district. March 26-28 in Balaka March 29 & 30 and April 1 in Machinga New Meetings with government Stakeholders for Njira in Blantyre and/or Zomba JMTR Team Members March 31 (Friday) Individual meetings by JMTR Team members with government partners based in Blantyre or Zomba as specified in Evaluation Protocol 24 Information Processing & Preparation for Verification Workshop JMTR Team Members April 2 - April 3 mid-day (Sunday and Monday) Information processing, and preparation for Verification Workshop 25 Verification (Ground￾Truthing) Workshop JMTR Team Members with Njira Implementing Staff April 3 mid-day & April 4 (Monday & Tuesday) Presentation of preliminary findings and recommendations to Njira implementers for verification and refinement, identification of "Gray Areas" for further investigation 26 Post-Workshop Processing, Preparation for Stakeholder Workshop JMTR Team Members April 5 (Wednesday) Finalization of key points to be put in the evaluation report and condensing information for both projects to half day presentations for each project to key stakeholders. JMTR Methodology 5 June 2017 120 new JMTR Team Travels to Lilongwe JMTR Team Members April 5 (Wednesday) Flight departure at 3:30 pm 27 UBALE & NJIRA Stakeholder Debriefing in Lilongwe JMTR Team Members April 6 (Thursday) Full day workshop with leadership of UBALE consortium and technical partners in the morning and the leadership of Njira consortium and technical partners in the afternoon . 29 De-Briefing with USAID/FFP in Lilongwe JMTR Team Leader April 7 (Friday) A brief of the JMTR product provided to representatives of USAID/FFP Malawi 30 JMTR Team Departs from Malawi JMTR team April 7 or 8 (Friday or Saturday) Departure Evaluation Recommendations Finalization and Processing 31 Draft Report Preparation, Including Development of Draft Recommendations JMTR Team Members April 10 - May 5 JMTR team members continue analysis and development of content for the Evaluation Report and JMTR team leader consolidates sections into final draft report 32 In-Country JMTR Evaluation Processing and Planning Workshop FFP AOR May 15 - 17 (Monday thru Wednesday) Based on information from draft report, Evaluation Recommendations finalized and implementation plans developed for each project. 33 Evaluation Report Finalized JMTR Team Leader May 22 Based on revisions made in the workshop to recommendations, Evaluation report finalized 34 De-briefing to DC-based Stakeholders FFP AOR? May 31 Evaluation final recommendations and preliminary plans for operationalizing recommendations presented to HQ-based personnel 35 Action Plans drafted and submitted to FFP PCI & CRS June 6 Draft Action Plans for operationalizing evaluation recommendations submitted to FFP 36 Review/revise Draft Action Plan with CRS FFP & CRS June 14 Meeting in DC with Malawi Staff joining remotely to finalize action plan 37 Review/revise Draft Action Plan with PCI FFP & PCI June 14 Meeting in DC with Malawi Staff joining remotely to finalize action plan 38 Action Plans Finalized PCI & CRS June 21 Final Action Plans, incorporating feedback from FFP, submitted to FFP JMTR Methodology 5 June 2017 121 Field Visit Schedule BLANTYRE March 11 & 12 Day 1, March 11 Team Member Activity MORNING: Mdala GVH/Chigaru TA Marit (P1) FGD with Seed Multiplication Group Interview with Lead Farmer and his/her follow-on farmers Observation of Demonstration Plot Aaron (Governance) FGD with Governance training participants Kumbukani (Gender) FGD with Gender Champions Interview with GVH Headman/VDC Chairperson Killian (Targeting) Interview with GVH Headman FGD with Youth Club AFTERNOON: Kaphikamtama GVH/Chigaru TA Marit (P1) FGD with SILC Group Interview with SILC PSP Aaron (Governance) Interview with VDC Interview with ADC Kumbukani (Gender) Sitting in on Interview done by Aaron Killian (Targeting) Sitting in on Interview done by Marit MORNING: Pensulo GVH/Kuntaja TA Rupert (P1) FGD with DiNER Fair Participants Mike M (P2) FGD with Care Group Beneficiaries Interview with Lead Parents (2-3 persons) Observe latrines Interview with CLAN Mary Pat (P2) Amos (P2 WASH) Mike D (P3) FGD with VCPC Charlotte (Environment) FGD with VNRMC AFTERNOON: Katchankhwala GVH/Kuntaja TA Rupert (P1) FGD with SILC Group FGD with Marketing Club Interview with Marketing FA or Marketing PSP Mike M (P2) Observe CCFLS Session FGD with CCFLS Participants and/or household visits with participants Interview with VNCC Mary Pat (P2) Amos (P2 WASH) Mike D (P3) FGD with ACPC Charlotte (Environment) FGD with ANRMC Day 2, March 12 Team Member Activity MORNING: Likomba GVH/Machinjiri TA Marit (P1) FGD with VCPC (conservation agriculture campaign) Interview with paravets Mike M (P2) Observation Care Group/Food Processing Session JMTR Methodology 5 June 2017 122 FGD with participants in Food Processing Session Mike D (P3) FGD with VCPC Observation of river bank protection project Interview with TA Headman/ADC Chairperson Killian (Targeting) AFTERNOON: Undi GVH/Lundu TA Marit (P1) FGD with Lead Farmer and his/her follow-on farmers FGD with Marketing Club Interview with Marketing FA or Marketing PSP Killian (Targeting) Mike D (P3) FGD with VCPC (resource mobilization) Observe maize and soybean plot MORNING: Kumponda GVH/Kapeni TA Rupert (P1) FGD with Leaders of Marketing Cluster Interview with FA associated with Marketing Cluster Aaron (Governance) FGD with VDC Interview with ADC Kumbukani (Gender) FGD with Gender Champions Interview with GVH headman or ADC/VDC Chairperson AFTERNOON: Kadikiri GVH/Kunthembwe TA Rupert (P1) FGD with Lead Farmer and his/her follow-on farmers Interview with FA or AEDO associated with the Lead Farmer Mike M (P2) FGD with Grandmothers Group Interview with CLAN or VNCC by DAPP Aaron (Governance) FGD with VCPC (afforestation and contingency plan) Kumbukani (Gender) FGD with Gender Champions Interview with GVH headman or ADC/VDC Chairperson MORNING: Mdala GVH/Chigaru TA Mary Pat (P2) FGD with men from CARE Group Interviews with Health Promoters Amos (P2 WASH) FGD with Water Point Committee (Not UBALE) Observation of WPC activities (Not UBALE) Charlotte (Environment) FGD with VCPC on River Bank Protection Observation of Nsambwali River Bank Protection FFA project AFTERNOON: Kaphikamtama GVH/Chigaru TA Mary Pat (P2) Observation of Follow-up of CCFLS participants FGD with CCFLS Participants Amos (P2 WASH) FGD with Water Point Committee Observation of WPC activities Charlotte (Environment) Observation of Govati Watershed Management Project FGD with Participants in the Watershed Management Project JMTR Methodology 5 June 2017 123 Field Visit Schedule CHIKWAWA March 14 & 15 Day 1, March 14 Team Member Activity MORNING: Nkwana GVH/Maseya TA Rupert (P1) FGD Marketing club FGD Marketing cluster Interview Marketing PSP Mary Pat (P2) FGD with DiNER Fair participants from previous rounds of fairs Interview with HSA Aaron (Governance) FGD with untrained VDC (and ADC if possible to bring them to Nkwana) Killian (Targeting & KM) FGD with Boys and Girls Youth to discuss participation in UBALE AFTERNOON: Salumeji GVH/Katunga TA Rupert (P1) FGD SILC Interview PSP SILC Interview Community Development Assistant Mary Pat (P2) Visit to a Health Facility FGD with Lead Parents Aaron (Governance) Interview with VDC ( and ADC if possible to bring them to Salumeji) FGD with Youth Club Killian (Targeting & KM) FGD with DiNER Fair Beneficiaries MORNING: Mkumaniza GVH/Ngabu TA Marit (P1) FGD with SILC group Interview with SILC PSP Mike M (P2) FGD with MCHN Ration Recipients Interview with Health Promoter Amos (P2) Visit to a Pit Latrine FFA site FGD with Water Point Committee Mike D (P3) FGD with FFA food recipients Interview Community Development Assistant Interview with representative from ADC Charlotte (Environment) DiNER Fair observation in GVH Goma Interview with DiNER fair participants in GVH Goma Observation water point committee with in Mkumaniza Kumbukani (Gender) Interview with Gender Champions Interview with a local leader from VDC or ADC AFTERNOON: Mwanabanjovu GVH/Ngowe TA Marit (P1) FGD with seed multiplier group FGD with follow-on farmers Interview with Lead Farmer and observe demo plot Mike M (P2) FGD with a CLAN FGD with Water Point Committee Observation of Water Point Amos (P2) FGD with a Care Group HH visits on WASH Mike D (P3) FGD with VDC JMTR Methodology 5 June 2017 124 Interview with GVH headman Charlotte (Environment) FGD with VCPC Interview with Youth club (DRM) FGD with representatives of village forestry club Kumbukani (Gender) Interview with Gender Champions Interview with a local leader from VDC or ADC Day 2, March 15 Team Member Activity MORNING: Jana GVH/Makhuwira TA Marit (P1) Interview with paravets FGD with DiNER Fair participants FGD with VNRMC with Mike D Mary Pat (P2) DiNER FAIR observation FGD with Lead Parents CCFLS follow up observation Amos (P2) FGD with VNCC FGD with CLAN HH visits to learn about WASH Mike D (P3) FGD with Youth Club FGD with VNRMC Observation Jama Watershed Project Kumbukani (Gender) Interview with Gender Champions Interview with a local leader from VDC or ADC AFTERNOON: Gangu GVH/Makhuwira TA Marit (P1) Interview with Lead farmer Observation of demonstration plots FGD with SILC Visit to Learning Center Mary Pat (P2) Care Group beneficiaries FGD Health facility visit Amos (P2) Visit Riverbank Protection Project (with Mike D) Health Facility visit (with Mary Pat) Mike D (P3) FGD with VNRMC Visit Riverbank Protection Project Kumbukani (Gender) Interview with Gender Champions Interview with a local leader from VDC or ADC MORNING: Kandeu GVH/Kasisi TA Rupert (P1) FGD Marketing club FGD Marketing cluster Interview Marketing PSP Mike M (P2) CCFLS follow up visit/observation Care Group Session HH visits to observe home visit Charlotte (Environment) Observation of Kandewu watershed FFA project JMTR Methodology 5 June 2017 125 FGD with VCPC/VNRMC on Kandewu watershed project Aaron (Governance) VDC (and with ADC if possible) Killian (Targeting & KM) FGD with FFA Beneficiaries AFTERNOON: Namila GVH/Mlilima TA Rupert (P1) FGD Lead farmer follow-on farmers Interview Lead farmer Interview AEDO and AEDC Observation of demonstration plots Mike M (P2) FGD with Care Group Interviews with 2 lead parents Interview with 1 promoter Charlotte (Environment) Will accompany Rupert Aaron (Governance) VDC (+ADC if possible) FGD with Youth Club Interview with VCPC (DRM training) Killian (Targeting & KM) FGD Marketing Cluster JMTR Methodology 5 June 2017 126 Field Visit Schedule NSANJE March 17 & 18 Day 1, Friday, March 17 Team Member Activity MORNING: Bitilinyu GVH/Ndamera TA Rupert (P1) FGD with producer group (Pansulo Irrigation group) Visit to one demonstration site Mary Pat (P2) Care group promoter ( Rose Paundi) FGD – Care group volunteers Ndifeamodzi Individual interview -HSA Amos (P2 WASH) Visit any water point in the community Visit household latrines (Members of Care Groups) Mike D (P3) FGD with VCPC Observe Nyangozi River Bank Protection Project FGD with Ndamera ACPC Aaron (Governance) FGD with VDC FGD with Control GVH Sit with Rupert in FGD with producer group (Pansulo Irrigation group) or with Mary Pat in FGD with Care group volunteers Ndifeamodzi Kumbukani (Gender) FGD with Gender Champions (6-8 people) Sit with Aaron on FGD with VDC AFTERNOON: Chibulu GVH/Ndamera TA Rupert (P1) FGD - VSL group (Nyansato) Interview Farmer extension Facilitator (Frank Nansangwe) Interview with PSP (Charles Malikinzi ) Mary Pat (P2) Individual interview with Family receiving safety nets ration (Rabeca Petro) FDG – Mothers – Tiyanjane Individual interview - Care group promoter ( AlufandikaAlufandika ) Amos (P2 WASH) Visit FFA latrines Interact with Medical Assistant at a Health Facility Mike D (P3) Interview with GVH Headman Interview with Community Development Assistant responsible for Ndamera TA Aaron (Governance) FGD with VDC FGD wth Nyansato Youth Club Sit with Mike in Interview with GVH Headman Kumbukani (Gender) FGD with Gender Champions Group (6 - 8 persons) Sit with Rupert in the FGD with a VSL Group MORNING: Magulugulu GVH/Malemia TA Marit (P1) FGD – VSL group (Mwayi wathu) Interview with PSP (Cecilia Beni) Mike M (P2) FGD – Mothers (no more than 10) – Tiyanjane Household Visits with Family receiving mother child ration ( Names to be provided later) and household visits JMTR Methodology 5 June 2017 127 Killian (KM & Targeting) FGD FFA beneficiaries Interview with Councilor/Traditional Leader Charlotte (Environment) Will stay with Mike M AFTERNOON: Dumba GVH/Tengani TA Marit (P1) FGD with lead farmers FGD with seed multiplication group (Dumba fruit tree nursery) Mike M (P2) FGD – Care group Volunteers (no more than 10) ( Jakaranda ) Meet with Promoter (30 minutes) - Tiyanjane (promoter should share register) Observe scheduled CCFLS (Jakaranda Care group) for 30 minutes Killian (KM & Targeting) Sit in with Marit on the VSL FGD FGD Youth group Charlotte (Environment) Sit with Marit in FGD with lead farmers and seed multiplication group Day 2, Saturday, March 18 Team Member Activity MORNING: Mphwiri GVH/Mlolo TA Rupert (P1) FGD with Marketing club (Mwayiwathu) Individual interview with PSP (Ernest Samuel) Individual interview with Agribusiness Community Agent (Mary Chipangula) Mary Pat (P2) FGD – Care group Volunteers ( Mtondo ) Individual interview with Family receiving safety nets ration ( Chrissy Madalitso) Individual interview -HSA ( Lackwell Chilongo) Amos (P2 WASH) FGD - Mlolo ADC for WASH Mike D (P3) FGD with FFA Recipients FGD with VDC Aaron (Governance) Sit with Mike D in FGD with VDC FGD with Mlolo ADC Sit with Mary Pat in FGD with Care group Volunteers or with Rupert in FGD with Marketing club Kumbukani (Gender) FGD with a Gender Champions Group (6 - 8 persons) Sit with Mike in FGD with FFA Recipients AFTERNOON: Chitseko GVH/Mlolo TA Rupert (P1) FGD with Marketing Club (Titukulane) Individual interview with PSP (Owen White) Individual interview with Agribusiness Community Agent (Sofia Hamuza) Mary Pat (P2) Individual interview - Care group promoter ( NdazionaLiwayu ) FGD – Care group Volunteers ( Tiwoloke) Individual interview with Family receiving safety nets ration ( Cecilia Mankhwala) Amos (P2 WASH) FGD- with Mathiya Water Point Committee Interview with VNCC Mike D (P3) FGD with VCPC Observe Mbwazi River Bank Project JMTR Methodology 5 June 2017 128 FGD with a group of non-participant HHs dependent on ganyu Aaron (Governance) FGD with VDC Interview with Mlolo ADC if they hadn’t gone to Mphwiri already Sit with Mike in FGD with VCPC Kumbukani (Gender) FGD with Gender Champions Group (6-8 Persons) Sit with Rupert on Marketing Club MORNING: Kachaso GVH/Chimombo TA Marit (P1) Interview with extension facilitator (Manyanya) FGD with Watershed mgmt. comm (Kachasu watershed) Mike M (P2) VNCC members FGD Health Facility Visit Killian (KM & Targeting) FGD wih VDC FGD girls youth club Charlotte (Environment) FGD with VNRMC Visit FFA activities with reps from VNRMC, reps for VCPC, reps for beneficiaries (10-12 people max, less if need to travel by car) AFTERNOON: Ngabu GVH/Ngabu TA Marit (P1) Interview with farmer extension facilitator (Rabecca Matekenya) Interview with agribusiness community agent (Geradi Akishoni) Mike M (P2) Individual interview - Care group promoter ( Conex Meza ) (40 minutes) Scheduled CCFLS visit - observation (20 minutes) Interview with Care group cluster leaders (2) (40 minutes) Killian (KM & Targeting) Interview with Councilor/Village Leader Sit in Marketing Club FGD Charlotte (Environment) FGD with VNRMC, VCPC, and FFA beneficiaries (12 people max total) Visit of FFA activity JMTR Methodology 5 June 2017 129 NON GOVERNMENT STAKEHOLDER MEETINGS Note: All JMTR members to check out at Ryalls Hotel by 10:00hrs JMTR Team member Stakeholders/Staff Venue Time Driver/Contact Vehicle # Departure Marit and Rupert ETG BT PCI Office 09:00hrs Chitani 0882678172 BS 6412 Departing Ryalls Hotel – 11:30hrs for Zomba Office S&A Cold Storage BT PCI Office 10:00hrs P1 Team Zomba Office 13:00hrs – 15:00hrs Mary Pat DCOP and Administration and Logistics Officer Zomba Office 13:00hrs Lucy Mike M Amy Blessings and Morris Zomba Office 13:00hrs Mdala 0997950135 BU 5192 Mike D COP BT PCI Office 10:00hrs Venansio 0991519360 BS 6415 Departing Ryalls Hotel for Hippo – 15:00hrs DRM Advisor (Burnnet) BT PCI Office 11:00hrs CD (Kurt) BT PCI Office 13:00hrs Killian Commodities/ Warehouse (Nazmul) BT PCI Office 10:00hrs – 12:00hrs G-Path (Daniel and Nazmul) BT PCI Office 13:00hrs – 15:00hrs Kumbukani Angela Translators (4) Innocent 0996538894 BU 7892 Translators (3) Winston 0998097797 BU 5074 Departing Zomba 15:30hrs JMTR Methodology 5 June 2017 130 Field Visit Schedule MACHINGA Day 1, Sunday, March 26 (Afternoon) Team Member Activity Distances/Time Driver/ Vehicle Contact # TEAM 1 AFTERNOON: Makata GVH/Kaploma TA From Ntaja Office to Makata 30 minutes (25 kms) Rupert (P1) + Martin FGD Production group (with marketing) Visit to kitchen gardens and observe stoves if in the same vicinity; Interview with Lead Farmer at Demonstration plot Innocent BU 7892 0996538894 Mike M (P2)+ Jimmy Care group – FGD (8 beneficiary mothers); Interview with 2 HSAs; HH visit with Lead Mother (1) with HSA and Lead Mother accompanying; Lucy (P2 WASH) Join Mike M in Care group FGD (8 mothers) Household visits for sanitation and hygiene issues Water point visit (tap) Winston BU 5074 0998097797 Kumbukani (Gender) FGD with WE Group Sit in on CARE group FGD with Zomba to Machinga BOMA: 40 kms (30 minutes) Zomba to Liwonde town: 55 kms (45 minutes) Zomba to Ntaja: 105 kms (1.5 hours) Liwonde to Ntaja: 50 kms (35 minutes) CONTACT INFORMATION: Symon Maseko (PM) 0991181377 for JMTR Team Leader Jam Masebo (P3 Lead) 0999137129 for JMTR TEAM 1 Moyenda Kaliyati (P1 Lead) 0888774242 for JMTR TEAM 2 JMTR Methodology 5 June 2017 131 Amy (Gender) + Chavuta Mike M Sit in on HP with Mike M or organize a separate interview FGD with any couples that participated in couples dialogue FGD with Gender Champions Kamwendo BS 6412 0882678172 TEAM 2 AFTERNOON: Mlimbula GVH/Nkoola TA From Ntaja Office to Mlimbula 25 minutes (22 kms) (No seed multiplication group) Marit (P1) + Zondi Visit to irrigation site FGD WE/VSL group Mdala BU 5192 0997950135 Mary Pat (P2) + Fareeda FGD with Father Group FGD with Lead Mothers Mike D (P3) + Anthony FGD with VCPC FGD with FFA Recipients Observe FFA site Martin BU 5191 0882799072 Killian (Targeting & ToC) + Chimwemwe FGD with Livestock beneficiaries Sit with Mike D with FFA Recipients Interview Village Leader Day 2, Monday, March 27 Team Member Activity Distance Driver/ Vehicle Contact # TEAM 1 MORNING: Ng'andu GVH/Ngokwe TA From Ntaja office to Ng’andu 1.5 hours (53 kms) Marit (P1) + Zondi Visit to irrigation scheme FGD with lead farmers (max. 8 persons) Mdala BU 5192 0997950135 Mike M (P2) + Jimmy FGD with CLAN (6 CLAN members) Observe Care Group meeting (1 promoter and Lead Mothers) JMTR Methodology 5 June 2017 132 Interview with Promoter Mike D (P3) + Anthony Interview with Representatives of ACPC FGD with FFA Management Committee Observe FFA site Kamwendo BS 6412 0882678172 Kumbukani (Gender) FGD with WE Group Visit an Ubwino Center & Interview Users of Center FD with Gender Champions AFTERNOON: Chidothi GVH/Ngokwe TA From Ng’andu to Chidothi 25 minutes (18 kms) From Chidothi to Ntaja office 2 hours (71 kms) Marit (P1) + Zondi FGD with CAWHs (max. 8 persons) Mdala BU 5192 0997950135 Mike M (P2) + Jimmy Visit to Health Clinic Interview with In Charge and HSA supervisor Interviews with HSAs to discuss CMAM and CCM Mike D (P3) + Anthony FGD with VCPC FGD with FFA (Safety Net Recipients) Observe FFA Site Kamwendo BS 6412 0882678172 Kumbukani (Gender) FGD with WE Group Visit an Ubwino Center & Interview Users of Center FD with Gender Champions TEAM 2 Day 2, Monday, March 27 MORNING: Mbwabwa GVH/Nyambi TA Rupert (P1) + FGD with WE/VSL Venansio 0991519360 JMTR Methodology 5 June 2017 133 Martin Visit watershed work with members of watershed development committee From Ntaja office to Mbwabwa 1 hr 20 minutes (48 kms) BS 6415 Mary Pat (P2) + Fareeda FGD Care Group beneficiaries HSA/HP interview (together) Innocent BU 7892 0996538894 Lucy (P2 WASH) FGD WPC Interview with Area mechanic Water point visit (borehole) Amy (Gender) + Chavuta Sit with Rupert in FGD with WE/VSL FGD with Couples who participated in Couples Dialogue FGD with Gender Champions Martin BU 5191 0882799072 Killian (Targeting & ToC) + Chimwemwe FGD Seed Multiplication Group FGD with Marketing Group Interview Village Leader Winston BU 5074 0998097797 AFTERNOON: Chikojo GVH/Nyambi TA From Mbwabwa to Chikojo 1 hour (30 kms) From Chikojo to Ntaja Office Rupert (P1) + Martin FGD Production group (with marketing) and irrigation scheme Visit watershed work with members of Watershed Development Committee Venansio BS 6415 0991519360 Mary Pat (P1) + Fareeda Visit Clinic Interview Community-Based Distributor A FGD with Lead mothers Innocent BU 7892 0996538894 Lucy (P2 WASH) Interview WASH Promoters and HSAs JMTR Methodology 5 June 2017 134 Household visits for sanitation facilities Water point visit 40 minutes (38 kms) Amy (Gender) + Chavuta Sit with Rupert in FGD with Production Group FGD with Couples who participated in Couples Dialogue FGD with Gender Champions Martin BU 5191 0882799072 Killian (Targeting & ToC) + Chimwemwe FGD with WE/VSL Group FGD with Irrigation Beneficiaries Interview with Village Leader Winston BU 5074 0998097797 TUESDAY, March 28 (MORNING) see page 5 Day 3, Tuesday, March 28 (Afternoon) Team Member Activity Distance Driver/ Vehicle Contact # TEAM 1 AFTERNOON: Chipolonga GVH/Chikweo TA From Ntaja Office to Chipolonga 1 hour (45 kms) Innocent BU 7892 0996538894 Rupert (P1) + Martin FDG with Producer Group Interview with Lead Farmer and visit his/her Demonstration plot. Mike D (P3) + Anthony FGD with VDC Observe FFA site with representatives of management committee Kumbukani (Gender) Interview managers of Ubwino Center FGD with Gender Champions Winston BU 5074 0998097797 JMTR Methodology 5 June 2017 135 FGD with participants in Couples Dialogue TEAM 2 AFTERNOON: Mbosongwe GVH/Kawinga TA From Ntaja Office to Mbosongwe 30 minutes (25 kms) Marit (P1) + Zondi FDG with Lead Farmers (max. 8 persons) Martin BU 5191 0882799072 Mike M (P2) + Jimmy FGD with 8 CG beneficiaries Observe Lead Mother (1) performing household visit Joint visit with Lucy Mary Pat (P2) + Fareeda Clinic HSAs Kamwendo BS 6412 0882678172 Lucy (P2 WASH) FGD Water User Association Interview trained masons/sanitation entrepreneurs/unskilled labourers Amy (Gender) + Chavuta Sit with Marit in FGD with Lead Farmers FGD with Care Group & HP Mdala BU 5192 0997950135 Killian (Targeting & ToC) + Chimwemwe Interview with Lead Farmers Interview with Village Leader Tuesday Morning Program with Stakeholders Stakeholder Place Time Venue DPD Machinga Boma 9-10 AM Boma office DHO, DMO, DEHO Liwonde 9-11 AM Machinga District Hospital Office DADO and team Ntaja 9-10 AM Machinga Agriculture Offices JMTR Methodology 5 June 2017 136 DPD: Director of Planning and Development DHO: District Health Officer DMO: District Medical Officer DEHO: District Environmental Health Officer DADO: District Agriculture Development Officer, Livestock Officer, Irrigation Officer, Crops Officer, Deputy DADO NB: If there is any work with groups going on during these days in any of the GVH, please alert the team so that they can observe the activities in process. Want to look at activities like CCFLS, growth monitoring, etc. Time plan: Need to keep introductions short so there is plenty of time for the actual meetings, FGDs and so forth. No big community meeting, speeches and so forth. There is much to cover in limited time. Big crowds are discouraged. Straight to business. Curtesy discussion can be done at the end depending on time (and this can be short). Team members have provided the size of groups: Interview is up to 3 people and FGD is 8-10 only. Choose sites for interviews and FGDs carefully….best in a member’s home area where there is some sort of boundary to avoid big crowds. There should be a plan for each site as to where each group will meet. Staff should be available but are not to influence what the beneficiaries have to contribute. They are not part of the interview or FGD but should be available for spontaneous discussions. There needs to be a key point of contact for each team and at each GVH. That staff member will travel with the appropriate team. [Include these staff and contact info on this schedule.] JMTR Methodology 5 June 2017 137 BALAKA DISTRICT FIELD WORK SCHEDULE Field Visit Schedule BALAKA Day 1, Wed, March 29 (Afternoon) Team Member Activity Distances/Time Driver/ Vehicle Contact # TEAM 1 Hinda Hinda GVH/Amidu TA From Balaka Office to HindaHinda is 43KM (30 minutes Drive) Marit (P1) + Zondi FGD with Lead Farmers (max 8 ppl) Interview with CAWH(s) Winston BU 5074 0998097797 Lucy (P2 WASH) Join Mike in Care groups Meet natural leader Household visits for sanitation and hygiene issues Mike M (P2) + Jimmy Care group FGD – beneficiaries Observe either CCFLS session/GMP session HH visits Innocent BU 7892 0996538894 Kumbukani (Gender) Interview with Village Chief Observe Ubwino Center Hippo to Balaka BOMA: 30 kms (25 minutes) Zomba to Balaka :85kms(1hr 20min) CONTACT INFORMATION: Venansio Chome (PM) 0888391595/0992538915/ Lefani Kachisuzi 0998097805 Overall Team Leader Wiscot Sapulayi 0998097082 TEAM 1 Ken Somanje 0998097786/0999158663 TEAM 2 JMTR Methodology 5 June 2017 138 TEAM 2 Mbera GVH/Kachenga TA From Balaka Office to Mbera is 27km (along dirty road) 30mins drive Rupert (P1) + Martin FGD with Njira promoted Marketing Club Joint interview with Community Agent(s) and Agribusiness Facilitator Patrice BU 5073 0993810847 Mike D (P3) + Anthony Interview with Njira DRM Field Facilitator FGD with VCPC Visit Afforestation FFA site Mdala BU 5192 0997950135 Killian (Targeting & ToC) + Chimwemwe FGD Seed Fairs Beneficiaries; FGD Stove beneficiaries Interview with Village Head Dan BU 7806 0995252621 Mary Pat(P2)+Fareeda FGD Lead mothers, interview HSA/HP (together) Martin BU 5191 0882799072 Amy (Gender) + Chavuta Join Mike FGD with VCPC FGD w/Gender champions and couples Interview water point committee, 1-3 members Interview with Village Head (If time permits) JMTR Methodology 5 June 2017 139 Day 2, Thursday (Afternoon), March 30th Team Member Activity Distance Driver/ Vehicle Contact # TEAM 1 Afternoon: Mgwira GVH/Sawali TA 12KM from Balaka office to TA Sawali (10 minutes) Mike D (P3) + Anthony FGD with Representatives from the VDC FGD with Mgwira Dam FFA Management Committee Observe Mgwira Dam Site Mdala BU 5192 0997950135 Rupert (P1) + Martin FGD with Njira promoted Marketing Club FGD with WALA initiated Market Club Joint interview with Community Agent(s) and Agribusiness Facilitator Mary Pat(P2)+ Fareeda Health facility, CBDA, HSA, youth group – about YFHS? Patrice BU 5073 0993810847 Kumbukani (Gender) FGD with VDC Observe Ubwino Center FGD with WE Group Visit an Ubwino Center & Interview Users of Center FD with Gender Champions JMTR Methodology 5 June 2017 140 TEAM 2 Day 2,Thursday, March 30 Afternoon : Toleza GVH/Sawali TA 15 KM from Balaka office (20 Minutes drive along dirty road Marit (P1) + Zondi Visit to watershed site FDG with livestock beneficiaries Martin BU 5191 0882799072 Killian (Targeting & ToC) + Chimwemwe FGD Seed multiplication group FFA beneficiaries Interview with the GVH Innocent BU 7892 0996538894 Mike M (P2) + Jimmy Interview with 2 promoters Interview with facilitator (nutrition) Visit to health clinic – focus on CMAM Lucy(P2) WASH Focus group with WPC Interview with Water Monitoring Assistant Interview with WASH Facilitator for Balaka (PCI Dan BU 7806 0995252621 Amy (Gender) + Chavuta FGD w/Gender champions and couples Interview with Gender Committee(Chair/Secretary Interview with Marketing group + Producer group Winston BU 5074 0998097797 JMTR Methodology 5 June 2017 141 Day 3, Saturday, April 1 (All day) Team Member Activity Distance Driver/ Vehicle Contact # TEAM 1 ALL DAY: Kuntiyani GVH/Kalembo TA 45 KM from Hippo View (30 Minutes drive and more walking) Innocent BU 7892 0996538894 Mike D (P3) + Anthony Interview with VCPC Visit River bank stabilization GVH Chatama TA Amidu FGD with Non-participant HHs Dependent on Ganyu Interview River Gauge Master Reader. Marit (P1) + Zondi FGD with Producer Group (max 8 ppl) FGD with seed mult group (max 8 ppl) FGD with Marketing Club initiated by WALA Mary Pat (P2) + Fareeda FGD Father Group, FGD Lead Mothers, FGD Beneficiaries, Health Clinic visit to ANC, Delivery, postnatal, <5 clinic, Youth Friendly health services Winston BU 5074 0998097797 Kumbukani FGD w/Gender champions and couples FGD with VDC Interview with Village Head and Visit Ubwino centre Dan BU 7806 0995252621 TEAM 2 MORNING: Mchenga GVH/Nsalama TA 28 KM from Hippo View Lodge to Rupert (P1) + Martin FGD Production Group and Marketing Club, with Lead Farmer present; Interview with Agribusiness Community Martin BU 5191 0882799072 JMTR Methodology 5 June 2017 142 Agent and respective Agribusiness Facilitator FDG WE/VSL group Mchenga (30 Minutes) Killian (Targeting & ToC) + Chimwemwe FGD with WE/VSL Group FGD Lead farmers Interview with Village Head Mike M (P2) + Jimmy Join Lucy with WPC Interview with nutrition facilitator HH visits Mdala BU 5192 0997950135 Lucy (P2 WASH) Focus group with WPC Interview with Water Monitoring Assistant Interview with WASH Coordinator for Balaka (PCI) Amy (Gender) + Chavuta FGD with production + Marketing group FDG – Gender Champion Interview with Village Head Patrice BU 5073 0993810847 TEAM 2 AFTERNOON: Ntalika GVH/NSalama TA 4 Km from Mchenga to Ntalika (10 minutes) Rupert (P1) + Martin Visit to irrigation scheme and FGD Water Users’ Association; Interview respective Facilitator Martin BU 5191 0882799072 Killian (Targeting & ToC) + Chimwemwe FGD Pigeon beneficiaries FGD Irrigation beneficiaries (Join Rupert for visit to the irrigation) Lucy (P2 WASH FGD with VDC Household visits Water point visit and Area mechanic interview Mdala BU 5192 0997950135 Mike M (P2) + Jimmy Observe GMP/CCFLS session FGD with Care Group beneficiaries JMTR Methodology 5 June 2017 143 HH visits Amy (Gender) + Chavuta Gender champions FDG with WE/VSL group Interview with Village Head Patrice BU 5073 0993810847 JMTR Methodology 5 June 2017 144 Thursday Morning Program with Stakeholders Stakeholder Place Time Venue JMTR Member Vehicle DPD Balaka 8:30 a.m. – 9:45 am District Council Mike D Patrice BU 5073 099381847 Departure from Shire to PCI Office - 07:45hrs Guide: Ken Somanje 0998097786 ADDRMO Balaka 9;45-10;45am District Council Mike D DIO Balaka 8:30 a.m. – 9:45 am District Council Kilian DHO Balaka 8:30 a.m. - 9:45 am DHO Office Mike M Innocent BU 7892 0996538894 Departure time from Shire Lodge to PCI Balaka Office- :07:45hrs Guide: Josephine 099791807 DNO Balaka 8:30 a.m. – 9:45 am DHO Office Mary Pat DEHO Balaka 8:30 a.m. – 9:45 am DHO Office Lucy & Mike M NUTRITION OFFICER Balaka 8:30 a.m. – 9:45 am DHO Office Mike M MNCH Balaka 8:30 a.m. - 9:45 am DHO Office Mary Pat District WASH Coord. Balaka 8:30 a.m. – 9:45 am DHO Office Lucy OPS/ Pindulani Balaka 09:00 am – 10:00am PCI Balaka Office Rupert and Marit Dan BU 7806 Departure from Shire to PCI Office 08:30hrs Guide: Wiscot 0998097082 DADO and Deputy Balaka 10 a.m. – 11:45 am DADO Office Rupert & Marit DADO team Balaka 10 a.m. – 11:45 am DADO Office Rupert And Marit CDA Balaka 10 a.m. – 11:45 am CDO District Office Kumbukani Winston BU 5074 DCDO Balaka 9.00am -10:00 a.m CDO Kumbukani JMTR Methodology 5 June 2017 145 Departure from Shire to PCI Office 08:30hrs Guide: Lefani 0998097805 Venansio Chome Balaka 11 am – 12.00 noon PCI Balaka Offices Mike D As Above P2 Facilitators Balaka 11am – 12:00 noon PCI Balaka Offices Mary Pat and Mike M Warehouse Staff Balaka 10:00am – 11:00am PCI Warehouse Killian NB: If there is any work with groups going on during these days in any of the GVH, please alert the team so that they can observe the activities in process. Want to look at activities like CCFLS, growth monitoring, etc. Time plan: Need to keep introductions short so there is plenty of time for the actual meetings, FGDs and so forth. No big community meeting, speeches and so forth. There is much to cover in limited time. Big crowds are discouraged. Straight to business. Curtesy discussion can be done at the end depending on time (and this can be short). Team members have provided the size of groups: Interview is up to 3 people and FGD is 8-10 only. Choose sites for interviews and FGDs carefully….best in a member’s home area where there is some sort of boundary to avoid big crowds. There should be a plan for each site as to where each group will meet. Staff should be available but are not to influence what the beneficiaries have to contribute. They are not part of the interview or FGD but should be available for spontaneous discussions. There needs to be a key point of contact for each team and at each GVH. That staff member will travel with the appropriate team. [Include these staff and contact info on this schedule.] JMTR Methodology 5 June 2017 146 VI. PERSONS INTERVIEWED AND SITES VISITED JMTR Methodology 5 June 2017 147 PURPOSE 1 - UBALE Date Location Name of Respondent or Type of Focus Group Team Member(s) Mar 9 Blantyre Joyce Njoloma, ICRAF Marit Mikter Chaola(CSA) & David Munthali (SMART Skills), UBALE PMU P1 staff Marit James Akai, Agribusiness Capacity Strengthening Advisor, UBALE-CLUSA, PMU P1 staff Rupert Marvin Kasambwe, Business Development Officer, Charter Insurance Rupert Mutunde Ng’ambi & Blessings Mlanga, Sales Repsresentatives, Old Mutual Life Insurance Company Rupert Jacob Kumwenda, Program Coordinator, Agricultural Commodity Exchange for Africa, ACE Rupert & Marit William Zgambo, Business Development Officer, Star Insurance Rupert Maurice Kalimba, NASFAM UBALE Coordinator, David Munthali, SMART Skills Specialist, UBALE P1 staff Rupert Chifunda Kapalamuli, John S. & John [x], CIP Marit Mary Mpinda, UBALE PMU P2 staff Marit Juma Masumba, UBALE PMU Ag./Marketing Technical Integration Lead Rupert & Marit Towera Jalakasi, Tools for Enterprise and Education Consultants (TEEC) Rupert Mar 10 Blantyre Save the Children P1 staff: Angus Nyangulu, Association Development Supervisor, NASFAM (other names not recorded) Rupert & Marit Mphatso Kafuwa, Linda Msiska, Pauleen Kadammanja, Evance Mtengula, Valens Mpheza, Charles Mthyoka, Fortunes Kapingin, Chimwamwe Chafuwa, Alexander Nguno, DADO officers & members of District Ag Extension Coordinating Committee, inc TLC, CADECOM, and SOS Rupert & Marit Patrick Kalula, Olive Temani, Agath Kasaila, Chimwamwe Chikotantha, Akoma Chigamba, Isaac Simeon, Rockford Mpohela, Agro-dealers Marit Mar 11 Rural Blantyre Mdala GVH Lead Farmers, 3m:3f Marit Seed Multiplication Group, 9 ppl, 7 members, 2 officers Marit Rural Blantyre Pensulo GVH/Kuntaja TA FGD with DiNER Fair participants: 6m:3f. Vesurio F. Zamasiya, Mary Daisi, Alex Khulumula, Anderson Masanga, Grace James, William Kapambanya, Magret Gilibati, Tomasi Chimera, Gift Phandamusala Rupert JMTR Methodology 5 June 2017 148 Date Location Name of Respondent or Type of Focus Group Team Member(s) Interview with women DiNER participant: Mary Daisi Rural Blantyre Kaphikamtama GVH SILC, 8 members, all f Marit, Killian, Kumbukani PSP providers, 2f Marit Rural Blantyre Katchankhwala GVH/Kuntaja TA FGD with SILC Group: 11f. Shira Chamba, Mary Jack, Josephine Buteo, Judith Henje, Francesi Jack, Estrey Gulo, Sella Kumasala, Magret Micheka, Dyna Kaipa, Grace Pindani, Melina Odala (PSP) FGD with Marketing Club: 3m:7f. Yusuf Zimba, Duncan Maliro, Joyce Yohane, Tadala Wisikesi, Rose Chaima, Maiden Chistina, Ruth Singano, Ireen Kazembe, Elina Manlidi Interview with Marketing FA: Nancy Lizi Rupert Mar 12 Rural Blantyre Likomba GVH VNRMC, 14 members Marit Pass-on livestock group members, 5f:2m Marit Lead Farmers, 10 members, 5 officers (m & f) Marit Rural Blantyre Gondi GVH Follow-on Farmers, 10 (mostly f), 2 Senior LF (1m:1f) Marit Marketing Club, 4 members (f), 2 officers (1m: 1f), AEDO (m) Marit Rural Blantyre Kumponda GVH/ Kapeni TA FGD with Leaders of Marketing Cluster: Esnart Kwatsalayani (Chairlady), Patrisha Misekeni (Secretary), Payuma Chinthema, Enipher Stima, Hanna Chipinda. Interview with FA associated with Marketing Cluster: Angeline Jiya Rupert Rural Blantyre Kadikiri GVH/ Kunthembwe TA Lead farmers: 4m:2w. Land Resources (2), Irrigation (1), Livestock (1), Nutrition (1), Crops (1). Interview with AEDC associated with the Lead Farmers: Clifford Malunga, AEDC Kunthembwe EPA Rupert March 13 Chikwawa CADECOM P1 staff: Lingstone Ganamba, Magret Mwale, Phillip Kondowe, Patrick Skanda, Christopher Malamba, Evans Matthews, Rupert & Marit Satiya Susan Mokwinga, Lewis Lipenga, Spencer Kamalitu (sp?), 2 DADO officers & ACD officer Rupert & Marit Ncholo Nelson Diniero, Regional Manager, Afrisian Rupert & Marit March 14 Chikwawa Mkumaniza GVH SILC, 10 members (5f:5m), 1 PSP (m) Marit PSP (m), same as above Marit Chikwawa Nkwana GVH/ Maseya TA FGD Marketing club, FGD Marketing cluster together. Interview Marketing PSP. No names taken Rupert Chikwawa Mwanabanjovu GVH Lead Farmer, 3 (2f:1m) Marit Chikwawa Kandeu GVH/ FGD Tisangalala SILC. Cathren Franckesi (Chairlady), Elizabeth Nyakhuwa (Secretary), Sisilia Changwa, Rupert JMTR Methodology 5 June 2017 149 Date Location Name of Respondent or Type of Focus Group Team Member(s) Kasisi TA Magret Pitilosi, Annie Chipangula, Rikines Hendereson, Aligineti Wotchi, Melita Sami, Hana Makwatha. Interview Gerard Chiona, Community Development Assistant and Joseph Malonga, PSP SILC and PSP Network Chairman. March 15 Chikwawa Jana GVH DiNER Fair recipients (6m:8f) Marit Paravets (3m) Marit VNRMC, 3 groups (4f:7m) Marit & Mike D Chikwawa Kandeu GVH/ Kasisi TA FGD Marketing club: 19 club members. FGD Marketing cluster: Tikambenji Chigamba (Chair), Dominiko Jamison (Vice Chair), Elube Genala (Secretary), Mary Siliva (Vice Secretary), Marko Chikudewa (Treasurer), Marketing: Alice Charles, Gosten Kasamu, Wedson Mangola, Kaunda, Adamison, Walasi Kuthabe and 5 members. Marketing Association: Tikambenji Chigamba Interview Marketing PSP: Lenard Dziko Rupert Chikwawa Gangu GVH SILC, 2 groups (~25 ppl, mostly f) Marit SILC sub-group, only f Marit Learning Center visit (~12 ppl, half m:half f) Marit Chickwawa Namila GVH/ Mlilima TA Likhubula Irrigation Scheme Basten Chilembwe, AEDC, Hanah Mwalikomo, AEDO, Jerico Chihana, AEDO, Dunia Chadla, AEDO, Lead Farmers: Josi Funsaani, Silver Dacksion, Bannet Vinya, Loveness Masinga Rupert March 16 Nsanje CARE P1 staff: Sonya Msiska, Bibana Nsunde, Catherine Ussein, Samson Banda, Peter Malata, Mphatso Mtukulo, Joesph Chibale, Potipher Mwale, Ackim Nyamba, Jackson Kachitede, Rupert & Marit Robert Malemba, Crops Officer, DADO Rupert & Marit Lucy Vumu, Community Development Officer, Alfred Ghilongo, Community Development Assistant Rupert March 17 Nsanje Magulugulu GVH/Malemia TA SILC, 9 members, 4 officers (all f) Marit PSP (f) Marit Nsanje Bitilinyu GVH/Ndamera TA FGD with Pansulo Irrigation group Present: Chair, Vice Chair, Secretary, Vice Secretary, Treasurer, 5 out of 10 Committee Members Rupert Nsanje Dumba GVH/Tengani TA Seed multiplication group 10 members (mostly women), 4 officers (all m) Marit, Charlotte Lead Farmers (4m) Marit, Charlotte Chibulu GVH/Ndamera TA FGD with VSL group: Treasurer, Key Holder, Money Counter, Secretary and 15 members. Interviews: Cosmas Musdwa, AEDO, and Eric Sambira Rupert JMTR Methodology 5 June 2017 150 Date Location Name of Respondent or Type of Focus Group Team Member(s) Community Development Officer, Charles Malikilozi, PSP. Interview with PSP: March 18 Nsanje Mphwiri GVH/Mlolo TA FGD with Mwayiwathu Marketing Club: Hamilton Sawasawa, Chairman, Musadane Nkhanyaule, Vice Chairman, Elida Livison, Secretary, Samison Ziwazako, Vice Secretary and 12 members (6m:6f) Interview with Mary Chipangula, Marketing Field Agent Rupert Nsanje Chitseko GVH/Mlolo TA FGD with Titukule Marketing Club: Marko Kavalo, Chairman, Kenedy Jeke, Vice Chairman, Veronica Kalisto, Secretary, Bone Meleka, Vice Secretary, Sofia Hamuza, Treasurer, and 16 members (8m:8 f) Rupert March 22 Blantyre Elsie Mwimba, Financial Products and Business Development Specialist, NCBA-CLUSA Rupert April 8 Lilongwe Marie Cadrin, CoP, Strengthening Agriculture and Nutrition Extension (SANE) Rupert PURPOSE 2 - UBALE Date Location Name of Respondent or Type of Focus Group Team Member(s) March 8 Blantyre Interviews with UBALE Staff Mike M, Mary Pat & Amos March 9 Blantyre Interviews with Blantyre Stakeholders for UBALE Mike M, Mary Pat & Amos March 10 Blantyre Interviews with Save the Chuldren Staff Mike M, Mary Pat Interviews with Rural Blantyre Stakeholders & Amos March 11 Pensulo GVH, Kuntaja TA FGD with Care Group Beneficiaries Mike M, Mary Pat & Amos Interview with Lead Parents Observe Latrines Interview with a CLAB Katchankwala GVH, Kuntaja TA Observe CCFLS Session Mike M, Mary Pat & Amos FGD with CCFLS participants Household visits with CCFLS participants Interview with VNCC March 12 Likomba GVH, Machinjiri TA Observation Care Group Food processing Session Mike M FGD with participants in Food Processing Session Kadikiri GVH, Kunthembwe TA FGD with Grandmothers Group Interview with CLAN and VNCC supported by Mike M DAPP Mdala GVH, Chigaru TA FGD with men associated with a Care Group Mary Pat Interviews with health promoters FGD with Water Point Committee not supported Amos JMTR Methodology 5 June 2017 151 by UBALE Observation of WPC activities Kaphikamtana GVH, Chigaru TA Observation of follow-up activities with CCFLS participants Mary Pat FGD with CCFLS participants FGD with Water Point Committee Amos Observation of WPC activities March 13 Chikwawa Interviews with CADECOM Staff Mike M, Mary Pat Interviews with Chikwawa Stakeholders & Amos March 14 Nkwana GVH, Maseya TA FGD with DiNER Fair participants from previous rounds of fairs Mary Pat Interview with an HAS Salumeji GVH, Katunga TA Visit to a health Facility Mary Pat FGD with lead parents Mkumaniza GVH, Ngabu TA FGD with MCHN Ration Recipients Mike M Interview with a Health Promoter Visit to a Pit Latrine FFA site Amos FGD with a Water Point Committee Mwanabanjovu GVH, Ngowe TA FGD with a CLA FGD with a Water Point Committee Mike M Observation of a water point FGD with a Care Group Amos HH visits for WASH March 15 Jana GVH, Makhuwira TA FGD with lead parents Mary Pat CCFLS follow-up observation FGD with a VNCC FGD with a CLAN Amos Household visits to learn about WASH Gangu GVH, Makhuwira TA FGD with Care Group Mary Pat Visit to a Health Facility Mary Pat & Amos Visit to a Riverbank Proection FFA project Amos Kandeu GVH, Kasisi TA CCFLS follow-up visit and observation Observation of a Care group Session Mike M Household visits to observe home counseling Namila GVH, Mlilima TA FGD with a Care group Interviews with lead parents Mike M Interview with Health Promoter March 16 Nsanje Interviews with CARE Staff Mike M, Mary Pat Interviews with Nsanje Stakeholders & Amos March 17 Bitilinyu GVH, Ndamera TA Interview with Care Group Promoter FGD with Care Group Volunteers Mary Pat Interview tih a Health Surveillance Assistant Visit to a Water Point Amos Visit to household latrines Chibulu GVH, Interview with a family receiving safety net rations Mary Pat JMTR Methodology 5 June 2017 152 Ndamera TA FGD with a mothers group Interview with a Care Group Promoter Visit to a Latrines FFA site Interniew with the Medical Assistant at the health Amos Facilitiy with the Latrines FFA Magulugulu GVH, Malemia TA FGD with a Mothers Group Household visits to a family receiving MCHN Mike M rations Dumba GVH, Tengani TA FGD with Care group Volunteers Interview tih Health promoter Mike M Observe a CCFLS session March 18 Mphwiri GVH, Mlolo TA FGD with Care Group Volunteers Interview with a family receiving safety net rations Mary Pat Interview with a HSA FGD with Mlolo ADC for WASH Amos Chitseko GVH, Mlolo TA Interview with a Care Group Promoter FGD with Care Group Volunteers Mary Pat Interview with a family receiving safety net rations FGD with Mathiya Water Point Committee Amos Interview with a VNCC Kachaso GVH, Chimombo TA FGD with VNCC members Mike M Visit to a Health Facility Ngabu GVH, Ngabu TA Interview with a Care Group Promoter Observation of a CCFLS session Mike M Interview with Care Group Cluster leaders PURPOSE 3 - UBALE Date Location Name of Respondent or Type of Focus Group Team Member(s) March 9 Blantyre Dorothy Ngirwa, UBALE Resilience Technical Integration Lead; Alinane Katenga-Kaunda, UBALE Senior Gender Adviser; Chimwemwe Limani, UBALE Community Capacity Strengthening and Governance Adviser; Lekeleni Maudzu, UBALE FFA Specialist Mike D March 10 Blantyre Moses, Getrude & Salani Njaya, P3 Staff, Save the Children Mike D & Charlotte Agnes Napwanga, District Community Development Officer, Blantyre District Mike D & Aaron Loveness Chikumba, Principal Nutrition & HIV/AIDS Officer & Charles Mphepo, Disstrict HR Manager, District Civl Protection Committee, Blantyre District Mike D & Charlotte March 11 Pensulo GVH Kuntaja TA VCPC Pensulo GVH - 7 women & 5 men Mike D & Charlotte VNRMC Pensulo GVH - 6 women & 2 men Mike D Observation VCPC Tree Nursery Mike D & Charlotte JMTR Methodology 5 June 2017 153 Charles, Resilience Field Facilitor covering Kuntaja TA Mike D Kuntaja TA Combined ACPC & ANRMC, Kuntaja TA, ACPC -3 women & 5 men, ANRMC - 1 women & 7 men Mike D & Charlotte March 12 Likomba GVH Machinjiri TA VCPC Likomba GVH - 2 women & 6 men Mike D Observation Riverbank Protection FFA Mike D, Killian & Marit Salani Njaya, FFA Assistant, Save the Children Mike D Undi GVH Lundu TA VCPC Undi GVH - 4 women & 6 men Mike D Observation Surface Water Source Mike D Observation VCPC Maize farm Mike D, Killian & Marit March 13 Nchalo Augustine Mphotonde, Phatso, Joseph & Gift Kasonda, CADECOM Implementation staff Mike D & Charlotte Chikwawa Spencer Kamalizeni, DCDO Mike D, Aaron, Kumbukani Raymond Chimsale, CADECOM Secretary Mike D March 14 Mkumaniza GVH Neverson Vintula, VDC Chairman Mike D FFA Recipients - 4 men and 3 women Mike D Mwanabonjo vu GVH VCPC - 5 women and 5 men Mike D & Charlotte March 15 Jana GVH Jana Youth Club - 6 girls & 6 boys Mike D VNRMC - 5 women & 6 men Mike D Gangu GVH Village Forestry Club - 6 women & 3 men Mike D March 16 Nsanje Steven Jumbe, Benjamin, Lloyd, Jonas, Emmanuel & Kris, CARE implementation staff Mike D & Charlotte Humphrey Magaliti, ADDRMO, Nsanje District Mike D Michael Chanza, District Logistics Officer & Martin Chiwanda, District Information Officer, members of the DCPC Mike D March 17 Bitilinyu GVH ACPC, Ndamera TA, 9 men & 1 woman Mike D VCPC, 10 men & 3 women Mike D Chibulu GVH GVH Headman and 6 Village Headman from the GVH, all men Mike D Eric Msambira, Community Development Assistant Mike D March 18 Mphwiri GVH VDC, 5 women & 9 men Mike D Chitseka GVH VCPC, 5 women & 5 men Mike D Observed Mbwazi River Bank Protection FFA Project Mike D CROSS-CUTTING THEMES Governance Date Location Name of Respondent or Type of Focus Group Team Member(s) March 3 Blantyre telephonic Chimwemwe Limbe, UBALE governance advisor Aaron C Andrew Mpesi, Malawian Farmer’s Union Aaron C March 9 Blantyre Story Workshop Aaron C JMTR Methodology 5 June 2017 154 Blantyre Times News Group Aaron C March 10 Limbe, Blantyre Save the Children – Capacity Strengthening Coordinator; GSFO Aaron C Blantyre District Community Development Officer Aaron C March 11 Mdala Kaphikamtama Ward Councilor Enoch Maxwell Mokhore Aaron C Kaphikamtama Watershed Groups Aaron C Mdala Households Aaron C March 12 Kumponda Kadikira VCPC Aaron C Kumponda VDC Aaron C Kumponda ADC Aaron C Kumponda GVH Aaron C Kadikira Leaders (ADC Chair, VDC Chair, GVH) Aaron C March 13 Chikwawa Anthony Mangani, CADECOM Capacity Strengthening Coordinator Aaron C Newton Munthali, Distric M&E Officer, DPD Aaron C Spencer Kamamlizeni, D/Director, District Community Development Office Aaron C Maseya ADC Aaron C Precious Mphwanthe, CADECOM Maseya GSFO Aaron C March 14 Maseya, Salumeji Salumeji Youth Voices Club Aaron C Salumeji VDC Aaron C March 15 Kandeyu Namila Namilia VCPC Aaron C Namila Youth Club Aaron C Namila VDC Aaron C Kandeyu VDC Aaron C Kandeyu GVH Aaron C March 16 Nsanje Mavuto Saulosi (CARE GSFO), Oliba Mwanza (CARE GSFO), Ivy Luka (CARE Capacity Strengthening Coordinator), Hardson Mphadso (GSFO) Aaron C Kennedy Adamson, District Forestry Officer (former DPD) Aaron C March 17 Chibule Bitilyenu Chibule VDC/Youth Group (combined) Aaron C Bitilyenu VDC Aaron C Bitilyenu GVH Aaron C Bitilyenu Ward Councilor Aaron C March 18 Fatima Chitseko Fatima ADC Aaron C Chitseko VDC Aaron C JMTR Methodology 5 June 2017 155 Gender Date Location Name of Respondent or Type of Focus Group Team Member(s) March 9 Blantyre PMU Dr Alinane Katenga Kaunda-Gender Senior Advisor-PMU-(female) Kumbukani March 10 Blantyre Save the Children Chipaso Nkhonjera-Gender Coordinator￾SAVE(male) Kumbukani Ruth Lemani-Gender Sustainability Field Advisor(female) Kumbukani Blantyre CDO Agnes Napwanga- District community development officer(female) Kumbukani, Mike D & Aaron Blantyre DC Valens Mhezu- District gender roles extension officer And also a Member of Gender network (male) Kumbukani March 11 Mdala GVH, Chigaru TA FGD with a gender chamions group, with 11 participants (6 men and 5 women) Kumbukani Kaphikamtam GVH, Chigaru TA FGD with a SILC Group with 8 particiants (all women) Marit, Kumbukani & Killian March 12 Kumponda GVH, Kapeni TA FGD with a gender champions group with 14 participants (8 men and 6 women) Kumbukani Mbanda GVH, Kunthembwe TA FGD with a gender champions group with 7 participants (3 men and 4 women) Kumbukan March 13 Chikwaw CADECO John Nyaika - Gender Coordinator CADECOM Linile Thupi & Kenneth Kandiero, Gender Kumbukani Sustainability Field Advisors Chikwawa District Spence Kamalizeni, Deputy DCDO Kumbukani March 14 Mwanabanjovu GVH, Ngowe TA FGD with gender champions group with 9 participants (4 men and 5 women) Kumbukani FGD with a VDCwith 8 participants (5 men and 3 women) March 15 Jana GVH, Makhuwira TA Interview with two gender champions (1 man and 1 woman) Kumbukani FGD with a VDCwith 5 participants (3 men and 2 women) March 16 Nsanje Jane Simwaka, gender Coordinator for CARE Kumbukani Efile Juwayeyi , Olipa Mwanza, Eneck Mphande & Lusejero Ngwira, Gender and Sustainability Field Officers, CARE Interview with Lucy Vumu, DCDO and Felix Chisale, DSWO March 17 Bitilinyu GVH, Ndamera TA FGD with a combned group of gender champions, DC and ADC, with 6 participants (3 men and 2 women March 18 Mphwiri GVH, Ndamera TA FGD wth a gender champins group wth 6 participants (3 men and 3 women) Kumbukani JMTR Methodology 5 June 2017 156 Chitseko GVH, Mlolo TA FGD with a VDC with 7 participants (4 men and 3 women) IMPLEMENTATION SYSTEMS - UBALE Management, Partnership and Finance Date Location Name of Respondent or Type of Focus Group Team Member(s) March 2 Lilongwe Debbie Shomberg, Country Representatives, CRS Mike D Rose Tchwenko, ACD Program, CARE Mike D Betty Chinyamunyamu, Deputy CEO & Henry Kalomba, Business Development Manager, NASFAM Mike D Tina Yu, Country Director & Stanley Phiri, Child Protection & Child Poverty Director, Save the Children Mike D March 9 Blantyre Philippa Sackett, ACD Programmes, GOAL & Hetherwick Mandere, Nsanje Regional Coordinator, GOAL Mike D Dane Fredenburg, CoP, UBALE Mike D Elton Mgalamadzi, WFP Regional Coordinator Mike D & Aaron March 13 Nchalo Bernardoh Gideon, Program Manager, CADECOM Mike D & Charlotte Chikwawa Raymond Chimsale, CADECOM Secretary Mike D March 23 Blantyre Dane & Angie, UBALE Senior Management Mike D James Akai, Value Chain Specialist, CLUSA Mike D April 4 By email Received written response to Topical outline questions from Wales Magumbi, Save the Children Mike D Knowledge Management Date Location Name of Respondent or Type of Focus Group Team Member(s) March 9 CRS Blantyre Jenny Haddle, MEAL Coordinator UBALE; John Kamanga, MEAL Officer UBALE; K Mutiro Tumbikani Msiska, C-MIS Data Specialist; Chimwemwe, PPM&E Officer March 10 CARE Blantyre Tenthema Nyondo, MEAL Coordinator K Mutiro FGD, Data entry clerks March 13 CADECOM Chikwawa MEAL Coordinator, Martha Kabanga & data entry team K Mutiro Dickson Chaima – Commodity Accountant Claudia Mwale – Logistics Officer Colin Phiri – Warehouse Officer March 14 CRS Blantyre Monalisa, Knowledge management & Learning Advisor K Mutiro JMTR Methodology 5 June 2017 157 CRS Blantyre Emmanuel Kasiya, Commodities & Logistics Coordinator March 16 CARE Nsanje Francisco Zuze, Project Manager, representing M&E Manager K Mutiro Focus Group Discussion with MEAL Assistants - Samuel, Chimwemwe, Dan Masese, Lucy, Musa, Robert, Gerad Menard Kika, Office and Finance Manager Warehouse Supervisor & Assistant March 23 CRS Blantyre Dr Alinane, Gender K Mutiro March 11-12 Blantyre Mdala GVH/Chigaru TA Interview with GVH Headman FGD with Youth Club K Mutiro Blantyre Kaphikamtama GVH/Chigaru TA SILC Group K Mutiro & Marit Blantyre Likomba GVH/Machinjiri TA Observation of river bank protection project Interview with TA Headman/ADC Chairperson K Mutiro Blantyre Undi GVH/Lundu TA FGD with FFA Ration recipients March 14-15 Chikwawa Nkwana GVH/Maseya TA FGD DiNER Fair Beneficiaries K Mutiro Chikwawa Salumeji GVH/Katunga TA FGD Boys & Girls Youth Clubs Chikwawa Kandeu GVH/Kasisi TA FFA Beneficiaries Chikwawa Namila GVH/Mlilima TA FGD self-targeting group - Marketing March 17- 18 Nsanje Magulugulu GVH/Malemia TA FGD FFA beneficiaries Interview with Councilor/Traditional Leader Nsanje Dumba K Mutiro GVH/Tengani TA VSL FGD FGD Youth group Nsanje Kachaso GVH/Chimombo TA VDC FGD FGD youth club Nsanje Ngabu GVH/Ngabu TA Interview with Councilor/Village Lead Marketing Club FGD JMTR Methodology 5 June 2017 158 PURPOSE 1 - NJIRA Date Location Name of Respondent or Type of Focus Group Team Member(s) March 24 Zomba PCI P1 PMU team: Isaac Munttau, Jeffrey Chisale, Angnes Mangoche, Jennie Vader, Felix Iere (sp?), Dyton Siyeni (Agribusiness Advisor) Marit & Rupert Matthew Jere (Agricane), Spencer Ng’oma (Total Land Care), Daniel van Vugt, Program Manager Malawi Improved Seed Systems and Technologies (MISST) – OFSP Component (CIP) Marit & Rupert Allan Kaliwo, Mphatso Gama, Goodluck Sibande, ADD of Eastern Region officers Marit & Rupert March 25 Blantyre Bhavin Patel, Export Trading Group (agro-dealer) Marit & Rupert Felix Makohdi, veterinary college (trainer) Marit & Rupert Zomba PCI P1 team: Isaac Munttau, Jeffrey Chisale, Agnes Mangoche, Jennie Vader, Felix Iere (sp?), Dyton Siyeni Marit & Rupert Ruth Ngulu, PCI Env (from Sun Mountain) Marit March 26 Machinga Emmanual International P1 team Machinga District: Benjamin Nyangulu, Moyenda Kaliatt (sp?), Chioniso Moyo, Thomas Mkovole, Alfred Chiphaka, Oliver Sweet Marit & Rupert Machinga Mlimbula GVH Irrigation site visit & committee (3f:2m) Marit WE/VSL (8f:1m) Marit Makata GVH/Kaploma TA FGD Producer Group with marketing club members: Chair and Vice Chair, Secretary and Vice Secretary, Treasurer, Lead Farmer and 6 members. 5m:10f. 8 of these are Marketing Club members Rupert March 27 Machinga Ng’andu GVH Irrigation site visit & committee, inc. Chief, 4 officers, 9 committee members (5m:9f) Marit Lead Farmers (3f:4m) Marit Machinga Mbwabwa GVH/Nyambi TA FGD with WE/VSL: Vice Chair, Secretary, Treasurer. 10f:0m Visit watershed work with members of watershed development committee: Chair, Vice Chair, Secretary, Vice Secretary and Treasurer and 9 members FGD Production group with marketing Rupert & Amy Machinga Chidothi GVH CAHW (1f:3m) Marit Chikojo GVH/Nyambi TA Producer Group (4 members), Irrigation (10 members) and Market Club (2 members), Contact Farmer, Lead Farmer (Watershed), Civil Protection Committee member Rupert March 28 Machinga Sosten Zulu, Yohane Maseleo, Assistant DADO & Land Resource Officer Marit & Rupert Machinga Mbosongwe Lead Farmers (3f:3m) Marit Sara Efram, AEDO Marit JMTR Methodology 5 June 2017 159 GVH Machinga Nampeya GVH/Chiweo TA FDG with Producer Group Contact Farmers: Anodi Mteuka, Bitoni Chichere, Lliyani Stande, Malita Kalulu, Masautso Mailosi, Daufutala Maili, Luth Musa. Magret Fazenda Interviews with Robert M’kandawire, Lead Farmer Estery Khana and Sam Chiwayu Banda, Agribusiness Community Agents Rupert March 29 Balaka PCI P1 Team Balaka District: Arnold Nthala, Wiscot Sapply, Emmanuel Khunza (sp?), Lefani Kachisuzi, Stanley Imedi Marit & Rupert Balaka Hindahinda GVH Lead Farmers (4f:4m) Marit CAHW & agribusiness volunteer (2m) Marit Dynamic team frontline staff (1f:7m) Marit Balaka Mbera GVH/Kachenga TA FGD with Marketing Club: Egily Alubi (Chair), Wasili Malemu (Vice Chair), Monica Kamoto (Sectretary), Loveness Abraham (Vice Secretary) Dorothy Ganet (Treasurer) and 8 members. 3m:10f Interview with Amina Swalattiand Waina Mussa, VSL Community Agents; Akimu Skauli, Lead Farmer and Dudney Banda, Agribusiness Facilitator Rupert March 30 Balaka Blessings Banda, Managing Director Oprah Supplies, OPS (agro-dealer) Marit & Rupert Glyn Chiteta, Ekron Kamanga, Herbert Mankhwazi, DADO Livestock, Irrigation and Extension Officers Marit & Rupert Balaka Toleza GVH Livestock beneficiaries & CAWH (7f:2m) Marit Watershed comm & site visit (3f: 6m) Marit Mgwira GVH/Sawali TA FGD with Chimwemwe Producer and Market Group: Mtendere Chazemba (Chair), Loveness Kamwendo (Vice Chair), Laitoni Loti (Secretary), Eliza Asima (Vive Secretary), Esme Daud (Treasurer), Mercy Mowere (Contact Farmer), Elson Alubi (Lead Farmer) and 8 members. Rupert April 1 Balaka Kuntyani GVH Producer Group (5f:5m) Marit Seed multiplication group (5f:3m) Marit Marketing Group (8f:7m) Marit Marketing group sub-unit, WALA carry-on’s (~8ppl) Marit Balaka Mchenga GVH/Nsalama TA FGD Production Group: Esime Ntanga (Chair), Estar Hannireck (Secretary), Egire John (Treasurer), Partiso Charamanda (Lead Farmer), and Ireen Brahim, Patric Chikarema, Suzen Kadango, Christina Ben (COnatct Farmers) and Grace Kondwero, Monica Chambo, Saires Chiwaro, Rith Chita (members) FGD Marketing club: Seveliano Daniel (Chair), Obvias Chitando (Secretary), Thokozani Chilambe (Lead Farmer), Eshere Mlaya, Edina Jabu, Egilly Billat, Jonas Daila, Feston Chimera (members) Rupert JMTR Methodology 5 June 2017 160 FGD GVH Marketing Committee: Agnes Ellias (Chairlady), Ester Kamwendo (Vice Chair), Elline Chalosi (Secretary), Mariam Kazembe (Treasurer), Fanuel Chimusendo (Lead Farmer) and 3 members and 4 Contact Farmers Balaka Ntalika GVH/Nsalama TA FGD Water Users' Association: Agines Kamwendo (Chair), Simion Maononga (Vice Chair), Linda Tyson (Secretary), Tchalos Chambo (Vice Secretary), Emmily Manyamba (Treasurer), Frank Chimphepo (VDC) and 12 members Association composition: 9m:9f. Rupert PURPOSE 2 - NJIRA Date Location Name of Respondent or Type of Focus Group Team Member(s) March 24 Zomba Meeting with P2 staff Lucy ,Mike M & Mary Pat WASH Advisor, PCI, Morris Tengatenga Lucy DCOP and Admin and Logistcs Officer for the PMU Mary Pat March 25 Zomba WASH Advisor, PCI, Morris Tengatenga Lucy March 26 Makata GVH/Kapoloma TA Meeting with P2 Coordinators Lucy ,Mike M & Mary Pat Interview with two HSAs Mike M Timothy Chikalipo, WASH Coordinator, Emmanuel International Lucy Care group meeting at Makata village Lucy & Mike M Water point visit and water point committee meeting @ Makata village Lucy Household visits for inspection of sanitary and handwashing facilities Mlimbula GVH FGD with a Father Groups Mary Pat FGD with Lead Mothers March 27 Ng'andu GVH FGD with 6 CLAn members Observe Care group meeting Mike M Interview tih the promoter Chidothi GVH Visit to a health clinic Mike M Interview with the hed of the cinic and HAS supervisor Interviews with HSAs Mbwabwa GVH/Nyambi TA FGD with a Care group Interview with a HAS and Health promoter Mary Pat together FGD with Mbwambwa water point committee and water point visit Lucy JMTR Methodology 5 June 2017 161 Interview with Area mechanic ,Gibson Mlamba from TA Nyambi Chikojo GVH/Nyambi TA Visit to a health clinic Mary Pat Interview with a Community-Based Distribution Agent FGD with a group of lead mothers Interview with Health Promoters and HSA Lucy Focus group with water point committee Household inspections for sanitary and handwashing facilities March 28 Machinga Stakeholder meeting Meeting with DWO Mr. Steve Major Mike M, Mary Pat & Lucy Meeting with DEHO, Matthews Kalaya Meeting WASH Coordinator, Gift Mkhala Mbosongwe GVH/Kawinga TA Focus group with natural leaders Household visits, inspection of sanitary and Lucy handwashing facilities FGD with Care Group beneficiaries Mike M Observe lead mother performing a household visit Visit to a health clinic Mary Pat Interview with an HSA March 29 PCI staff meeting Meeting with P2 Coordinators- Josephine Kalepa and James Nyirenda Mike M, Lucy, Mary Pat Meeting with James Nyirenda, WASH Coordinator Lucy Hinda Hinda GVH/Amidu TA FGD with Care Group beneficiaries Mike M & Lucy Observe a CCFLS or GMP session Mike M Visit households benefitting from the project Focus group with natural leaders Lucy Household visits, inspection of sanitary and handwashing facilities Lucy Mbera GVH FGD with lead mothers Mary Pat Interview with an HAS and HP 30 March Stakeholders meeting Meeting with DEHO, Thomas Muchipa Mike M, Mary Pat & Lucy Meeting with WASH Coordinator, Charles Mkwate Lucy Mgwira GVH, Sawali TA Visit to a health facility Mary Pat FGD with a youth group Toleza GVH/Sawali TA Interview with two Health Promoters Interview with a P2 Facilitator Mike M Visit to a health clinic Interview with WASH Facilitator, Debora Afera, Lucy Interview with Water Monitoring Assistant, Dickson Daudi Focus group discussion with Water point committee 01 April Kuntyani GVH, FGD with a father groups Mary Pat JMTR Methodology 5 June 2017 162 Kalembo TA FGD with lead mothers Visit to a health clinic Mchenga GVH/Nsalama TA Focus group discussion with WPC Mike M & Lucy Interview with a P2 Nutrition Facilitator Mike M Household visits Interview with Health Promoter, Liness Gawanyumbo Lucy Focus group discussion with lead mothers Water Monitoring Assistant, Oliver Nawanga meeting Water point visit and Household inspection of sanitary and handwashing facilities Ntalika GVH/NSalama TA Observe GMP/CCFLS session FGD with Care Group beneficiaries Mike M Husehold visits FGD with VDC Lucy Water point visit PURPOSE 3 - NJIRA Date Location Name of Respondent or Type of Focus Group Team Member(s) March 24 Zomba Burnnet Khulumbo, Simon Tembo, Venansio Chome, PCI Implementation Staff Mike D Shepherd Jere, ADDRMO, Machinga District Mike D March 25 Blantyre Continued interview with Burnnet Mike D March 26 Ntaja (Machinga) Jam Masebo, DRM Coordinator, EI Mike D Mlimbula GVH VCPC 6 men & 0 women Mike D FFA food recipients, 4 women & 1 man Mike D Observed Road Rehabilitation FFA site Mike D March 27 Ng'andu GVH ACPC, Ngokwe TA 4 men & 1 woman Mike D FFA Management Committee 9 women & 5 men Mike D Observed irrigation protection FFA site Mike D Chidothi GVH VCPC 8 men & 1 woman Mike D March 28 Machinga Boma Mbawaka Makhwawa, Director of Planning & Development, Machinga District Mike D Chipolonga GVH VDC 2 women & 5 men Mike D Observed Afforestation FFA Site with VNRMC representatives 5 men & 3 women Mike D March 29 Balaka Ken Somanje, DRM Coordinator, PCI Mike D Mbera GVH Stedson, DRM Field Assistant, PCI Mike D VCPC 2 women & 2 men Mike D March 30 Balaka Violet, District Land Administration Officer Mike D JMTR Methodology 5 June 2017 163 Mgwira GVH VDC 6 women & 4 men Mike D Observed Mgwira Dam FFA with FFA Management committee, 4 men & 2 women Mike D April 1 Chatama GVH Observed stream bed dredging FFA site Mike D, Marit & Mary Pat Kuntiyani GVH Observed River gauge site with James Changi, River Gauge Reader Mike D VCPC 5 men & 3 women Mike D Non-participating households dependent on ganyu9 women & 2 men Mike D CROSS-CUTTING THEMES - NJIRA Gender Persons Interviewed/Sites Visited in the Malawi JMTR Date Location Name of Respondent or Type of Focus Group Team members 24/3/17 Zomba-PMU Meeting w/P1 PMU team members  Agnes Mangoche, Njira WE/VSL Coordinator  Isaac Munthali, Agriculture Advisor  Jefrey Chisale, BCC Specialist  Felix Jere, Livestock Advisor  Dyton Siyeni, Agribusiness Advisor  Jennie Vader, PCI Associate Technical Advisor for WE, Africa Region Sunseri, A. Best, R. Wilkerson, M. Zomba-PMU Interview w/ WE VSL team  Agnes Mangoche, Njira WE/VSL Coordinator  Jennie Vader, PCI Associate Technical Advisor Sunseri, A. Zomba-PMU Interview w/ District Community Development Officer, Mr. Joseph Chipekwe Kumbukani, B. Sunseri, A. Zomba-PMU Interview w/ Njira BCC Specialist, Jefrey Chisal Sunseri, A. MACHINGA DISTRICT 25/3/17 Zomba-PMU Q&A meeting WE/ VSL team Agnes Mangoche, Njira WE/VSL Coordinator Jennie Vader, PCI Associate Technical Advisor for WE, Africa Region Rupert. B. Sunseri, A. 26/3/17 Ntaja, EI district office Leah Katuya, District Gender Coordinator, EI Kumbukani, B. Sunseri, A. Makata GVH / Kaploma TA FGD with Tadala WE group members  25 women, 3 men Kumbukani, B. Sunseri, A. FGD with couples and male champions  12 women, 10 men Kumbukani, B. Sunseri, A. 27/3/17 Mbwabwa GVH/ Nyambi TA FGD with WE/ VSL, jointly with P1 team member  10 women Sunseri, A. Best, R. JMTR Methodology 5 June 2017 164 FGD with couples and male champions  4 women, 4 men Sunseri, A. Chikojo GVH, Nyambi TA FGD with couples and male champions  5 couples  9 male champions (in addition to above) Sunseri, A. Ng'andu GVH/TA TA Ngokwe Meeting with Talandila 1, 18 women attended Kumbukani, B. Meeting with Ngokwe champions, 9men Kumbukani, B. Meeting with Ng’andu Ubwino center users (2f and 2m) Kumbukani, B. Chidothi GVH/Ngokwe TA Meeting with Gender champions (2m), WE group (13 f) but membership is 15 (14 f and 1m), Ubwino users (4f and 1m) Kumbukani, B. 28/3/17 Ntaja, EI District Office Interview with Community Development Officer, Mr. Lenex Nkhoma Kumbukani, B. Sunseri, A. Mbosongwe GVH, Kawinga TA FGD with Lead Farmers jointly w/ P1 team member Wilkerson, M. Sunseri, A. FGD with newly established Care Group  Women only Sunseri, A. Chipolonga GVH, TA Chikwewo Meeting with Champions and couples in Chipolonga (8 champions and 3 couples Meeting with Ubwino centre users (2f and 1m) Kumbukani,B BALAKA DISTRICT 29/3/17 Balaka, PCI District Office Carolyne Kamba, District Gender Coordinator, PCI Kumbukani, B. Sunseri, A. Mbera GVH / Kachenga TA Interview with 2 male gender champions  Mr. Alyson Kamako  Mike Sunseri, A. Interview with Village Head (man) Sunseri, A. Interview with representatives of Water Point Committee Sunseri, A. Hinda Hinda GVH, TA Amidu Meeting with champions in Hinda Hinda (2 m) Meeting with GVH Hinda Hinda and 2 Village heads Kumbukani, B. Meeting with Ubwino Center users (2 m & 3 f) 30/3/17 Balaka, PCI District Office Interview w/District Community Development Officer, Balaka - Mr. Moses Chirambo Kumbukani, B. Sunseri, A. Interview w/Community Development Assistant, Balaka - Ms. Ida Mwale Kumbukani, B. Sunseri, A. Toleza GVH/Sawali TA FGD with male and couples gender champions Sunseri, A. FGD with Producer Group & Marketing Club - 5 women, 5 men Sunseri, A. Mgwira GVH TA Sawali Meeting with Thundu WE VSL group (9 women) in Kumbukani,B Meeting with male champions (2 males) Kumbukani,B Meeting with Mgwira ubwino Centre (3 f and 1m) Kumbukani,B 1/4/17 Mchenga GVH / NSalama TA FGD with WE/ VSL Group - All women Sunseri, A. Interview with Head of GVH; Ms. Forrest (woman) Sunseri, A. Interview with male champion (one individual) JMTR Methodology 5 June 2017 165 Ntalika GVH/ NSalama TA FGD with WE/ VSL Group - 12 women, 1 man Sunseri, A. Interview with male gender champion, Samson Chicalogwe Sunseri, A. Interview with WE/VSL Community Agent, Patricia Sunseri, A. Interview with head of GVH (male) Sunseri, A. Kuntiyani GVH TA Kalembo Interview with VDC (5 men and 4 women) Kumbukani,B Interview with male champions (2 men) Kumbukani,B Interview with Ubwino centre users (3men and 4 women) Kumbukani,B Interview with GVH Kuntiyani, Kumbukani,B IMPLEMENTATION SYSTEMS - NJIRA Management, Partnership & Finance Date Location Name of Respondent or Type of Focus Group Team Member(s) March 24 Zomba Paul Jones & Helen Jones, EI Country Office Mike D March 25 Blantyre Michael Ghebrab, CoP Njira Mike D Kurt Henne, PCI Country Director Mike D March 28 Ntaja Symon Maseko, PM, EI Machinga Mike D March 30 Balaka Venancio Chome, PM, PCI Mike D April 4 By Email Received written response to topical outline questions from Trent from Total Land Care Mike D May 17 By Email Received written response to topical outline questions from Bouke from Agricane Mike D Knowledge Management Date Location Name of Respondent or Type of Focus Group Team Member(s) March 24 Zomba Daniel Tsegaye Teshome, M&E Manager; Martin Kachingwe, IT K Mutiro March 24 Zomba Eric Johnson Lespin, Independent Consultant hired to support M&E K Mutiro March 25 Blantyre Nasmul, IT & Commodities Manager; Daniel Tsegaye Teshome, M&E Manager K Mutiro March 26 Machinga, EI Offices FGD; Thomas Sambiri, M&E Manager and team ( Chisomo Kumbulani, Desmond Kachepa, Sakina Mumba, Yammie White, Chirwa, Priscila Taulo ) K Mutiro March 26 Machinga. Mlimbula GVH/Nkoola TA FGD Livestock beneficiaries/FFA Beneficiaries + Village Head K Mutiro March Machinga. FGD Seed Multiplication Group/Marketing Groups + K Mutiro JMTR Methodology 5 June 2017 166 27 Mbwambwa GVH/Nyambi TA Village Leader Machinga. Chikojo GVH/Nyambi TA FGD VSL Group/Irrigation Beneficiaries + Village Leader K Mutiro March 28 Machinga. Mbosongwe GVH/Kawinga TA FGD Lead Farmers + Village Leader K Mutiro March 29 Balaka M&E Manager, IT Officer & team of data entry clerks K Mutiro March 29 Mbera GVH/Kachinga TA FGD Seed Fairs Beneficiaries; FGD Stove beneficiaries Interview with Village Head K Mutiro March 30 Toleza GVH/Sawali TA FGD Seed multiplication group; FFA beneficiaries Interview with the GVH K Mutiro March 30 Balaka David Mbvundula, Distribution & Logistics Coordinator K Mutiro March 31 Machinga Angela, Knowledge Management Coordinator K Mutiro April 1 Mchenga GVH/Nsalama TA FGD with WE/VSL Group; FGD Lead farmers Interview with Village Head K Mutiro Ntalika GVH/NSalama TA FGD Pigeon beneficiaries FGD Irrigation beneficiaries (Join Rupert for visit to the irrigation) K Mutiro & Rupert JMTR Methodology 5 June 2017 167 FINAL WORKSHOP PLANS FOR THE VERIFICATION WORKSHOPS JMTR Methodology 5 June 2017 168 Malawi Joint Mid-Term Review (JMTR) UBALE VERIFICATION WORKSHOP PLAN Tuesday, March 21 & Wednesday, March 22 (8:30 am to 5:00 pm) WORKSHOP PURPOSE Based on information obtained from field work and other interviews undertaken thus far, the JMTR Team has started formulating observations and recommendations for the remaining life of the UBALE Project around what is working well and what can be improved in the project. These will be shared in the Verification Workshop with project implementation staff and discussed further to ensure that the observations reflect reality and the recommendations are appropriate and feasible. WORKSHOP OUTPUTS Two major outputs are targeted for the workshop. These are (1) agreement on the key observations assembled so far from the JMTR and (2) refinement of recommendations being considered for the remaining life of the project. WORKSHOP FLOW Day 1, Tuesday, March 21 SESSION 1-1: Introductions, Workshop Purpose, Objectives and Plan, Logistics (1/2 hour, Mike) Participants will introduce themselves, and the workshop purpose, objectives, and agenda will be presented, along with details on logistics for the workshop. SESSION 1-2: History & Operating Context (1/2 hour, Mike) Key events in the history of the project will be presented and discussed for clarity and completeness, ensuring that major events in the life of the project that have affected implementation have been captured. Observations on contextual factors that have influenced the project will also be presented, along with observations on how the project adapted to these. Finally, emerging contextual issues will be discussed, along with possible actions that the project could take to be positioned to adapt effectively to these. In preparation for the next session, a brief overview of the project strategy will also be presented. SESSION 1-3: Purpose 1 - Income (2 hours, Rupert and Marit) The JMTR Team members responsible for investigation the Income Component of the project will present an overview of their key observations and preliminary recommendations. SESSION 1-4: Purpose 2 - MCHN (2 hours, Mike M, Mary Pat, & Amos) The JMTR Team members responsible for investigation the Maternal and Child Health and Nutrition Component of the project will present an overview of their key observations and preliminary recommendations. JMTR Methodology 5 June 2017 169 SESSION 1-5: Purpose 3 - Community Capacity Building (2 hours, Mike D., Aaron & Kumbukani) The JMTR Team members responsible for investigation the Community Empowerment Component of the project, including DRM/FFA (Mike D), governance (Aaron) and gender (Kumbukani) will present an overview of their key observations and preliminary recommendations. Day 2, Wednesday, March 22 SESSION 2-1: Cross-Cutting Themes – Gender Equity, Governance, Environment & Youth Targeting (2 hours, Kumbukani, Aaron, Charlotte & Killian) The JMTR Team members responsible for investigation the cross-cutting themes in the project will present an overview of their key observations and preliminary recommendations related to implementing Gender Equity (Kumbukani), Governance (Aaron), Environment (Charlotte) & Youth Targeting (Killian) as cross-cutting themes for the UBALE Project. SESSION 2-2: Overall Program Design - Theory of Change, Targeting, & Project Impact (2 hours, Killian & Mike D) In the previous sessions, an overall picture for the project has been presented in terms of key observations on outputs and resulting outcomes as well as activities associated with the cross￾cutting themes. In this session, the evaluation team will present (1) a summary of observations on targeting in terms of who is benefitting from the program (Killian), (2) an overall assessment on the project-level impact collectively across the Purposes toward achieving the project's Goal (Mike D), and (3) the implications for the project's Theory of Change (Killian). SESSION 2-3: Implementation Systems (2 hours, Mike D, Killian, Rupert, Mike M and Mary Pat) The JMTR team investigated the project's implementation systems, including the management systems, partner relations, collaborative learning and action systems (including monitoring and evaluation), program integration and complementarity, financial resource management, human resource management, and material resource management. The JMTR Team members responsible for these investigations will present an overview of the key observations and recommendations being considered for addressing challenges in these systems that are affecting implementation or for capitalizing on strengths that have been observed. SESSION 2-4: Next Steps (1/2 hour, Mike D). Over the course of the workshop, some topics may have emerged from the discussions that represent areas that need to be further investigated by the JMTR team. In this final session of the day, participants will discuss these topics and agree on how best to obtain information to resolve the questions that have arisen. JMTR Methodology 5 June 2017 170 Malawi Joint Mid-Term Review (JMTR) NJIRA VERIFICATION WORKSHOP PLAN Monday, April 3 (1:30 to 5:30) & Tuesday, April 4 (8:00 to 5:30) WORKSHOP PURPOSE Based on information obtained from field work and other interviews undertaken thus far, the JMTR Team has started formulating observations and recommendations for the remaining life of the Njira Project around what is working well and what can be improved in the project. These will be shared in the Verification Workshop with project implementation staff and discussed further to ensure that the observations reflect reality and the recommendations are appropriate and feasible. WORKSHOP OUTPUTS Two major outputs are targeted for the workshop. These are (1) agreement on the key observations assembled so far from the JMTR and (2) refinement of recommendations being considered for the remaining life of the project. WORKSHOP FLOW Day 1, Monday, April 3 SESSION 1-1: Introductions, Workshop Purpose, Objectives and Plan, Logistics (1/2 hour, Mike D) Participants will introduce themselves, and the workshop purpose, objectives, and flow will be presented, along with details on logistics for the workshop. Participants will be asked to write a short answer to the following question on a small slip of paper: If you had the power to change anything in the Njira Project to enable it to have greater impact or to use project resources more efficiently, what one thing would you change? Participants will not present their answers. Workshop facilitators will collect the answers for review by the JMTR Team later. SESSION 1-2: History & Operating Context (15 minutes, Mike D) Key events in the history of the project will be presented and discussed for clarity and completeness, ensuring that major events in the life of the project that have affected implementation have been captured. Observations on contextual factors that have influenced the project will also be presented, along with observations on how the project adapted to these. Finally, emerging contextual issues will be discussed, along with possible actions that the project could take to be positioned to adapt effectively to these. SESSION 1-3: Purpose 1 - Income (1 1/2 hours, Rupert and Marit) The JMTR Team members responsible for investigating the Income Component of the project will present an overview of their key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. JMTR Methodology 5 June 2017 171 SESSION 2-2: Purpose 2 - MCHN (1 1/2 hours Mike M, Mary Pat, & Lucy) The JMTR Team members responsible for investigating the Maternal and Child Health and Nutrition Component of the project will present an overview of their key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. Day 2, Tuesday, April 22 SESSION 2-1: Recap (5 minutes, Mike) A brief recap will be provided of the work from the previous day and the stage will be set for Day 2. SESSION 2-3: Purpose 3 - Capacities to Prepare for, Manage and Respond to Shocks (1 1/2 hours, Mike D & Kumbukani for Ubwino Centers) The JMTR Team members responsible for investigating the Disaster Risk Management and Response Component of the project will present an overview of their key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-4: Cross-Cutting Themes – Gender Equity, Governance, & Environment (1 1/2 hours, Kumbukani/Amy, Rupert & Marit) The JMTR Team members responsible for investigating the cross-cutting themes in the project will present an overview of their key observations and preliminary recommendations related to implementing Gender Equity (Kumbukani & Amy), Governance (Rupert), & Environment (Marit) as cross-cutting themes for the Njira Project. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to any of the topics presented. SESSION 2-5: Program Design & Measurement - Theory of Change, Targeting, & MEL (1 1/2 hours, Killian) In this session, observations will be presented and discussed on (1) the Project's Theory of Change, (2) targeting in terms of who is benefitting from the project, and (3) how the project's MEL system is being used to inform project decision-makers and assess project impact. Priority recommendations for the remaining life of the project for reviewing/using the ToC, improving the targeting systems, and improving the MEL systems will be presented and discussed. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-6: Implementation Systems (2 hours, Mike D for Management/Partnership/Finance, Rupert for Integration & Complementarity, Killian for Commodities, Mike M for HR management and Mary Pat for Admin Support) The JMTR team investigated the project's implementation systems, including the management systems, partner relations, program integration and complementarity, financial resource management, commodity management, human resource management, and administrative support JMTR Methodology 5 June 2017 172 systems. An overview of the key observations and recommendations being considered for addressing challenges in these systems will be presented and discussed. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to any of the topics presented. SESSION 2-7: Project Impact Summary (1/2 hour, Mike D) While it is too early to assess project impact, the JMTR Team would like to offer its impressions on how well the project seems to be moving toward achieving the intended effects and impact under each of the three Purposes and at the Goal level. SESSION 2-8: Next Steps (1/2 hour, Mike D) Over the course of the workshop, some topics may have emerged from the discussions that represent areas that need to be further investigated by the JMTR team. In this final session of the day, participants will discuss these topics and agree on how best to obtain information to resolve the questions that have arisen. In addition, participants will be asked to write on a small slip of paper short answers to the following questions and will be asked to leave their answers with a JMTR team member before they leave the workshop: In your opinion, what recommendations are most important? Why? What recommendations are least important? Why? What additional recommendations would you suggest be considered by the JMTR team? JMTR Methodology 5 June 2017 173 JMTR PROCESS REVIEW JMTR Methodology 5 June 2017 174 Malawi JMTR PROCESS REVIEW The following summarizes the main observations from JMTR team members, UBALE implementing staff and Njira Implementing staff on the JMTR process to inform the planning and organization of future reviews. Initial Planning, Pre-Arrival in Country It was very useful having dates fixed and unchanging so that team members could plan accordingly, although last minute changes by the USAID Mission in Malawi caused cancellation of a critical meeting that had been on the schedule for weeks. Future reviews should try to set the schedule as early as possible and keep dates fixed as much as possible. Key stakeholders also need to review and internalize the schedule, so that if an obvious conflict arises, it is brought to the attention of the review organizers early on and not at the last minute. The primary organizer for the Malawi JMTR had to scramble around at the last minute to find team members to fill open slots on the review team. This was primarily related to the time commitment involved, which is addressed in another selection below. The late recruitment of team members, however, meant that introduction and orientation conference calls during the initial planning period did not include all or even most team members, reducing the value of the calls. In addition, time differences precluded having field-based team members participate in calls, further reducing their effectiveness. The lesson is that, while the conference calls are necessary, they should not be expected to fulfill much of the function of orienting JMTR team members. JMTR SOW & Protocol It was not entirely clear as to why this review was called a "joint" review. For some, this was interpreted to mean that the review was covering two projects and cross-project learning was one of the objectives, although there was no objective stating this in the SOW. Another interpretation was that the exercise was joint because representatives of USAID and the implementing agencies were on the review team. In any case, future reviews should provide more clarity on this for "joint" reviews. The JMTR protocol was very useful as a guide on the big picture, for defining expectations and for providing operational plans for the exercise. There was some perceived redundancy in terms of ideas around what the JMTR’s purpose and impact of the review process/products would be. This was mainly a function of the protocol having to build on an original SOW that was quite detailed. Future reviews should consider developing a less detailed SOW that is used primarily to recruit team members and a team leader. Once the team leader has been identified, a detailed protocol should be developed by the team leader working as much as possible with other team members, along the lines of that used by the Malawi JMTR. Background Documentation The Malawi JMTR tried to use both DropBox and Google Docs for making background documentation available. In the end, the team found the DropBox platform more useful since there are limitations on access to Google Docs for non-USG team members. For future reviews, Drop Box should be the preferred repository, and all team members should have an email address capable of accessing Drop Box. JMTR Methodology 5 June 2017 175 The specification of a list of documents as required reading for all JMTR team members and a second list of additional resources for background reading in the JMTR Protocol was useful, and most of the key documents in the required reading were useful. In the future, background documentations should include more current human resources information such as current organizational charts, and having some budgeting information would also be useful. In the case of the Malawi JMTR, the available background information did not provide enough explanation for the changes that occurred in the design at the start of the UBALE Project. The project document (proposal) described a strategy based on a CRS Theory of Change, but the M&E documents coming out the inception M&E workshop resulted in a significantly different Theory of Change and results framework, with little explanation as to why. It's not clear, however, whether there is a background document available that would explain this. IN any case, if not a background document, some background explanation, especially from FFP's perspective, would have been useful. The Drop Box used by the Malawi JMTR could have been better organized with directories, and the team leader (or someone else who has been designated) should spend more time ensuring that the Drop Box is organized. A DUMP sub-directory should also be created under each directory for persons placing documents in the Drop Box to move older versions of documents that they have replaced so that there is no confusion on which document is current. Each directory should contain only current documents outside of this DUMP sub-directory. Team Orientation In-Country Some team members in the Malawi JMTR commented that they felt that there should have been more team building exercises, even if just done virtually, but the problem was more not having all team members available from the start of the process. Some basic orientation to the exercise did occur after the team assembled in-country and these were appreciated by those who were present to attend. Those members who arrived late missed these exercises/discussions. Future JMTRs should make every effort to have all team members assemble together at the same time in-country at the beginning of the exercise. The approach of having JMTR Team Members NOT identify themselves by their home organization was effective at reinforcing the purpose of the review and reassuring implementing agencies that the JMTR was producing products for the projects. This facilitated better information gathering, especially on project problems and challenges. There was a bit of confusion in the Malawi JMTR as to who was responsible for providing a security briefing to the JMTR team after they had arrived in-country. In future JMTR's, this responsibility should be clearly defined. Assignment of Team Member Responsibilities The willingness and diligence of JMTR team members to step up and take on additional responsibilities related to cross-cutting themes and implementing systems was appreciated. It would be good if these assignments can, if possible, be delegated earlier on in the process. Observers There was a strong consensus that observers do not add value to the review process, complicate the logistics, and their presence raise questions in the communities and with project staff, e.g., "who are these people and why are they here?". To the credit of the observers who joined the Malawi JMTR, they did very well at following the ground rules and organizing their own logistics. Nevertheless, the value of having observers just does not justify their participation, relative to disruptions at the community level during interviews and focus group discussions. JMTR Methodology 5 June 2017 176 Project Orientations, PMU and District-Level The project orientations at the PMU level were useful, although they could be improved by having rotating responsibilities for presentations so more project team members can be heard. It would be also be useful for project technical staff to have a chance to review the JMTR topical outlines in order to provide feedback on topics that may be included but are irrelevant and on topics that need to be added. Sometime before the project orientation meeting, draft topical outlines should be sent to the project management teams for review, and time should be allocated in the project orientation meeting to discuss questions, comments and suggestions on these. At the district-level, the format used in the last district by the Malawi JMTR , with introductions, stakeholder meeting schedule review, interviews with implementing staff by sector, and a one hour plenary presentation on unique features and challenges for the project in the district, was most useful. The least useful presentations were those that focused on quantitative targets or other rigid metrics of success or that listed all the things that the project was doing across all the purposes. The JMTR team also should be coached to avoid asking too detailed questions in the plenary that are best discussed with sectoral staff individually. Site Selection for Field Visits For the most part, the selection of sites for field visits produced a good balance in the selection of sites according to the criteria that had been defined. There was not enough time, however, to verify that the selected sites captured all of the major interventions as well as some of the unique interventions in the district. It seemed to take a long time for the projects to classify GVHs according to the criteria; and given the schedule for the JMTR, there was relatively little time for the JMTR team leader to sit down with project management teams to select sites. As a result, sites were selected only a few days before the JMTR was expected to visit them, and the schedules had to be revised numerous times as JMTR team members reviewed them and requested changes or additional interviews/FGDs. For future reviews, the site classification list needs to be made available earlier, and more time needs to be built into the schedule to allow the team leader/project management teams to select sites well before the JMTR team is supposed to begin field visits and to verify with project implementing staff that key project activities are present among the sites selected. Stakeholder Interviews, at the National, Project, and District-Levels The Malawi JMTR did not have enough time scheduled for interviews with stakeholders in Lilongwe at the start of the exercise. Future reviews should build more time for these interviews, and more team members (only the team leader arrived early enough to schedule interviews in Liongwe) should plan to hold these interviews. In general, JMTR team members need to use the protocol and their background reading more effectively to identify the stakeholder interviews well before they need to be scheduled. At the District level, the Malawi JMTR was unable to meet with many government representatives since they had only been informed a few days before of the impending visit of the team, after they had already made other commitments. An additional one or two days needs to be built into the initial in￾country planning period for the JMTR team to identify and propose stakeholder meetings at the district levels. The stakeholder interviews should also include the entire array of stakeholders, not just government and and implementing partners, but also the private sector. Some stakeholder interviews should actually be held AFTER the team has conducted its field work. For example, a WFP representative was available at the beginning of the Malawi JMTR process before the team had seen any field activities. This interview would have been much more useful at the end of the field work after the team had a better understanding of the relationship between the projects and WFP. JMTR Methodology 5 June 2017 177 District-Level Data Collection Schedule The district schedule used in the last two districts during the Malawi JMTR was most useful with the sequence of project orientation/staff interviews, field visits, stakeholder meetings and more field visits. Having the chance to see some field activities before interviewing stakeholders made these interviews more productive. The district schedule needs to be extended by a day, however, to allow more information processing and the sharing of information between team members. The schedule for the Malawi JMTR had no time for processing information until just before the Verification Workshop. For sharing information, the Malawi JMTR scheduled an evening meeting on the second day in each district, which was useful, but this was not nearly enough time for good sharing of information. A full day for processing should be allocated in each district before the final day of site visits, and additional evening meetings should NOT be considered. The team is too tired to meet effectively in the evenings every day. It would also be good to have a chance for the JMTR team to meet again with district-level implementing staff by sector before leaving the district. Many of these staff will not be able to participate in the Verification Workshop, and it will give a chance for the JMTR to obtain further clarification on observations, especially if the implementing staff have not been able to accompany the team to the field. A short meeting should be scheduled with project staff before the JMTR team leaves a district to discuss observations, especially with frontline staff. The challenges with USG staff having to stay in approved accommodation meant long days for these team members. If travel time has to be extended for this or other reasons, then the review should plan to visit only one GVH in a day rather than two. Overall Data Collection and Processing As mentioned above, a full day in each district for processing and sharing of information between team members should be included in future reviews. Future reviews should also look for more opportunities for deliberate cross-pollination between JMTR team members. This could be either sitting with each other during interviews, or reviewers conducting interviews or FGDs with groups that are NOT in their narrow technical focus. The practice of having project implementing staff accompany the JMTR team to the field was extremely useful, not only for facilitating introductions to communities but also for facilitating a better understanding by the JMTR team members on what they were seeing and hearing from respondents. Both UBALE and Njira provided excellent translators for the Malawi JMTR for the most part. The translators had to be effectively oriented to their role, i.e., to translate questions and responses but also to help JMTR team members understand responses. But once that was done, the communications went smoothly. Future reviews should be allocated sufficient time in the schedule to have a meeting between the team and the translators before the field work starts to conduct this orientation. Verification Workshops The time in the Malawi JMTR was too short for (1) processing information and preparing for each Verification Workshop and (2) each workshop itself. Because of the short time available, the JMTR team was unable to share with each other their PPT presentations before the workshop to provide input into the recommendations developed by each team member to further refine them. And in the workshops themselves, there was not enough time for sufficient discussion on issues and challenges, and project leadership tended to dominate discussions. Future reviews should schedule four days before the Verification Workshop for processing and preparation (two days working as individuals, one day JMTR Methodology 5 June 2017 178 presenting and discussing as a team, and one day to finalize preparations) and the Verification Workshops themselves should be two and one-half to three days in length, depending on the scope and scale of the project being reviewed. During the preparation time for the workshops, JMTR team members should have a chance to talk again with project management teams to obtain clarification on observations and to share preliminary recommendations that may be debated in the workshop. Doing so may minimize the dominance of management teams in the discussions and allow others to provide input. The short time in the Malawi workshops suggested the need for better, independent facilitation, i.e., a facilitator who is not from implementing agencies or the JMTR team, in order to draw out a diversity of voices and encourage a safe space for all staff. Future reviews might consider finding an independent facilitator for the Verification Workshop, although extending the time for the workshops and having the discussions with management teams before the workshop are more important factors for enabling broader participation in the workshop. With a short time frame for a workshop, JMTR team members MUST adhere to the time limits for their presentations. The use of opening and ending questions to which all participants write a short response (see the Njira Verification Workshop Plan in the previous section of this report) as well as giving all participants an opportunity to write on note cards additional information after each session, is useful for verifying that major issues and challenges have been addressed, as well as for giving all participants an opportunity to express their thoughts. After the Malawi Workshops, meetings were held with project management teams on their immediate reaction to the recommendations, particularly to obtain their input on those recommendations that they felt were most useful and those that were perceived to be less useful either because they were already being undertaken by the project to some extent or the time and resources available for the project would make it difficult to implement. This information was useful for helping the JMTR team prioritize the recommendations. Stakeholder Debriefings The Malawi JMTR really did not have enough time to prepare the stakeholder debriefings. Individual team members with lighter loads had to take responsibility for assembling the stakeholder debriefing presentation on behalf of the team. In the case of Njira, the presentation for the afternoon was assembled while the UBALE presentation was being given in the morning, and there was little time in the schedule for the team to actually review a Stakeholder Debriefing presentation before it was to be given. In the case of the Njira presentation, for example, the PPT was reviewed while the team was having lunch! Future JMTRs should schedule at least one full day for preparation for each Stakeholder Debriefing. The grouping of priority recommendations into themes by the time of the stakeholder debriefings was greatly appreciated. Representatives from FFP were not encouraged to attend the Stakeholder Debriefings for the project. The USAID debriefing described below was intended to be the event at which the JMTR recommendations were presented for their input. Because of time constraints for this debriefing, however, the information presented was not very detailed. Future reviews should consider allowing the FFP representatives responsible for the project, i.e., the project's AOR and the FFP representative in the mission responsible for the project, to attend the Stakeholder Debriefing, so that they have a chance to see more detailed information on the recommendations before the final report comes out. JMTR Methodology 5 June 2017 179 Joint Learning & Higher Level Observations The JMTR Summary Report contains sections describing joint learning opportunities and higher level observations, e.g., observations that require responses that go beyond the scope of the projects that were reviewed. The Malawi JMTR did not have any specific mechanism for generating the content for these sections. Future reviews should consider having a short meeting with both projects together after the Stakeholder Debriefings but before the USAID Debriefing to hold discussions on these two topics. USAID Debriefing While the USAID debriefing went well enough, the preparation time for developing an important presentation was too short. The team leader actually prepared the debriefing during the Njira Stakeholder Debriefing which meant that he was unable to participate fully in the latter. There should be at least one full day in the schedule to allow for the preparation and review of the debriefing PPT by the JMTR team. Logistics USG staff on the JMTR team expressed a need to be informed earlier about the booking processes for accommodations and transport. As already mentioned, the translation services provided by the two projects were very good for the most part. Also as mentioned, because USG staff have to stay in specifically approved accommodation, so travel time to sites was extended. Future reviews should seek an exception to this so that less time is spent in cars and more time in villages. A notable difference between the logistics for UBALE and for Njira was that the latter provided a full￾time logistical support person to accompany the team and coordinate logistical arrangements. During the UBALE phase, for example, the JMTR team leader had to organize lunches for the team and vehicle assignments, while for Njira, this was all handled by the logistical assistant. Future reviews, especially those conducted by a large team such as in Malawi, should request projects to provide a logistical support person to accompany the team. JMTR Team Composition Having a mix of USG staff, especially from FFP, and representatives from implementing agencies on the team was useful for facilitating discussions and thinking "out-of-the-box". However, the demands on these team members, when they return to their full-time jobs, make it difficult for them to complete writing assignments. In addition, there are substantial opportunity costs for implementing partners to assign senior staff from headquarters to participate in a JMTR. Having an independent team leader was critical to being able to meet production targets for reports. The JMTR team leader, however, should not have responsibility for investigating a major component of the project in addition to coordinating the review process with a large team. Perhaps he or she could focus only on implementation systems in addition to coordinating the process. Future reviews might consider a team composition in which independent consultants are hired as team leaders for major components of the project (each of the three purposes for the projects in the Malawi JMTR) with the independent team leader coordinating the process and reviewing implementing systems. From the perspectives of the projects being reviewed and the importance of the review, it is important that JMTR team members have appropriate and significant experience in the topics important to the project. The team should also have at least some members who have experience with FFP JMTR Methodology 5 June 2017 180 programming to complement those who come from other backgrounds and can bring an outside-the￾box perspective on the project. It was useful not to have team members who had been part of the design of the projects or were currently engaged in supporting the projects. External reviewers come with fewer preconceptions about the particular projects and can also make more objective recommendations. In the case of the Malawi JMTR, one team member had to be reminded to focus on JMTR responsibilities rather than on personal work interests. Doing Reviews of Two Projects There is a strong consensus that future joint reviews should not be conducted back to back on two projects. In order to enable team members to commit to the time required, the review schedule for the Malawi JMTR had to be reduced to the minimum, and the resulting pressure to meet benchmarks in the schedule was extreme. None of the team members who responded to this process survey and who conducted the full review for both projects will commit to doing it this way again. It was useful to have the same JMTR team members review both projects. Not only were team members able to see the strengths and weaknesses of the two projects, the second review was also conducted more efficiently. However, there needs to be some down time between the two reviews of at least a couple of weeks. Draft Report The JMTR team produced a JMTR Draft Report (format shown in the protocol in Section 3) as scheduled by May 5 so that the two project teams would have more information on the recommendations to inform the planning for their JMTR Processing and Planning Workshops. This format was based on consultation with the two projects and FFP, most of whom requested a report of no more than 50 pages covering both projects. After the completion of the two workshops, it appeared that there may not have been enough information in this report to provide full clarity on the recommendations. A somewhat longer report with more information and explanation on observations would be more useful.