BY: DIANA HARPER, DR. WALES SINGINI, MACDONALD CHIMLAZA, CHIMANGO MATUNDU BY: DIANA HARPER, WALES SINGINI MACDONALD CHIMLAZA, CHIMANGO MATUNDU MAY 2020 MID-TERM EVALUATION: WORLD CONNECT PROJECT “INVESTING IN LOCALLY-LED DEVELOPMENT IN MALAWI” MID-TERM EVALUATION: WORLD CONNECT PROJECT “INVESTING IN LOCALLY-LED DEVELOPMENT IN MALAWI” Submission Date: 31 May 2020 This publication was prepared by LINC as an independent contractor to World Connect. For any inquiries related to this publication, please contact Diana Harper at dharper@linclocal.org. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of World Connect, the United States Agency for International Development, or the United States Government. Table of Contents Acknowledgments ....................................................................................................... i About the Evaluation Team ..................................................................................... ii Executive Summary................................................................................................... iii I. Introduction......................................................................................................... 1 II. Methodology......................................................................................................... 2 III. Findings .............................................................................................................. 9 Evaluation Question 1: To what extent has the current portfolio of grants progressed against the overall program objectives?..................................................................................................................................................... 9 Evaluation Question 1.1: How responsive are World Connect projects to local priorities?............................... 10 Evaluation Question 1.2: How do local partners and communities perceive and value WC support such as trainings, feedback, and site visits?........................................................................................................................................ 11 Evaluation Question 1.3: How do local partners and communities see the program as similar or different from other development efforts?.................................................................................................................................................... 15 Evaluation Question 2: What evidence exists that individuals and organizations involved in the program have increased local leadership, as defined by improvements in confidence, credibility, and mutual trust? ........................................................................................................................................................................................17 Evaluation Question 3: What evidence exists that the communities in which the program operates have increased agency, as defined by improvements in collective action, social capital, and inclusiveness?..22 Evaluation Question 4: What strategies have been used to mobilize community contributions (financial and in-kind)?.................................................................................................................................................................30 Evaluation Question 5: How do leadership and agency (Q2 and Q3) relate to development outcomes such as those in health, education, and economic growth? What additional evidence is needed to understand this relationship?...................................................................................................................................32 IV. Conclusions and Recommendations ........................................................... 35 Annex A: Evaluation SOW...................................................................................... 38 Annex B: Survey Tool .............................................................................................. 39 Annex C: Key Informant Interview Guide ........................................................... 42 Annex D: Social Network Analysis Interview Guide.......................................... 44 Annex E: Community Workshop Outline ............................................................ 45 i Acknowledgments The authors would like to thank Frank Kasonga, Ruth Chamangwana, and Bernadette Mweso from World Connect in Malawi for their programmatic insights as well as extensive support to the logistics and coordination of the evaluation. We would also like to thank Patrick Higdon and Pamela Nathenson from World Connect headquarters for their leadership to the evaluation process and insights on the global World Connect approach and operations. LINC team members Rich Fromer and Kati Chilikova provided valuable technical and programmatic support to the evaluation. We would like to acknowledge USAID Local Works and USAID/Malawi, in particular Patrick Phoso, for the technical leadership and funding of the World Connect project and for their encouragement and orientation toward learning through this evaluation. This effort was also successful as a result of the support of local project partners including CorpsAfrica, Flame Tree Initiative, Peace Corps and Returned Peace Corps Volunteers, Segal Family Foundation, and the Young African Leaders Initiative (YALI) in connection with the U.S. Embassy in Malawi Office of Public Affairs. Finally, we are extremely grateful to the research participants for their time and openness to share their views and perspectives. ii About the Evaluation Team Diana Harper, International Team Lead: Ms. Harper is a senior MEL expert who specializes in systems change, participatory research, and coalition building. Across the past 15 years, she has advanced data-driven decision making in the democracy, youth, and health sectors. She holds a Masters of Public Health from Johns Hopkins University and a Bachelors of Arts in Political Economy from Georgetown University. Wales Singini, PhD, Local Research Director: Dr. Singini is an Environmental and Natural Resource Scientist with over 15 years of experience in Malawi and is currently the Director of Research at Mzuzu University. He has conducted progressive research and evaluations in Malawi focusing on livelihoods, education, natural resource and environmental management, climate change, agriculture and food security, water, sanitation and hygiene. He holds a PhD and MSc in Natural Resources and Environmental Management and a BSc in Agriculture. Macdonald Chimlaza, Local Researcher: Mr. Chimlaza is a researcher and project management specialist with 13 years of experience in the fields of education, gender and early childhood development. He has worked with the Ministry of Education and various local and international organizations such as Foundation for Children’s Rights and Plan International Malawi in facilitating rural development in Malawi. He holds a Masters of Science in Transformative Community Development focused in Development from Mzuzu University and a Bachelor of Arts in Social Sciences offered by Uganda Martyrs University. Chimango Matundu, Local Researcher: Ms. Matundu is a food security, human nutrition and animal science expert with more than 3 years of experience in fields including livelihood assessment, agriculture, value chain analysis, monitoring and evaluation and project management. She has worked as a facilitator and researcher for various organizations, spearheading different development programs such as the Business Innovation Facility (BIF), a DFID funded market systems program; Impact Evaluation with Center for Agriculture Research Development in collaboration with the World Bank; and a feasibility study of Msika Project with Land O Lakes and Agro Dealer Assessments with the Association of Women in Agribusiness Project. She holds a Bachelor of Science in Food Security and Nutrition from University of Livingstonia and Diploma in Animal Health and Production from The Lilongwe University of Agriculture, Natural Resources College. iii Executive Summary Introduction Local community engagement has long been championed as a central factor for sustainable international development. Similarly, most donor and national development programs emphasize the need to empower local leaders as well as strengthen local institutions. However, the extent to which development programs successfully put these principles into action is frequently questioned, particularly in low-resource and donor-dependent countries such as Malawi. A closer examination of efforts to strengthen locally-led development can offer insights for the development community in Malawi and more broadly. Our team conducted an independent mid-term evaluation of the World Connect Project “Investing in Locally-Led Development in Malawi.” This three-year, approximately $1.5 million activity is funded by the United States Agency for International Development (USAID) Local Works office and has a life-of-activity target to conduct 100 community-led projects through small grants. The evaluation was intended to assess progress of the program in Malawi to date, validate the theory of change in the local context, and generate recommendations for program implementation and monitoring and evaluation. Desk research and field work was conducted by LINC and researchers affiliated with the Mzuzu University from January to April 2020. The program has conducted several rounds of grant-making to project teams throughout the country, with 139 projects approved to date. Each project is sourced through a network of Established Field Partners (EFPs), who submit proposals in cooperation with local leaders and community-based organizations (CBOs). Proposals are not limited to specific technical areas nor tied to a pre-defined set of outcomes. Rather, projects are intended to reflect community-defined needs. Projects are implemented across a period of six to nine months on average. During the design and implementation stages, World Connect provides grantees and communities with support through trainings, site visits, and virtual assistance. The program’s theory of change posits that this process of generating and implementing community grants results in short-term benefits in terms of achieving the direct objectives of the grant, as well as longer-term benefits, including increased community capacity and local leadership. Methodology The evaluation used a mixed-methods evaluation design that included secondary document review, key informant interviews (KIIs), social network analysis (SNA), community workshops, and a survey of project participants. Field work collected and analyzed data to examine outcomes at both the project level (to extent to which project objectives were achieved and sustained) as well as at the community level (the extent to which community capacity had been strengthened). A total of 12 project sites were selected for field work, 4 of which were fully conducted in person and 8 of which were conducted virtually due to the emergence of COVID-19 during the evaluation period. Conclusions The World Connect program is strongly valued by grantees and local partners, and is providing high-quality support to local project teams. Data collected and analyzed during this evaluation lead the evaluation team to conclude that the World Connect program in Malawi has progressed very well according to its objectives. Through surveys, field work, and program reporting, grantees demonstrated extensive approval and support for the World Connect program. iv Grantees and partners assigned a high and positive value for the support provided by World Connect, including trainings, site visits, and virtual support. Project teams benefited from operational support as well as technical support in their interventions. Moreover, grantees particularly valued the mentoring and encouragement they gained through engagement with the World Connect team. Activities funded by World Connect grants are responsive to the needs of local communities. The lack of restrictions on World Connect grants has resulted in projects that reflect local needs, which are often multisectoral. In addition, this structure has allowed many communities to fulfill a need that has long existed and remained overlooked or unaddressed by other development efforts. The responsiveness of World Connect grants has been supported by an efficient strategy of working through an EFP network, as well as providing training in the project design and community engagement process. Project teams demonstrated improvements in their leadership skills and momentum for development activities. Many local project teams reported that their leadership skills and capacity had been improved through the direct support and training of the program, as well as the practical experience of implementing activities in their communities. Follow-up with project teams after the implementation period of their grants showed that most projects were sustained. Additionally, project teams had used the skills and momentum generated through the World Connect grant activities to tackle other problems in their communities. Beyond individual grant results, it was plausible that World Connect contributed to improvements in overall community capacity. Workshops were conducted with community members who were not directly involved in the World Connect projects. In an open-ended exercise, participants identified what internal capacities are needed within the community to drive long-term development. Participants identified the same types of community capacities that are targeted in World Connect’s theory of change, including local leadership, collective action, and mutual trust. Additionally, the majority of participants indicated that that internal community capacity for development had increased in the last year. Because there was a logical pathway between the World Connect interventions and the underlying drivers of community capacity (as identified by the community members themselves), it was plausible that World Connect contributed to these changes. However, additional research is needed to fully understand this relationship, as a wide variety of contextual and other factors influence this outcome. Community cost-sharing requirements have been met through a variety of strategies, and are more successful when approached slowly and collaboratively. Grants include a requirement of a 25% contribution from the community, 10% of which must be cash. Project teams and communities predominantly demonstrated support for the idea of a community contribution and reflected that it was important to the ownership and sustainability of local initiatives, though many participants found the process challenging. Local project teams most frequently met the requirements for in-kind contributions through donations of local materials, unskilled labor, and land, and financial contributions through fundraising from the community, existing village and school development funds, and interest from village savings and loan funds. Projects that engaged community members collaboratively to define the method, amount, and distribution of the financial contribution were more likely to gain traction. 1 I. Introduction World Connect is a U.S.-based 501(c) (3) non-profit organization with a registered subsidiary office in Malawi that invests in and builds relationships and partnerships at the grassroots level. Through a small grants program, World Connect seeks to catalyze local knowledge and initiative, as well as support women and communities to prioritize and tackle their own development challenges in their own unique ways. World Connect projects are implemented with small amounts of funding (the average grant size is $3,000) and in periods of six to nine months on average. In all projects, World Connect carefully sources proposals through a network of Established Field Partners (EFPs), who submit proposals in cooperation with local leaders and community-based organizations (CBOs). World Connect reviews and provides feedback on project design, works with local partners to design monitoring and sustainability plans, and collects mid-project progress and final reports. World Connect also conducts site visits to provide in￾person support to project partners and to validate the results and sustainability of ongoing and completed projects. In Malawi, World Connect is implementing the “Investing in Locally-Led Development in Malawi” Project funded by the United States Agency for International Development (USAID) Local Works office under Award Number: 7200AA18FA00004, a three-year project with a total budget of $1,433,576. Through this project, World Connect is advancing the mutually agreed objective, Citizen Rights and Responsibilities Exercised, documented in Development Objective Agreement (DOAG) Number 3 between the United States of America and the Government of the Republic of Malawi. The program in Malawi differs from World Connect’s program in other countries mainly due to the presence of a small, local implementation team who provides expanded oversight and support for grantees. In 2019, World Connect sought a partner to conduct an external mid-term evaluation of the program in Malawi. The evaluation was intended to assess progress to date, validate the theory of change in the local context, as well as provide input to improve future monitoring and evaluation approaches in Malawi and worldwide. This report provides findings, conclusions, and recommendations based on research and field work conducted by LINC and researchers affiliated with the Mzuzu University from January to April 2020. Composition of Grant Project Teams Established Field Partner (EFP): In Malawi EFPs are drawn from networks including CorpsAfrica, Flame Tree Initiative, Peace Corps and Returned Peace Corps volunteers, Segal Family Foundation, and Young African Leaders Initiative (YALI). EFPs facilitate grant applications, support local leaders, and serve as a bridge for communication and reporting between local leaders and World Connect. Local Leader: The local leader is a Malawian national who was born in, is living in, or is otherwise deeply familiar with the project site. Local leaders commonly hold a formal leadership role in government or community organizations, or hold informal roles such as activists. Local leaders are responsible for leading the projects side￾by-side or in place of the EFP. Community-Based Organization (CBO): Community organizations are defined by World Connect broadly as a local committee, association, cooperative or group, formal or informal, based in or working deeply in the project site. In some cases, they are partnering with the local leader, and in other cases, the local leader is part of the CBO. 2 II. Methodology Program Theory of Change and Evaluation Design In the development of the evaluation questions and planning process, a series of consultations with World Connect served to summarize the current theory of change and identify which linkages in the theory of change were lacking evidence. The result is pictured below. At a general level, World Connect hypothesizes that if local communities generate their own ideas for community-led projects, built on local knowledge, assets, skills and momentum, and if World Connect provides training, tools, direct investment, and trust, then these community-led projects will achieve direct project results as well as strengthen community capacity, as defined by local leadership and agency. As a result, then the direct activities of the projects as well as the increased community capacity will lead to long-term, sustainable development outcomes in areas including health, economic opportunity, and education. As development outcomes improve, a reinforcing feedback loop is generated to improve community capacity as well. As extensive information had already been collected in the program documentation related to the relationship between specific community-led projects and their direct project results, this evaluation was designed to focus on collecting new information to understand the relationship between projects and the development of community capacity (local leadership and agency), as well as understanding the role of World Connect’s design and support in advancing this process. The evaluation used a mixed-methods evaluation design that included secondary literature review, key informant interviews, community workshops, social network analysis, and a survey of project participants. Field work was planned to collect and analyze data at two levels: at the project level and at the community level, which is generally defined as village, group village, or township. Evaluation Questions The following evaluation questions were generated based on iterative feedback and discussion with World Connect. 1. To what extent has the current portfolio of grants progressed against the overall program objectives? 1.1 How responsive are WC projects to local priorities? Program Theory of Change (Adapted) 3 1.2 How do local partners and communities perceive and value WC support such as trainings, feedback, and site visits? 1.3 How do they see the program as similar or different from other development efforts? 2. What evidence exists that individuals and organizations involved in the program have increased local leadership, as defined by improvements in confidence, credibility, and mutual trust? a. Confidence: Local partners have the capacity and drive to catalyze development in their communities. b. Credibility: Local partners have respect and recognition to lead change in their communities. c. Mutual trust: Local partners have bonds with each other that encourage future collaboration. 3. What evidence exists that the communities in which the program operates have increased agency, as defined by improvements in collective action, social capital, and inclusiveness? a. Collective action: Communities have desire and momentum to work together on shared goals. b. Social capital: Relationships and social structures support action for community development. c. Inclusiveness: Community development is driven by a broad base of community members, including women and marginalized groups. 4. What strategies have been used to mobilize community contributions (financial and in￾kind)? 5. How do leadership and agency (Q2 and Q3) relate to development outcomes such as those in health, education, and economic growth? What additional evidence is needed to understand this relationship? M&E Data Review and Thematic Analysis We conducted a review of the data in the World Connect M&E system, project reports, and other relevant documentation. In addition, a thematic analysis of grantee final reports was conducted using thematic coding in WebQDA. All reports completed as of March 2020 were included in this thematic analysis. Of particular interest were grantee narratives related to description of World Connect support, additional actions that had been taken by the community, additional investment by the community, unexpected/emergent leadership during the project period, and recommendations for World Connect. Field Work Community field work locations were selected in coordination with the World Connect team. An initial list of sites was generated randomly from the list of projects, and then sites were replaced in order to meet the selection criteria. As a result, the sample was a purposive sample, but it was not solely selected by the World Connect team, which may have led to a bias toward the projects considered to be the most successful. Selection criteria and justification for field work included: • Communities that conducted projects earlier in the program period (funded 2018-early 2019) in order to assess longer-term and community-level effects 4 • Inclusion of some communities that generated additional in-kind/cash contributions above the 25% match requirement • Diversity of geographic regions; rural and urban settings; diversity of project sector; projects with and without construction • Acceptability, willingness, and availability of project teams and communities to participate in the research during the field work time period • Logistical considerations and the total time allocated for field work The final selected projects for field work are listed in the table below. Project Title Region Data Collection Malizweni Nursery School Project North In person Sanitary Pads for Kazomba North In person S/he Innovates North In person Secondary School Science Laboratory Project Central In person Kapiri Sweet Foods Central Virtual Tiwatukule Youth Health Information Center Central Virtual Ndikudikilira (I Will Wait for You) Central Virtual Menstrual Hygiene Management in Mpemba South Virtual Tsogolo Langa Community Physiotherapy South Virtual Computer Lessons for Youth Development and Employment South Virtual Water for Kalima South Virtual SESA Youth South Virtual Field work was planned for March – April 2020, spending 1.5 to 2 days in each community. This included engagement with the project team (EFP, local leader, and CBO) as well as other community members and local authorities. Field work was conducted in the local language, Chichewa. The components of the field work are listed below. 1. Consultation with local authorities: Researchers met with relevant local authority(ies) to introduce themselves and the evaluation activities. 2. Interviews with project team (EFP, local leader, and CBO): Researchers conducted individual KIIs with the project team members. 3. Project site visit (clinic, school, well, etc.), as appropriate: Researchers visited the project site. In cases where the project was not a physical site, researchers met with relevant organizations and individuals benefiting from the project. 4. Community workshops: Researchers conducted two workshops in each community. The intent of the workshop is to gather perspectives from community members who were not directly involved in the project implementation. 5 Prior to conducting field work, the international team member facilitated a 2-day training in Mzuzu for the full evaluation team (local research director, 2 local researchers) and the three local World Connect staff. This training included an orientation to systems thinking and systems evaluation, pre-testing and translation of data collection tools, a simulation of the community workshop, and detailed review of the implementation schedule. Changes to Field Data Collection Researchers completed in-person data collection as planned for four sites (3 North and 1 Central) in March. At this time, the COVID-19 pandemic emerged and it was no longer advisable to travel throughout the country nor to organize group meetings. For the safety of the researchers and local communities, for the remaining eight (3 Central and 5 Southern) sites, researchers conducted KIIs by phone with the EFPs and local leaders. This presented limitations to the completeness of the dataset, as researchers could not visit and observe the sites independently and did not complete community workshops in those sites. In addition, most KIIs were conducted in a shorter time period by phone (~60 minutes) than in person (~90 minutes) due to the air time expense that was incurred. However, the evaluation team in consultation with World Connect determined that there was sufficient data available to continue with the completion of the evaluation. Drawing from the existing secondary data, in-person data collection, virtual data collection, and survey data, the evaluation team conducted the analysis and developed conclusions and recommendations. Plans have been proposed to complete the community workshops at a later date when the COVID-19 travel restrictions are lifted in order to better inform World Connect’s understanding of local progress, particularly the evaluation questions related to community agency. In addition, as noted above, the World Connect Malawi team was trained on using the evaluation’s data collection tools, which allows them to incorporate additional inquiry into their existing site visit and monitoring processes. Community Workshops Two workshops were held in each community with in-person data collection, for a total of 8 workshops. Each workshop lasted approximately 2.5 hours and included 6-12 participants. Participants were drawn from groups such as: village/town development committee, women’s committee, parent/teacher association, youth group, health action group, other formal and informal local authorities, and neighbors. The specific participants for each workshop were determined according to the project type and local context. In general, participants within each session were selected so that they were known to each other and of similar status in order to encourage an open and honest discussion. As appropriate, workshops were separated by gender to allow for increased female participation. The workshops included a mix of individual, small group (<5 in each group), and large group activities, and emphasized participatory and open-ended exploration. Themes explored through the workshop included: • How local communities define and perceive community capacity that is necessary for long￾term, sustainable development, • What are current community capacities related to agency (as defined by participants), and • Whether community capacity has changed recently, and if so, what has driven those changes. 6 Social Network Analysis As part of the key informant interviews, researchers collected data for a basic social network analysis (SNA) to understand who the project team interacted with in the design and implementation of their projects. SNA is a way of thinking about social systems that focuses attention on relationships among actors in a system. (To learn more about SNA, click here.) SNA was selected as a research tool for this evaluation according to the interest in understanding the extent to which mutual trust and social capital was developed as a result of the grants program. Our simple analysis resulted in a set of ego networks: these graphs put the respondent in the center and show who they were involved with during the project design and implementation (see example, at right). An ego network only shows the relationship between the respondent and their connections; it does not show relationships among connections. Individual attributes Relationship attributes Organizational affiliation (government, local government, local NGO, local structure, international NGO, academic, private sector, citizen, faith-based) Typically involved in development discussions (yes/no) Frequency of involvement (high/low) New relationship (yes/no) SNA was completed for all the in-person data collection sites (4) as well as almost all of the virtual data collection sites (7). One site was omitted from the SNA due to the respondents’ lack of time available for an extended interview. The original SNA tool included an active exercise that allowed the respondent to draw their connections, and the tool was slightly modified when the evaluation team switched to phone interviews. While the ego networks were not intended to be compared to each other, it is worth noting that it is likely that the SNA data collected in person was more robust than the data collected by phone, due to having less time available for the interviews. SNA data was analyzed in Kumu and summary statistics analyzed in Excel. Participant Survey A survey was developed based on the evaluation questions and literature review, as well as consultations with World Connect. The survey had an estimated completion time of 20-30 minutes and was offered in English and Chichewa. Originally, the surveys were intended to be collected by two methods: electronically for EFPs who generally have access to the internet, and by paper for local leaders and community organizations, who have less connectivity. It was planned that the EFPs would distribute these paper surveys to their project team and return the completed forms to the evaluation team. However, due to movement restrictions put in place as a result of the COVID outbreak, the survey was only offered in the electronic format. All project teams were invited to participate in the survey online through SurveyMonkey, with up to three participants per project team. Survey data was collected over a 2-week period. Example of SNA map, drawn by participant. (Photo: MChimlaza) 7 A total of 66 responses were received, with more respondents who were male compared to female, rural compared to urban, and involved in projects that included construction compared to those without construction. Survey Respondent Profile Gender Location Construction Respondent Total Female Male Rural Urban With Without EFP LL/CBO 30 36 37 29 42 24 46 20 66 45% 55% 56% 44% 64% 36% 70% 30% In order to preserve anonymity of responses and encourage honest responses, the survey did not ask which project the respondent was associated with. Accordingly, it is not possible to determine whether the EFP respondents were from the same projects as the LL and CBO respondents. As a result, we can assume the response rate relative to the total number of projects (116) ranges from 40% (46/116, assuming complete overlap of EFP and non-EFP respondents) to 57% (66/116, assuming no overlap). Survey data was collected by Survey Monkey and analyzed in Excel and Tableau. Limitations Comparison groups: As this is a performance evaluation and not an impact evaluation, data collection was focused on sites that had received the intervention without any control communities. While it would have been interesting and useful to include additional sites that had not received grants, or sites that had received other types of grants, this was beyond the scope and resources of a performance evaluation. A wide range of contextual factors might affect the outcomes of interest including community agency and leadership. While changes observed cannot be fully attributed to the program, to the extent possible, data collection instruments included a comparison to the time period before the grant was provided, as well as to other current development programs. This served to identify the program’s contribution to the observed outcomes. Response rates: As noted previously, due to the emergence of COVID during the evaluation period, several data collection activities were conducted virtually rather than in-person. This negatively affected the survey response rate and the number of field work KIIs, as well as eliminated many of the community workshops. Biases: Understanding that much of the data was collected among grantees, we consider that there was a potential social desirability bias. This can lead participants to report more positive outcomes to demonstrate progress as a result of the grant. We attempted to reduce this bias through assuring confidentiality of participants, assuring that the evaluation results would not affect current or future benefits, data collection instruments that examined successes and challenges, the use of “devil’s advocate” questions, and an emphasis on using the evaluation as an opportunity for learning and improvement. Analysis, Triangulation, and Validation The evaluation team completed note-taking sheets for all key informant interviews, SNAs, and community workshops for team review. Across a series of extended meetings, the team reviewed each site’s data jointly. The team shared reflections related to the evaluation questions, identified 8 common themes, and compared and contrasted site examples to one another. In the final joint analysis session, the team reviewed each evaluation question and generated overall conclusions based on supporting data from each site. During these sessions and afterwards, data from the secondary review, survey, and other sources was integrated into the analysis in order to triangulate findings and strengthen the conclusions and recommendations. Initial findings were presented to the World Connect team for validation, and the final report was updated as a result. Additional validation activities planned to occur among a wider group of stakeholders were not possible during the evaluation period due to travel restrictions as a result of COVID-19. However, the evaluation team encourages the World Connect team to continue engaging with stakeholders on topics raised in this evaluation as they apply the learning in their work. Evaluation findings were distributed to stakeholders, project partners, project teams, and participants in the research, including a summary in Chichewa. 9 III. Findings In this section, we present a summary of findings according to the evaluation questions. Evaluation Question 1: To what extent has the current portfolio of grants progressed against the overall program objectives? Much of World Connect’s progress against program objectives is captured in its M&E system and existing reports. World Connect has a strong and well-organized online M&E system for data management, analysis, and visualization. Each project’s files including the grant application, interim and final reports, site visit reports, and photos and other documentation are saved in a series of online folders. The local team conducts reviews of the quality and completeness of the data, as well as conducts summary analyses on key indicators of interest. Data from these back-end sources are linked to an online dashboard which allows users to search, filter, aggregate, and analyze data across the project portfolio in real-time. Data in the M&E system are updated regularly as new reports, site visits, and other activities are completed, as reflected by the completeness of the data in the M&E system at the time of the evaluation. At the onset of the evaluation, World Connect had launched 116 individual projects. During the evaluation period, World Connect completed an additional grant review cycle, bringing the total number of approved projects to 139. This exceeds the overall program goal of 100 projects. Among approved projects, 59 have been completed and 57 have submitted their final reports. According to data in the M&E system as of March 2020, the total number of people directly reached and or/engaged in projects was 19,554 (Female: 11,542, Male: 8,012), and the total number indirect reached and/or actively engaged in projects was 72,196 (Female: 40,827, Male: 31,369). Grants are offering opportunities to new and informal organizations, as demonstrated by program data that shows that 72% of project teams have not received international funding before, 38% of community organizations do not have legal registration, and 33% of community organizations do not have a bank account. Community contributions are at or above the targeted amount, with $54,244 (10% of grant spending) contributed in cash and $136,837 (24% of grant spending) contributed in kind. In addition, World Connect’s M&E system captures a set of common indicators in Malawi as well as in other countries. Those figures indicate that most projects completed all or mostly all of their goals and nearly all project teams reported they had the necessary support from World Connect and would recommend working with World Connect. Beyond the grant requirements, Screenshot from World Connect M&E system detailing data and characteristics of Malawi projects. 10 the M&E system reflects that the majority of projects resulted in new actions and activities by the local community (77%), additional investment beyond the cost-share requirement (53%), and unexpected or emergent leadership in the community (64%). Project Status (World Connect M&E System)1 Yes No Accomplished all or mostly all of goals 56 (79%) 15 (21%) Had necessary support from World Connect 64 (94%) 4 (6%) Recommend working with World Connect 68 (99%) 1 (1%) Emergent leadership in local community 44 (64%) 25 (36%) Additional investment in/by local community 35 (53%) 31 (47%) Additional actions in/by local community 50 (77%) 15 (23%) In the next sections, we will summarize findings for the evaluation subquestions related to the responsiveness of World Connect projects to local priorities, perceptions of the value of World Connect support, and how stakeholders view World Connect as similar or different from other development programs. Evaluation Question 1.1: How responsive are World Connect projects to local priorities? World Connect grants are intended to fund ideas and priorities generated from local communities in a grassroots manner, rather than imposing ideas or executing an agenda in a top￾down manner. Interviews with partner organizations, field data collection, and review of project documentation overwhelmingly supported that the project ideas were generated from within local communities, without restrictions or redirection from World Connect. In interviews, the fact that project ideas are locally-generated was one of the first and most frequently mentioned characteristics of the program. Grantees reported that advice from World Connect during the project design and proposal stage served to refine their ideas and add considerations, rather than direct the activities toward specific goals or priorities. Grantees and community members demonstrated a high level of appreciation for the ability to define their own needs and obtain grant funding based on their priorities. For example, when asked what, if anything, sets World Connect apart from other development initiatives, 59% of survey respondents selected “opportunity to define my own project goals” among a set of fixed choice responses.2 Many of the projects visited during the field work as well as those reviewed in the documentation offered solutions that were not solely focused on 1 As of March 2020, data was available for 71 completed projects, with some data remaining to be analyzed by the project staff. 2 Almost all survey respondents (64/66, 97%) identified at least one way in which the World Connect program was different from other development efforts. Illustrative Quotations from Grantees “World Connect does not impose project ideas or thematic areas. however, it listens and follows up with keen interests on projects that would positively impact rural communities.” “The World Connect model is so innovative and has proved to be the only way it can help to solve the challenges our communities are facing. With World Connect, projects and solutions are from the communities themselves which helps a lot towards sustainability of the project comparing to the imposed solutions.” “For over 10 years the community tried to build the school but most of the time it failed, to have the school built now is one of the great achievements to [our] community.” 11 one sector or one outcome. These seemed to reflect the holistic needs of individuals and communities. For example, two separate projects that train girls to make menstrual pads using locally-sources materials also included an income generating component. In Sanitary Pads for Kazomba, they gave girls chicks to raise so they could generate income. In Menstrual Hygiene Management in Mpemba, the project team noted that menstruation was not the only reason for absenteeism, and that girls were needed at home for income generating activities and to take care of their siblings. As a result, this project also included the development of school gardens to help needy students pay their school fees and offset family incomes. On a related note, in many cases, World Connect grants were reported to address needs that had long existed in the community, but had not been met by other donor or development activities. Again from Menstrual Hygiene Management in Mpemba, the project team explained that an international donor project had operated in that community for some time to distribute pads to girls. However, the donor project had not engaged them to determine a sustainable way to source pads in the future. As a result, once the international activity left, the needs in the community remained. In the Kapiri Sweet Foods project, an agricultural cooperative that sought to expand its operations had been unable to receive assistance from other economic strengthening activities. They explained this was because they wanted to focus on sweet potatoes, which is not a priority value chain for international groups who tend to focus on maize, soya, and ground nuts. In Malizweni, the village had been running a nursery school for more than 15 years in temporary locations or under a tree, without the ability to obtain funding for a safe physical structure for learning and preparing nutritious food for young children. It was also noteworthy that funding from the USAID program could only be used for non￾construction activities, according to the type of funding. In light of the past experience that many communities were interested in construction, World Connect did not add any restrictions to their grant-making procedures. Instead, they used other sources of funds to complement USAID funding. This reflected a dedication to making the grant-making process open-ended and responsive to community needs. These factors inform whether the grants were responsive to the priorities of grant applicants, but not necessarily whether the requests of grant applicants represent local priorities. Interviews with partners institutions and World Connect as well as field work highlighted a division between: a) ideas that come from an individual within a local community, and b) ideas that represent the collective will and priorities of a community. This theme will be discussed in further detail in the evaluation questions to follow. Evaluation Question 1.2: How do local partners and communities perceive and value WC support such as trainings, feedback, and site visits? Malawi is the only country that World Connect currently supports with an in-country implementation team. At the onset of the activity, grants were disbursed with oversight and support from the international team while a local program manager was recruited to provide training, site visits, and grantee support. In the second year of the activity, two additional team members were added to provide greater support for these activities. In later iterations of the trainings, the sessions were conducted in the local language and included not only the EFPs, but also the local leaders. The main types of support that the field team provides are: 12 • Project proposal and design: An in-person training on writing proposals, engaging community members in project design, generating community cash contributions, and other topics. • Project management, monitoring and reporting: An in-person training for EFPs, local leaders, and community organizations on finance, operations, data collection, and reporting through the World Connect systems. • Site visits: In-person visits by the local and/or international staff to assess progress and provide support, conducted during and after the implementation period. • Virtual support: Communication and support through email, phone calls, and WhatsApp on grant procedures and project implementation. Survey respondents reported a high level of satisfaction with World Connect overall, with 100% (66/66) of respondents indicating that they are very satisfied or satisfied with their experience with the program, and 98% (65/66) of respondents indicating that World Connect’s support was very important or important to achieving their goals. Trends across different types of respondents (male/female, with/without construction, urban/rural) were generally similar.3 Grantees and local partners regularly praised World Connect for following through on its promises to provide grants, and for disbursing the promised amount in a timely manner. Stakeholders interviewed cited many examples of where other donors and projects had visited their communities and promised resources that arrived much later, or not at all. 3 Considering the relatively small sample size used for this analysis, differences among the disaggregated groups are not statistically significant. 40 16 24 25 15 18 22 26 14 12 17 9 19 7 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% All Female Male Construction No construction Rural Urban Overall, how do you rate your experience with the World Connect program? Very Satisfied Satisfied Dissatisfied Very Dissatisfied 13 Respondents indicated a high level of satisfaction with specific types of World Connect support, with 80% or more reporting that they had received support beyond grant funding and that it was very useful or useful. Support to project proposal, design, and management were viewed as the most useful types of support. 54 20 34 36 18 32 22 11 9 2 6 5 5 6 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% All Female Male Construction No construction Rural Urban Overall, how important was the World Connect support to achieving your project goals? Very important Important Somewhat important Not important 14 As noted in the limitations section of the methodology, there is likely a bias in survey responses, with those who have more extreme and more positive views being more likely to respond. In order to further understand the perceptions of World Connect support, these topics were explored more through field work and document review. In terms of the training provided, most project teams visited during the field work had participated in the World Connect training and appreciated its usefulness. Project teams commented on the fact that the trainings helped them to understand what World Connect was seeking in the proposal review and reporting process, as well as to successfully implement their interventions. Project teams particularly noted the usefulness of gaining project management skills for financial planning and management, resource mobilization for community contributions, and monitoring and evaluation. The extent to which World Connect support provided sufficient guidance to all grantees on community engagement was difficult to assess. For example, in project documentation some grantees cited training on community engagement as a highlight of World Connect’s support. World Connect staff note that during the project design training and proposal review process, they have helped to identify activities that were not fully embraced by local communities, and supported project teams to conduct amplified engagement activities. In some cases, this led to a different project design or implementation plan that was more representative of community priorities. In contrast, field work reflected that a few projects did not seem to have fully engaged their communities broadly in the design and implementation of their grant. This may be a factor to explore further in the grant-making and support process. 42 40 33 29 31 21 22 25 26 25 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Project proposal and design Project management Project monitoring and reporting Site visit Virtual support Beyond the grant funding, did you receive any support from World Connect? If so, please rate whether or not the support was useful. Very Useful Useful Somewhat useful Not useful NA/Did not receive support “We were adequately trained on how to engage the community members before even coming up with a project to ensure the project is indeed to the benefit of the community members. We were also trained in how to implement, monitor and evaluate our projects all the while, engaging the community members to ensure that they too gain a sense of ownership of the project.” – Grantee 15 Site visits were similarly well received by survey respondents, with more than 80% indicating that the visits were very useful or useful. Ten respondents indicated they had not yet received a site visit, though according to program records, all projects have been visited at least once by the local World Connect team. It was unclear whether the reason for this difference was due to turnover, failures in recall, or other factors related to the survey instrument. In field work and project documentation, project teams demonstrated a high level of value for the project site visits, citing several benefits. First, the site visits provided motivation and encouragement, and demonstrated that World Connect was a committed partner. Grantees noted that the site visits energized the project teams and the local communities. In one case, the data collection visit by this evaluation team also served to reinvigorate the project team’s energy, leading them to build more bricks for the construction. Second, the site visits provided the opportunity for the World Connect team to jointly problem solve and offer ideas for improving the operations and effectiveness of the interventions. For example, grantees cited examples of how World Connect had helped them to increase the reach of their programs without additional costs through suggestions such as moving their training site to a more trafficked location, or conducting shorter daily outreach sessions during the school day rather than on the weekend. In addition to the specific areas of support noted above, across the field work and project documentation, grantees often highlighted the value of the supportive and mentoring relationship with the in￾country team. Grantees commented on the high levels of responsiveness and availability of the implementation team, and were pleased with the high level of individual support they received. Grantees often said that World Connect is “constantly” in communication and “cares” about them and their success. Some grantees noted that they would like additional support in fulfilling the grant reporting requirements and mobilizing community contributions. Because the grants are implemented on a rolling basis and new trainings and staff support were added over time, it was difficult to discern whether these needs were due to the lack of support provided by World Connect during the early implementation period, or a deficiency in the support currently provided, or other factors. These themes may warrant further exploration. In particular, community contribution is discussed further in evaluation question 4. Evaluation Question 1.3: How do local partners and communities see the program as similar or different from other development efforts? Almost all survey respondents (64/66, 97%) identified at least one way in which the World Connect program was different from other development efforts. Respondents were given a set of fixed-choice responses based on program literature and the theory of change, and asked to select as many or as few options as they thought applicable.4 More than half of all respondents indicated that World Connect offered a different approach through 1) the opportunity to define their own project goals, 2) receiving and managing a grant directly, 3) support during project 4 This question format helps to understand the factors that respondents prioritize or value the most. “The one thing that truly stands out for us is how much the World Connect team cares about our project. Even though the World Connect team has a lot of projects under their supervision, the relationship we have with them makes us feel like we are the only project they have running. We knew from the very beginning that they truly wanted us and our community to succeed and they were there to help in any way possible.” – Grantee 16 design, 4) support during project implementation, and 5) community financial and in-kind contribution. This question also had an open-ended response option, which was selected by very few respondents. What about working with World Connect was different from other development projects, if anything? (Multiple choice, check all that apply) # of respondents % of respondents Opportunity to define my own project goals 39 59% World Connect support during project design 37 56% Receiving and managing a grant directly 37 56% Community financial and in-kind contribution 36 55% World Connect support during project implementation 36 55% Community engagement process 33 50% Partnerships in the community for implementation 29 44% During the field work, project teams very frequently cited the fact that community financial contributions was different and new to their communities. In most communities, other projects and processes exist that require in-kind contributions, and some communities have proactively prepared for this by stockpiling bricks and building materials. Project teams also highlighted that the lack of remuneration for participation in project activities such as trainings and meetings was also different from other donor-funded activities. This did not necessarily mean that project teams recommend to change the World Connect approach of not paying for participation. Rather, some grantees reflected that, now that they have been through the process, they would have improved their communications and outreach approach with local participants and community members. Generally, grantees and community members contrasted the World Connect program to top￾down donor initiatives that they were familiar with in their local settings. However, it was noted in interviews with project partners and key informants that there are other programs in Malawi that offer small, community-led grants. Among these programs, World Connect was praised for the simplicity of its model and its accessibility to rural, low-resource, and first-time applicants. In addition, World Connect differs from other programs in its sourcing of applications through a network of EFPs, with the vast majority of applications moving forward to award. This is in contrast to other programs that cast a wide net and are open to any applicants, and then apply judgment of an external group to determine the small portion of applicants who will eventually receive awards. However, a full exploration of the differences among small grants programs in Malawi was beyond the scope of this evaluation, and may be of interest for future research. 17 Evaluation Question 2: What evidence exists that individuals and organizations involved in the program have increased local leadership, as defined by improvements in confidence, credibility, and mutual trust? In the development of this evaluation, we developed with World Connect a more specific definition of what are the factors of leadership that the program is intended to affect. These aspects were: • Confidence: Local partners have the capacity and drive to catalyze development in their communities. • Credibility: Local partners have respect and recognition to lead change in their communities. • Mutual trust: Local partners have bonds with each other that encourage future collaboration. To begin with findings from the survey, respondents indicated high levels of agreement that measures of their leadership have increased since implementing their projects. More than 60% strongly agreed and 35% agreed that 1) they have improved their leadership skills overall, 2) they are better able to identify needs in their community (related to confidence), and 3) improved their relationships with others in their community (related to credibility and mutual trust). These findings were similar to what was found in project reports. Many grantees reported that in the course of implementing their projects, they improved their confidence and leadership skills in planning and managing development activities. When grantees reflected on what they had learned and what they would do differently, common themes were: • To better estimate the overall resources necessary for the interventions, as well as to include contingencies in the budget to allow for fluctuations in market and gas prices. • To set less aggressive timeframes implementation, including to consider delays due to seasonality, weather, school schedules, etc. 44 43 41 21 23 25 1 0 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% Since implementing the project, I have improved my leadership skills. Since implementing the project, I have improved my relationships with others in the community. Since implementing the project, I am better able to identify needs in the community. Please read the following statements and answer if you strongly disagree, disagree, agree, or strongly agree. Strongly Agree Agree Disagree Strongly Disagree 18 • To pilot interventions in a smaller setting before trying to scale it up to meet the overall goals of the project. • To conduct more stakeholder/market research and establish more relevant contacts to improve the relevance and effectiveness of their interventions. Most of these topics are covered to a certain extent in World Connect’s training and support provision. At the same time, it is well known that practical experience is necessary for leadership development, and as a result it is unclear if there was any deficiency in the support provided. In combination with the fact that many participants are receiving grants for the first time, these thoughtful reflections from grantees likely reflect their development as leaders of future activities. One theme from the field work was that those involved with the World Connect projects felt that the experience had improved their ability to be responsive to their communities. For community leaders, they are accustomed to hearing a lot of issues from people in the community. They felt that their ability to listen broadly, provide feedback, and accommodate different views improved as a result of the program. At the same time, in a few projects stakeholders reported that the activity was heavily dependent on the EFP who had since left, or was based in another location and not part of the local community. In these cases, there did not seem to be sufficient local leadership at the onset of the grant, and the challenges to progress and sustainability were apparent as the grant period ended. In terms of confidence, in project reports grantees demonstrated pride in their accomplishments to date. Many described that they had been transformed through the process of receiving and implementing the grant, and they felt more confident in their future leadership activities. Without a specific prompt in the final reporting template, they readily offerred future plans and next steps for their projects and for tackling other challenges in their communities. In some cases, grantees discussed how they have extended the reach of their intervention, for example, by forming additional women’s groups, or by asking those who received training to share the learning with others. In addition, nearly all the grantee reports reviewed included a specific plan for sustainability of their projects, identifying exactly who would be involved and what role they would play. During field work, this theme was also apparent in the way that grantees were able to generate many new project ideas, as well as identify and seek out resources in support of these ideas. Once the teams had successfully completed one project, they appeared to gain momentum to look for other opportunities and solve other challenges. In the Malizweni Nursery School project, following the grant support to construct the nursery school, the project team together with community members desired to better meet the nutritional needs of the children, and have started growing soya for the school’s feeding program independent of additional World Connect funding. In Mpalale, after receiving a World Connect grant to build a secondary school science lab, local leaders and community members have identified a need for a girls hostel and are planning to use a similar mix of in-kind contributions and external grant support to build it. In project documentation as well, many grantees noted that upon completion of their grant, they are planning future activities using the funds generated internally through community contributions, village development and school funds, and savings and loans funds, and/or seeking external sources of funding. The fact that grantees outlined plans for follow-on activities and that field work confirmed many of these plans had been put in motion demonstrated that the momentum was lasting beyond the end of the grant period. 19 Another example of generating new development ideas was found in the Ndikudikilira (I Will Wait for You) project. In this case, the World Connect grant supported several activities to return girls to their families and schools as well as prevent future early marriages. Activities were mainly focused on policy, advocacy and communication efforts. During the course of implementation, stakeholders reflected that the financial pressures of families were one of the main reasons why girls were being sent to early marriage, and education was not enough to remove this pressure. As a result, they have developed plans for a vocation center to benefit these girls and families, and in this way they plan to address a systemic, root cause of the problem of early marriage. Related to credibility, grantees could readily discuss who were the planned and emerging leaders in their projects; more than 60% of project final reports listing examples of how unexpected leaders emerged during the course of implementation. However, there were fewer examples of individuals gaining respect and recognition as a result of the project implementation (new leaders). Rather, the narratives reflected that the projects worked with individuals who already had respect and recognition in their communities (existing leaders). In this way, projects were more likely to build on the internal assets available in the local setting. Examples of narratives related to credibility from grantee reports are listed in the table below. Credibility: Existing Leaders Credibility: New Leaders “One of the lead teachers has been an amazing resource during the training. She is very passionate about girls empowerment.” “There is one man who was a common group member but he was a leader in a savings and loan group that had existed in the community before there he had built up community entry and mobilization skills. During the implementation of the project, this man helped the project team in liaising with the group and coordinated the goal setting that took place jointly with the field partner, local leader and the community club members.” “She showed extreme dedication to get the project going at the very start, her humbleness and willingness to sacrifice her time for the good of the project was noticed by her fellow project participants and she was made the chairperson of the group.” “One of the sex workers living with HIV/AIDS is now coordinating the activities of all the support groups. She is the one who is facilitating motivation talks and meetings of members of the support groups.” During field work, the evaluation team investigated and observed how community members perceived project leaders. This generated several examples to support that community members believed project leaders to be credible and trustworthy. For example, in the Tiwatukule Youth Health Information Center project, a team member from the local CBO discussed how the development of a library for the local community has helped them to develop and improve trust from the community. In contrast to other organizations who make promises to the community and do not deliver, this CBO followed through on its plans to bring an infrastructure investment to the community, as well as established methods of maintaining the library over time. The library has also offered other benefits to the community that have bolstered the CBO’s status, including the opportunity to generate income by selling food and other wares to library visitors. In different community workshops, participants cited that when they contributed funds to the local projects, they had the confidence that it would be used for the given purpose and result in tangible improvements. In addition, community members felt comfortable sharing information with project teams. 20 Grantee reports included many narratives of how different stakeholders collaboration in the implementation of the project. These groups appeared to establish or strengthen their relationships, contributing to mutual trust. At the same time, several grantees discussed the potentially disruptive role of the introduction of the grant in their communities. These included disagreements about who would control money and resources, as well as stated discomfort with the high level of power that an EFP held over the process. While further investigation would be needed to understand the causes and conditions of these tensions, it seemed to occur more frequently in projects where the EFP did not have a permanent or close connection to the community. This was also noteworthy in light of the survey results, which less support that progress has been made in “community members trust each other” compared to other aspects of leadership and agency (further detailed in the following section on agency). Related to this topic, examining the full project portfolio through the program documentation and field work, there were a few examples identified in which there was a misuse of funds or, in one case, coopting of the project by a political leader. It was outside of the scope of this evaluation to conduct a financial audit, but it should be noted that relative to the size of the grant portfolio, these cases were very few and appeared to be resolved in the end. What was relevant to the evaluation questions of trust and accountability was that when a breach occurred, the project team and community members came together to confront the offending party. Generally, they used the justification that the project was for broad community benefit as part of their request for resolution. In the case of diverted funding, the individual was requested to return the funds. In the case of political coopting, the politician was asked to change their campaign messaging. From the evaluation team’s perspective, it was less noteworthy that these incidents occurred in the first place given the size and diversity of the project portfolio. Instead, the evaluation team found it relevant that there was a joint community response to a potential threat to the project that appeared to be resolved in the end. Findings from Social Network Analysis To further understand aspects of mutual trust, the evaluation team also conducted a basic social network analysis (SNA) to understand who project teams engaged with in the design and implementation of their projects. This was intended to understand the composition of project teams and the extent to which structures exist within communities to support development efforts. The results of the social network analysis are visualized in the graphic on the following page. • Each cluster indicates a specific project that was selected for field work. A total of 11 SNAs were completed among 1 to 3 members of the project team. • Each actor is represented by a dot. Central dots represent the EFP or local leader who was interviewed. Peripheral dots represent the individuals they were connected to during the project. • The color of each actor represents the type of organization. • The color of each connecting line represents whether the connection was new as a result of the project (blue) or existing before the project began (orange). • The size of each connecting line represents whether the actor was frequently involved in the project (thick line) or not frequently involved (thin line). • The bullseye indicates that the actor does not typically have an opportunity to participate in development processes. 21 As visualized in the SNA graphic, a wide variety of actor types were involved in the design and implementation of the projects. These networks were dominated by people within the local community, rather than from district, national, and international organizations. Individuals from local structures (defined as VDCs, ADCs, school committees, and health committees) and local government were most frequently cited. Government participants at the district, regional, and national level were included in above half of the sampled projects. Some projects included individual citizens as important actors involved in the projects, with one project in particular (visualized on the righthand side of the graphic) naming a large number of citizens. Private sector and faith-based actors were less frequently cited. Most connections listed were those who were frequently involved in the projects, the majority of whom had relationships with the project team before the project began. However, new connections were generated as a result of implementing the projects as well. Additionally, most of the connections cited were those who were frequently involved in the projects, and respondents were less able to recall those who were infrequently involved. It is not surprising that it is easier to recall individuals who were more frequently involved, though this may also reflect that the projects were led by relatively smaller core groups. This finding raises some questions about whether actors who are more distant, but could be strategic partners, are involved in the projects. Among projects where more than one SNA was completed by project team members (visualized as clusters that have multiple hubs), project team members did not name the same individuals in their networks very frequently. Government Citizen Private Sector Local NGO Academic Local Government Local Structure 22 The SNA also examined inclusion (related to evaluation question 3) by asking whether each connection typically had a chance to participate in development activities or not. This analysis uncovered relatively few new voices: six total, three of which were from one project. We consider that this question had some challenges in its implementation, both with the researchers remembering to capture this data correctly in the interview process, as well as with the research participants understanding the intent of the question. Nevertheless, this finding does warrant reflection on whether the community-led process in its current form is encouraging inclusion of marginalized groups, or reinforcing existing power structures in the communities. The SNA did not collect data about the gender of each individual, but this would be useful information for future such efforts. Evaluation Question 3: What evidence exists that the communities in which the program operates have increased agency, as defined by improvements in collective action, social capital, and inclusiveness? Similar to the evaluation question 2 on leadership, in the development of this evaluation, we developed with World Connect a more specific definition of what are the factors of local agency that the program is intended to affect. These aspects were: • Collective action: Communities have desire and momentum to work together on shared goals. • Social capital: Relationships and social structures support action for community development. • Inclusiveness: Community development is driven by a broad base of community members, including women and marginalized groups. These themes were examined from different data sources (grantee survey, social network analysis, grantee reports, and field work), which are further detailed in the sections to follow. Findings from Grantee Survey Survey respondents were presented with a series of statements about community capacity, which were drawn from the program’s theory of change, literature review, and review of project documentation. Theme Statement Collective action Community members come together to discuss community needs Community members are confident in their ability to lead change Community members work together to achieve shared goals Social capital Community members respect each other Community members trust each other Community members ask for and provide help to each other Community members hold each other accountable for development Community members hold their leaders accountable for development Inclusiveness Different types of community members are engaged in the development processes, including women and youth 23 Respondents were then asked if each statement is important for achieving project goals, which among the statements were most important, and whether these aspects had gotten better, stayed the same, or gotten worse since implementing the project. Respondents had near universal agreement that each of the themes were important. While this high level of agreement likely reflects some bias toward agreeability and positive response, it is useful validation that the project teams agree that community capacity is important in advancing project goals and development. When asked to select up to three themes that were most important, respondents identified statements related to collective action (community members come together to discuss community needs, community members work together to achieve shared goals) and inclusion (different types of community members are engaged in the development process). Respondents were allowed a free choice response to add their own themes, but very few added their own themes in this format. Of the following themes, which are the most important? Choose up to 3. # of respondents % of respondents Community members come together to discuss community needs 51 77% Community members work together to achieve shared goals 40 61% Different types of community members are engaged in the development processes, including women and youth 33 50% Community members are confident in their ability to lead change 26 39% Community members trust each other 13 20% Community members hold each other accountable for development 13 20% Community members hold their leaders accountable for development 11 17% Community members ask for and provide help to each other 6 9% Community members respect each other 4 6% With regard to the trends of these themes in their communities, 70% or more of survey respondents indicated the themes had gotten better since implementing the project. The most frequently cited improvements were in collective action (community members are confident in their ability to lead change, community members work together to achieve shared goals) and inclusion (different types of community members are engaged in the development process). It is noteworthy that two of these themes, community members working together and engaging different types of community members, were also identified as the most important themes for achieving their goals. 24 Survey respondents indicated moderate levels of agreement that measures of their inclusion have increased since implementing their projects. 33% percent strongly agreed and 59% agreed that decision-makers pay more attention to their priority issues. The survey also asked whether people whose voices are not usually heard have more opportunities to participate in development, which related both to inclusion and local leadership. 48% strongly agreed and 47% agreed with this statement. 53 60 57 46 47 54 54 49 59 13 6 9 19 17 12 12 17 7 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Community members come together to discuss community needs Community members are confident in their ability to lead change Community members work together to achieve shared goals Community members respect each other Community members trust each other Community members ask for and provide help to each other Community members hold each other accountable for development Community members hold their leaders accountable for development Different types of community members are engaged in the development processes Please indicate if you think the theme has gotten better, stayed the same, or gotten worse since implementing the project. Improved Stayed the Same Gotten Worse 22 32 39 31 3 3 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% Since implementing the project, decision-makers pay more attention to the issues that I care about. Since implementing the project, people whose voices are not usually heard have more opportunities to participate in development. Please read the following statements and answer if you strongly disagree, disagree, agree, or strongly agree. Strongly Agree Agree Disagree Strongly Disagree Collective Action Social Capital Inclusion 25 Findings from Grantee Reports and Field Work These findings were aligned with grantee reports, which detailed many examples of collective action. The main themes in collective action activities involved community members who organized themselves to: • Collect and manage resources, • Maintain current activities of the intervention • Extend the reach of the intervention to additional individuals, groups, or communities, • Solve related or additional local problems. Grantees reported many examples of engaging with different types of organizations in order to launch these collective action activities, which is aligned with the findings of the SNA, detailed in the previous section. During field work, some community members reflected that the process of jointly discussing local problems, developing solutions, and then implementing the solutions together was a new idea. These community members were more accustomed to receiving news about what was coming to their village or town or the results of decisions made by others. Closely related to this theme is the idea of social capital, and whether relationships and social structures support action for community development. Most frequently, grantees discussed how local leaders and champions used their influence to encourage community involvement in the programs and maintenance of structures. Less frequently, grantees discussed using connections with technical specialists within or outside their communities. For example, one grantee working on employment training noted a missed opportunity to affiliate with a center of higher education to confer certificates and credentials upon the project participants. Another grantee working on new methods for briquette making notes that several other groups in the country are working on a similar activity, and they would have benefited from learning from them before they started. Many grantees and community members discussed how they strengthened bonds among each other through their in-kind contributions to the project, specifically by spending time together on labor tasks. In the words of one grantee: “I think it promotes oneness among the community members, bring different people from villages to work on one common goals.” The grantee went on to describe how as the women worked together on the construction, they would share ideas and advice with one another, such as better ways to cook beans. In other examples, community members appeared to enjoy the time spent with each other in support of the project goals, in particular while working on construction or agriculture (either as part of the project or to generate income for the project). Illustrative Quotations from Grantees “The school management committee has begun to construct a similar structure as the one erected from the project.” “Women with disabilities have formed committee within their zone so that they should encourage each other to bring their children for the physio therapy and rehabilitation. The committees have helped to register more cxhildren at the centre.” “The project has managed to initiate the community led economic development thinking and communities are demanding professional expertise from [our] group which has led to formation of other new (four) beekeeping groups awaiting resourcing for modern beehives.” “The high pressure in the deliver pipes has led to frequent bursts which has forced people to constantly engage in maintenance to restore water flow. This has seen strengthened water point committees which collect monthly contributions and use such proceeds to carry out maintenance costs.” 26 Grantees also cited many examples of reciprocity, meaning that community members contributed something to the projects and built stronger bonds in their community as a result. Broadly, this included the in-kind contributions by community members, but other examples were noted as well. For example, a community member volunteered to take on facilitation activities when the project staff member needed to take time away for personal reasons. In another example, a local NGO offered an additional training on innovation to individuals who were receiving employment training through the project. With regards to inclusion, many projects included a direct goal of increasing inclusion as part of the project design. These objectives included increasing youth employment, offering services to people with disabilities, and increasing school participation for girls. In these cases, it was common that the affected group was involved in the design and implementation of the projects. In some cases, this was achieved directly by having marginalized group members as leaders, and in other cases it was achieved indirectly by taking a youth- or girl-centric view to the project design. However, the intentional engagement of marginalized groups was generally limited to those projects that were focused on marginalized groups. As a result, it was less likely that other types of projects, such as a general economic development project, had an intentional approach to inclusion. This appeared to be aligned with findings from the SNA on inclusion, described previously. Field work revealed some additional learning related to inclusion and specifically gender. Some of the World Connect projects are offering opportunities for women to engage in traditionally male activities. In S/He Innovates, young women were targeted to be part of the project’s training for barbershop work. This was novel in the local context where it is rare that women are barbers. It was reported that when this activity emerged, the community didn’t understand why women are shaving men’s heads, but that over time the people from the community have accepted it. A similar example was noted in Water for Kalima, which had women participate in digging the trenches for a new water system, which is also an activity that women do not typically participate in. It was reported that the women were involved in the physical construction not “because the project said so,” but rather because everyone would benefit from the water source, and everyone should work as a team to contribute. As a result, the community decided the women would take part in the trench digging, referencing the 50/50 campaign for gender equity that is ongoing in the country. In the case of S/He Innovates where community workshops were done, it was also interesting to note that women who had participated in the program were very active participants in the workshops, feeling confident to share their opinions even in a mixed gender setting. Internal Community Capacities: Findings from Workshops In the four communities where field work was conducted completely in-person, the evaluation team facilitated a series of workshops with community members. (As noted in the methodology section of this report, due to the emergence of the COVID-19 pandemic, in-person field work was reduced from 12 communities to 4 communities.) During these sessions, participants reflected on the major development challenges in their communities. They then generated a set of ideas on what is needed to address these challenges from outside their community, as well as inside their community. The purpose of this part of the workshop was to generate an open￾ended understanding of what internal capacities and assets communities believe are relevant to long-term development, allowing the evaluation team to compare World Connect’s theory of change with local perspectives. In the final part of the workshop, using the specific community 27 capacity themes that the participants had generated, the groups voted and held conversations on whether each theme was getting better, staying the same, or getting worse in the last year, and why. The purpose of this activity was to understand trends in community capacity, and provide some insight into the extent to which World Connect support may have contributed to any changes. A summary of the community capacities, their status, and drivers of perceived changes is listed in the table below. Community Capacity Status Drivers of Change Trust in leadership Getting better - Leaders’ hardworking spirit and dedication - Leaders listen to people’s voices - Leaders are humble - Leaders are able to complete projects - Mentoring others and fostering the development of future leaders Lack of corruption Getting better - Leaders are accountable in management of funds - Leaders are transparent Getting worse - Leaders support their own interest and not the community’s shared goals - Leaders don’t listen to people’s views Unity among leaders and local structures Getting better - Increased commitment to area development - Understanding that all leaders need to work together to solve community problems - Everyone plays their role About the same - Decreased commitment to area development because they lack transparency and accountability - Lack of coordinated efforts among leaders Networking among local structures Getting better - Sharing of information, knowledge and skills - Openness in doing things Oneness among community members Getting better - Respect for one another encourages others to join hands - Desire to solve own problems compels community members to come together and deal with a common challenge. About the same - Desire to solve own problems, but do not achieve effective results - People take what they want and don’t consider the community needs Ownership of projects by community members Getting better - Leaders encourage community members to develop their own ideas - Leaders help people understand the benefit to the community as a whole - People are getting to know that they are in charge of their development Inclusion Getting better - Local leaders emphasize that everyone is equally important in development - NGOs raise awareness on importance of social inclusion and how marginalized groups can contribute to the development of their community 28 It can be considered that there is a social desirability bias in these responses: participants are more likely to tell a positive story that they believe will please researchers and reflect their communities in a positive light. It should also be recognized that, although the evaluation team sought to encourage participation by community members who were not directly involved in the projects, workshop participants were selected and assembled by the local project teams for practical reasons. That said, it is still noteworthy that among the capacities that participants thought were most necessary for addressing the most pressing development challenges, many participants generally believed that these capacities are improving in their communities. In general, participants more readily discussed the role of local leadership, often referring to formal and established leaders in their communities. For example, when discussing the topic of inclusion, workshop participants were likely to discuss how local leaders could foster the inclusion of marginalized groups. When discussing community ownership, they discussed what approaches leaders could take to confer ownership among the community members. Related to social capital, participants prioritized a few different types of relationships that were important for addressing development issues: • Relationships between leaders and their community members • Relationships between leaders and local structures (such as the ADCs and VDCs) • Relationships among local structures While participants discussed relationships among average community members, particularly related to how they respected each other and worked together on community goals, this type of relationship was less frequently emphasized compared to other relationship types. Corruption and how leaders engage with their communities were prominent themes in the discussions. In some communities, these aspects were perceived to be improving, and in other communities, staying the same or getting worse. Regardless of the trajectory, communities identified the role of transparency, accountability, and whether leaders listen to community needs as driving forces. On a related note, most communities discussed the theme of oneness among community members, which related to the evaluation themes of mutual trust and collective action. In most cases, participants said oneness was improving, driven by Participants identified what internal capacities are most important for development. (Photo: MChimlaza) Participants identified trends in community capacity in their community. (Photo: CMatundu) 29 improvements in respect and dealing with common challenges. In some cases, participants considered that oneness among community members was not improving, which was driven by individualism and the failure to achieve results effectively. Internal Community Capacities: Interpretation To interpret findings from the community workshops, the first consideration is that the questions posed about community capacity were at community level and not directly restricted to World Connect projects. This was purposeful in order to explore the definition of community capacity as well as understand World Connect’s contribution to the high-level outcome of community capacity. Of course, in any one community, there is a constantly evolving range of development projects as well as contextual factors and events that could affect how the community functions overall. However, it was noteworthy that among communities sampled, overall, community capacity was reported to increase at the local level, both in terms of leadership and agency. Most participants reflected that increases in community capacity highly depended on local leaders. In defining community capacity and drivers of change, the workshops used an open-ended format. This activity showed that the types of community capacity that are targeted or expected through the World Connect grants (such as leadership skills, collective action, and mutual trust) were also identified as important factors by local communities. In addition to the factors specifically explored through this evaluation, participants also emphasized the important role of transparency and accountability. Particularly in combination with data collected in other parts of the evaluation, it was plausible that the World Connect intervention contributed to these community-level changes. There was a logical pathway between the World Connect interventions and the underlying drivers of community capacity as identified by local community members, as described in the table below. World Connect interventions Drivers of change Community ownership/engagement in project design Leaders listen to people’s voices Leaders encourage community members to develop their own ideas Leaders help people understand the benefit to the community as a whole Development of leadership skills and project management skills Leaders are able to complete projects Mentoring others and fostering the development of future leaders Community ownership/engagement in project implementation Local leaders and local structures (such as VDC, ADC, school committees) work together to solve community problems Respect for one another encourages others to join hands Desire to solve own problems compels community members to come together and deal with a common challenge Community contribution Leaders are accountable in management of funds 30 In areas where communities said certain aspects of community capacity were staying the same or getting worse, common drivers were lack of effectiveness, lack of transparency and information sharing, and lack of consideration of community-wide needs. Understanding how communities view the process of strengthening their internal capacities can serve as an important foundation for supporting projects in the future. Evaluation Question 4: What strategies have been used to mobilize community contributions (financial and in-kind)? One of the grant requirements is to have a 25% cost-share from the community, which must include a minimum of cash (10%) contributions. This requirement applied across different types and settings of projects, though in a few cases the cost-share was reduced or waived—for example, projects in the refugee camp. The evaluation team examined the strategies that project teams used to generate community contributions, as well as perceptions of grantees and community members about the community contribution requirement. Strategies for Mobilizing Contributions Survey respondents were asked how the 25% community contribution was generated. Most respondents cited local materials and/or supplies, local labor, and local land, which are all in-kind contributions. Fewer responses were gathered related to the financial contributions, which most frequently cited new financial contributions, fundraising events, existing village development fund, and village saving and loans. Additional write-in responses included school fees and school development fund. For the financial contribution, other response options in the survey included new contributions from local businesses, contributions from diaspora family members, and other donor/project funds. Each of these options was only selected by 1 or 2 respondents. The greater number of responses related to the in-kind contributions may reflect a stronger understanding and recall of in-kind contributions compared to financial contributions. This summary from the survey is aligned with project documentation and field work. For construction projects, in-kind contributions typically came from donated unskilled labor from local community members. Often these members were individuals who would benefit from the project or lived in the vicinity of the project—for example, parents and neighbors helped build 7 8 8 14 44 46 54 0 10 20 30 40 50 60 Interest from village savings and loans group Existing village development fund/savings Fundraising event in community New financial contributions from beneficiaries Local land or property Local labor at low or no cost Local materials and/or supplies How did the local community raise funds and resources for the 25% community contribution? Check all options that apply. 31 the school that their children would attend. In other cases, contributing individuals had some connection to the project’s purpose, such as youth from the local technical college who helped to build a science lab for secondary school students. In-kind contributions also included land or property on which to build the structure. In one of the field work sites (Water for Kalima), a local agriculture firm offered use of their existing irrigation infrastructure, so that the community could build a pipe connection and add an access point for clean water. For materials donation, in some cases the materials were already available in a community stockpile, in recognition of the fact that many donor activities require a community contribution. Brick-making was a frequent contribution, and in other cases building materials were collected through a community drive. For projects without construction, in-kind contributions typically included labor and time of volunteers as part of the project delivery, as well as materials. In some cases, materials were items that project participants used to conduct the project – for example, the calculated value of one’s own scissors for a sewing skills training, or pens and paper to take notes in a health education session. In order to meet the requirements for the cash contribution, for both projects with and without construction activities, project teams used a few main strategies: • Use existing community funds: Particularly for construction projects and for those associated with schools and health centers, project teams obtained funding from VDCs/ADCs, school committees, and health committees, who maintain their own small development funds. • Raise new funds (one-time): Project teams and local leaders requested contributions from the community. In most cases, they asked for an equal amount from each individual or household. Selling maize, other agricultural products, and forestry products were other methods used to raise funds. • Raise new funds (recurrent): Project teams created recurrent community contributions through increased school fees, membership fees, and use fees. In some cases, these contributions were applied to individuals involved in the program, such as youth who participated in entrepreneurship training. • Develop income generating activities: Some project teams included an income￾generating component to raise their contribution as well as sustain the project over time. This included activities such as purchasing and raising livestock, as well as launching or expanding small businesses. On the whole, cash contributions were likely to be small amounts raised from a larger number of people, rather than a large amount raised from one or a few people. Among the field work sites, in most cases World Connect was the only funding going toward the project. In a few cases, World Connect funding complemented activities being funded by other donors. These grantees seemed confident and consistent in explaining what was funded by World Connect versus other donors. Perceptions of Community Contribution Participants spoke with great pride about their in-kind contributions of labor and time to the projects. There was a strong perception that when community members participate in the execution of the project, they have greater ownership of the project. Grantees and community 32 members reflected that the joint effort strengthened their bonds with each other, as well as contributed to inclusion because women, youth, and people with disabilities could all contribute in one way or another. As noted earlier under evaluation question 1, the community cash contribution was one of the most frequently discussed aspects of how World Connect grants are different from other donor programs. While the majority of project teams were eventually able to collect the necessary cash contributions, they consistently highlighted fundraising as a challenge. Interviewees referenced the poverty of their communities and reflected on the discomfort they felt when they asked for contributions in a low-resource setting. Interviewees often commented that there is a “hand-out culture” and that people expect to receive, not give, to donor-funded initiatives. A similar theme was reflected in several projects that had challenges in obtaining participation from community members in project meetings and trainings without remuneration, as they are accustomed to receiving fees from other donors. In terms of raising the cash contribution, a difference was observed in the ease with which project teams could generate community contributions according to several dimensions: • “Hardware” projects that included construction and had a tangible space in the community were easier to raise funds, versus “software” projects that provided education, technical assistance, and other types of support. • Economic projects that could generate income were easier to raise funds versus social projects that provide public good and are in competition with other public needs. • Projects that were seen to have a community-wide benefit were easier to raise funds versus those that were seen to be private. Some participants, particularly those that seemed to face fewer challenges in raising the cash contribution, referenced how community-generating funding gives more control to the community, as well as how communities that have funded a resource are more likely to maintain it. Some of the effective strategies used by project teams included: • Work through local leaders such as chiefs and block leaders • Emphasize the community-wide benefit and recognize shared responsibility for public goods • Emphasize local design and ownership, i.e., that the project was chosen by the community, not external groups or donors • Develop a plan for how resources will be generated and distributed among community members collaboratively with the community members themselves • Inclusion of the project in the main community/school development priorities Evaluation Question 5: How do leadership and agency (Q2 and Q3) relate to development outcomes such as those in health, education, and economic growth? What additional evidence is needed to understand this relationship? According to the program theory of change, grant projects have a direct pathway to affecting long-term outcomes in health, education, and economic growth, as well as an indirect pathway to change based on strengthening community capacity, which in turn strengthens the foundation for other development efforts. The final evaluation question sought to understand stakeholder 33 perspectives and available evidence for this connection between community capacity and long￾term development outcomes. The role of community capacity and assets in advancing development outcomes has been documented in the literature. As summarized in a recent USAID report, “Collaborating, Learning, and Adapting Fosters Local Ownership and, Ultimately, Better Results: A Review of the Evidence”:5 Local engagement—i.e., when local actors are involved in defining development challenges and solutions via program activities—leads to local ownership and, ultimately, improved development outcomes. Evidence from a deep dive on the Ebola crisis in Liberia, a deep dive on Community-led Total Sanitation in Zambia, and several CLA case stories demonstrates that when local stakeholders experience ownership over development efforts, the results are more relevant and effective, which in turn increases local stakeholders’ commitment to and engagement in identifying sustainable solutions to community challenges. This is consistent with the literature, which increasingly emphasizes that local ownership—with solutions that are locally negotiated and delivered—leads to more effective development. The Global Fund for Community Foundations offers additional perspectives: 6 In recent years, interest has grown around a more people-led paradigm that shifts power closer to the ground. It has found expression among funders, researchers and activists alike. #ShiftThePower is both a call for new behaviours, mindsets and ways of working and a reminder both that few interactions are ever power neutral and that often, those we seek to “help” have much more power – knowledge, skills, networks – than they are given credit for. It is the job of those institutions that are serious about real, lasting change to know when to give up their own power, stand back and help make that happen. These themes are reflected by USAID in its policy framework for the Journey to Self-Reliance, outlining how the Agency is reorienting itself to support “country capacity to plan, finance, and implement solutions to local development challenges, as well as the commitment to see these through effectively, inclusively, and with accountability.”7 With regard to the focus of this evaluation, as described in previous sections, project teams and communities valued the building of internal community capacity, including leadership and agency. They cited the role of community capacity related to the sustainability of interventions as well as for long-term growth and development more broadly. Small grants from World Connect attracted additional funds for development activities, both from internal community sources as well as external donor sources. Stakeholders emphasized that a critical component of attracting new funds and follow-on activities was that the original project idea came from the community and that the community contributed its own labor and funds to see it succeed. 5 Lindell, K. “Collaborating, Learning, and Adapting Fosters Local Ownership and, Ultimately, Better Results: A Review of the Evidence.” USAID LEARN (2020). Available at: https://usaidlearninglab.org/sites/default/files/resource/files/cla_local_ownership_briefer_2020.pdf 6 Global Fund for Community Foundations. “Community Philanthropy and #ShiftThePower.” Available at: https://globalfundcommunityfoundations.org/what-we-stand-for/shiftthepower/ 7 USAID. Policy Framework: Ending the Need for Foreign Assistance. (2019). Available at: https://www.usaid.gov/selfreliance/ 34 As a result, it was plausible that projects that were collaboratively designed and implemented with community members would contribute to broader changes in outcomes such as school enrollment, reading and numeracy skills, child health, youth employment, agricultural productivity, income levels, and gender equity. During field work and in project documentation, some grantees made claims that outcomes had changed as a result of their projects. At the same time, due to small size and the short period of implementation of the grants, it seemed reasonable that any major changes observed were driven by a wider variety of factors and set on a positive or negative trajectory well before the project began. A broader and longer-term study that examines the relationships among community-level interventions, agency, and development outcomes would be needed to assess such changes. This study may make use of quasi-experimental methods such as propensity score matching of communities that did and did not receive the intervention, or a stepped-wedge comparison of communities that received intervention earlier versus later. In theory, a future study could use an experimental design through randomization, though this can often be politically and operationally challenging. Alternately, if adopting a systems theory of how change occurs in communities, which this evaluation team would recommend, a future study may use approaches such as mapping and modelling to better understand the pathways among different interventions, community capacities, and development outcomes. Ideally, such an effort would be in a participatory manner with local communities and with local leadership. Insights from this evaluation would serve as useful insights into such a study. It may be argued, however, that whether and how development actors support community leadership and agency may be driven more by values and principles, rather than statistics and evidence. Sunflower fields in northern Malawi (Photo: DHarper) 35 IV. Conclusions and Recommendations World Connect Progress and Support: Data collected and analyzed during the course of this evaluation lead the evaluation team to conclude that the World Connect program in Malawi has progressed very well according to its objectives. Grants are well aligned with the priorities of local grantees and responsive to needs in the local communities. In addition, grant activities and support have been efficiently executed. Data is efficiently and effectively integrated into the program M&E system, and easily accessible for verification, analysis, and visualization. Grantees and partners assigned a high and positive value for the support provided by World Connect, including trainings, site visits, and virtual support. Project teams benefited from operational support as well as technical support in their interventions. Moreover, grantees particularly valued the mentoring and encouragement they gained through engagement with the field team. Grantees appeared to benefit when the full project team was engaged in trainings and support from World Connect, as opposed to focused only on the EFP. Recommendation 1: Maintain provision of high-quality support through in-person site visits, virtual support, and training. Recommendation 2: Continue to include the full project teams including local leaders in World Connect support activities. Grant Processes: World Connect grant processes were reported to be manageable in rural and low-resource settings, including for those receiving direct grants for the first time. The program was highly praised for its flexibility in allowing grantees to adapt, allowing grantees to learn from mistakes and build leadership and management skills. This was particularly useful in adjusting timelines and adjusting the details of project implementation plans, which were found to be useful as there were changes in the local context or learning generated through implementation. In some cases, grantees voiced the needs for additional offline tools and strategies for communication in low-connectivity settings, for example, alternatives to uploading large files like receipts. Recommendation 3: Maintain flexibility in grant processes, including adjustments to timelines and project implementation plans. Recommendation 4: Maintain the simplicity of grant application, management, and reporting processes, including offline tools. Community Support and Engagement: World Connect in Malawi uses a strategy to generate locally-driven ideas by working through a network of EFPs. In most projects reviewed, this approach functioned well to generate ideas that offered a community benefit, had a wide base of engagement, and were prioritized at the local level. In a few cases, projects were generated that had less community support. Consistently, these project teams had challenges in attracting community contributions, implementing, and sustaining their interventions. Recommendation 5: Review and, if necessary, refine approach to community engagement in EFP outreach, grant reviews, and project team trainings. Emphasizing the World Connect theory of change to build local leadership and agency at the community level may aid in this process. 36 Environmental Effects: In some cases, actions taken by project teams had negative or unsustainable environmental effects. For example, cutting down trees to generate income for community contribution or to burn wood for making bricks. The team reflects that the simplicity of the grant process and the lack of extensive oversight fosters the independence of the project times. At the same time, given the large portion of construction projects as well as the observed practices of communities, it appears that additional support is needed to avoid environmental degradation. Recommendation 6: Consider how to incorporate environmental controls and assessments into grant processes, either directly or through partnerships. Inclusion: There was strong evidence of women’s participation and leadership in World Connect projects, as well as the role of some projects in affecting gender equity and other gender issues. It was less clear the extent to which projects were engaging new and unheard voices in development, including those of other marginalized groups such as LGBT, people with disabilities, and the extreme poor. At the same time, communities highlighted that meaningful inclusion of women and marginalized groups is important in advancing local capacity and development. Recommendation 7: Incorporate a more diverse inclusion lens in support to project design and implementation support. Engaging a more diverse group in the monitoring project progress may aid in this effort. Community Contribution: Project teams and communities viewed the community contribution as important to the ownership and sustainability of local initiatives. In-kind time and labor contributions helped to build bonds among community members in the process. The cash contribution was one of the most readily and frequently mentioned ways in which World Connect programs are different than other donor initiatives. While the move toward sustainability and local ownership is laudable, continued awareness and sensitivity to the historical and cultural system in which cash contributions are being requested may be useful. Seeing the contribution as part of the community ownership of the project rather than a prescribed donor requirement was useful for many project teams, though this took time for community communication and engagement. In addition, a difference was observed in the ease with which project teams could generate community contributions according to several dimensions: • “Hardware” projects that included construction and had a tangible space in the community were easier to raise funds, versus “software” projects that provided education, technical assistance, and other types of support. • Economic projects that could generate income were easier to raise funds versus social projects that provide public good and are in competition with other public needs. • Projects that were seen to have a community-wide benefit were easier to raise funds versus those that were seen to benefit a smaller group. Projects that engaged community members collaboratively to define the method, amount, and distribution of the financial contribution were more likely to gain traction. Additionally, many projects incorporated creative strategies to generate income, even when they were not economic projects. These strategies served to increase local resource availability for development as well as generate increased community support for the activities. 37 Recommendation 8: Consider how to support grantees to adopt different resource generation strategies for different types of projects, as well as to better engage community members on decision-making related to financial contributions. Recommendation 9: Support grantees to understand the range of income-generating strategies that can be incorporated into projects. Partners working on social entrepreneurship may be useful for this. Community Capacity: In alignment with the program’s theory of change, the data collected and analyzed for this evaluation demonstrated good evidence that community capacity is increasing at the local level, both in terms of leadership and agency. Moreover, it was plausible that the World Connect intervention was contributing to these community-level changes. There was a logical pathway between the World Connect interventions and the underlying drivers of community capacity as identified by local community members. New tools introduced through this evaluation, specifically the social network analysis and community capacity workshops, were feasible to implement and well received in the local setting. Recommendation 10: As appropriate, include efforts and tools to assess community capacity in future monitoring and evaluation efforts. Potential Topics for Future Learning Policy change and implementation: Relatively few projects were identified that focused on policy change. However, the one example in the field work sample (ending the practice of girls’ early marriage) appeared to be very successful in generating community support for implementing policy changes, as well as building consensus to address the root causes of the problem. It may be useful to examine why policy interventions are so few in the portfolio; whether it is due to lack of interest/prioritization among communities or lack of knowledge/motivation to use the grants for these purposes. Networking, sharing, and advocacy: Grantees did not commonly discuss many current or desired activities to network and share practices among other grantees. This was surprising to the evaluation team, as networking among members is often a feature of grant programs and/or a request among grantees. It may be useful to verify this finding, as it was not a major focus of the evaluation. At the same time, it was noted that it is very common culturally that when neighboring communities hear about a successful activity, they take the initiative to visit and seek to replicate the activity. There may be a role for World Connect or its partners to support project teams in sharing and replicating successful approaches to community-led development, in order to achieve a wider effect. In addition, the World Connect team may further extend their effects by joining with other partners who are advancing community-led development in advocacy initiatives. 38 Annex A: Evaluation SOW Scope of Work In Malawi, World Connect is implementing the “Investing in Locally-Led Development in Malawi” Project funded by the United States Agency for International Development (USAID). Through this three-year Project, World Connect is advancing the mutually agreed objective, Citizen Rights and Responsibilities Exercised. This mid-term evaluation will assess progress to date, as well as analyze the project's approach to M&E, in order to inform the remainder of the project implementation period. Tasks • Conduct a desk review of existing data, M&E systems, and approaches • Travel to Malawi for 2 weeks to 1) conduct meetings with project staff and stakeholders, and 2) accompany project staff on project site visits • Develop an evaluation methodology and implementation plan • Train and oversee local partners on data collection and analysis • Product a mid-term evaluation report with findings, conclusions, and recommendations Draft evaluation questions • To what extent has the current portfolio of grants progressed against the overall program objectives? • What evidence exists that the theory of change is valid and complete? What shifts, if any, in the implementation strategy are needed? • What evidence exists that World Connect is contributing to sustainable and community￾level outcomes (such as leadership, confidence, social capital, and collective action)? • What additional evidence is needed to validate the theory of change and to communicate results by the end of the project? How should this data and information be collected, analyzed, and validated? Final questions will be defined in the evaluation plan. Deliverables 1. An evaluation plan including methodology, tools, and implementation timeline. 2. An evaluation report detailing the work undertaken, results against key learning objectives and questions, and recommendations for future monitoring and evaluation strategies. 3. Presentation of results (virtual or in person in New York). 39 Annex B: Survey Tool Note: The survey was distributed on SurveyMonkey and appeared slightly different than the questions listed below due to the computer-generated, fixed-choice format. The survey was also offered in Chichewa. 1. Beyond the grant funding, did you receive any support from World Connect from the design through the completion of the project? • Project proposal and design: Support to apply for, plan, and launch project • Project management: Support to finance, operations • Project monitoring and reporting: Support for measuring success or change and how to complete reporting on projects • Site visit: In-person visit from the World Connect team • Virtual support: Phone, WhatsApp, or other virtual support to help us be more successful • Other (please describe) If so, please rate to what extent the support was useful. • Not useful Somewhat useful Useful Very useful 2. Overall, how important was the World Connect support to achieving your project goals? Circle your response. • Not at all important Somewhat important Important Very important 3. World Connect projects include a minimum 25% contribution from the community. How did the local community raise funds and resources for this contribution? Check all that apply. Local materials and/or supplies Local labor at low or no cost Local land or property Existing village development fund/savings New financial contributions from local businesses New financial contributions from beneficiaries Financial contributions from diaspora family members Interest from village savings and loans group Fundraising event in community Other donor/project funds Other (Please describe): _________________ Other (Please describe): _________________ I don’t know. 4. Please read the following statements and consider the whether you strongly disagree, disagree, agree, or strongly agree. Since implementing the project… a. I have improved my leadership skills. 40 b. I have improved my relationships with others in the community. c. I am better able to identify needs in the community. d. Decision-makers are paying more attention to the issues that I care about. e. There are more opportunities for people whose voices are not usually heard to participate in development. • Strongly Disagree Disagree Agree Strongly Agree 5. We would like to learn more about how you work with the project team. In the table below, please rank the overall strength of your relationship with each individual. Then, please indicate if your relationship has improved, stayed the same, or gotten worse since implementing the project. Do not fill out the row that refers to yourself. For example, if you are a local leader, leave the row for relationships with the local leader blank. Individual What is the overall strength of your relationship?. Has your relationship has improved, stayed the same, or gotten worse since implementing the project? Community organization Weak Moderate Strong Very Strong Improved Stayed the same Gotten worse Local leader Weak Moderate Strong Very Strong Improved Stayed the same Gotten worse Field partner Weak Moderate Strong Very Strong Improved Stayed the same Gotten worse 6. Please read the statements in the table below. In the first column, check the box if you think the theme was important to achieving the project goals. You can select as many or as few themes as you wish. If you think an important theme is missing, you may suggest your own ideas at the bottom of the table. Then, please indicate whether in general, you think this theme has improved, stayed the same, or gotten worse since you started implementing the project. • Community members come together to discuss community needs • Community members are confident in their ability to lead change • Community members respect each other • Community members trust each other • Community members work together to achieve shared goals • Community members ask for and provide help to each other • Community members hold each other accountable for development • Community members hold their leaders accountable for development • Different types of community members are engaged in the development processes, including women and youth • Other (please describe) 41 7. Of these themes, which are the 3 most important to achieving your project goals? If you think an important theme is missing, you can write in your idea. 8. Overall, how do you feel about your experience with the World Connect program? • Very Dissatisfied Dissatisfied Satisfied Very Satisfied 9. What about working with World Connect was different than your experience with other development projects, if anything? Check the box. You can select as many or as few options as you like. If you think a theme is missing, you can add it. Opportunity to define my own project goals Community engagement process Community financial and in-kind contribution World Connect support during project design World Connect support during project implementation Partnerships in the community for implementation Receiving and managing a grant directly Other (Please describe): 10. Please write any other comments you would like to share with us. 42 Annex C: Key Informant Interview Guide INTRODUCTION AND CONSENT: My name is [name] and I am working with World Connect to conduct a mid-term evaluation of their program in Malawi. Since 2017, World Connect has been providing small grants and other assistance for locally-led projects throughout the country with support from USAID. At this stage, we would like to understand what is working well, what can be improved, and what lessons we can share with the wider development community about this work. Your responses are confidential, meaning that we will not attribute any comments directly to you in our report and our discussions with World Connect and USAID. Nothing we discuss will be used to inform your current or future benefits from the project. Following completion of our evaluation, we will share the results back with you through [EFP name]. If you have any questions about this evaluation, you can contact one of the evaluation research directors: Dr. Wales Singini at the University of Mzuzu, or Diana Harper at LINC. I will leave their contact information with you. Do you consent to participate? Yes No PART I: PROJECT IMPLEMENTATION (45 MINUTES) 1. Please tell me about yourself. What are your roles and responsibilities? 2. When did you first hear about the WC project? What did you think about it at that time? 3. How was the project idea generated? Who was involved in this process? Did the idea exist before the call for proposals, or did the call for proposals prompt development of the idea? 4. Did you participate in the following processes with World Connect? If so, please tell me your impressions about it. a) Project Design workshop b) Project Management, monitoring and reporting workshop c) Project Site Visits d) Virtual support for implementation e) Any other type of support? 5. Tell me about the implementation of the project. What went well? What was challenging? 6. Tell me how women and other marginalized groups were involved? Did they take leadership roles? If so, how? 7. What are your overall impressions of working with WC? What is positive? What is challenging or could be improved? 8. Do you know that this project’s budget included a contribution from the community? a) Was the full contribution (25%) achieved? If No: Why not? b) How was the contribution generated? (in-kind, financial) c) What was difficult about generating the contribution? What was easier? d) What do you think about having communities contribute to financing of the project? e) Specifically, do you think there is a relationship between communities contributing their own funds and their ownership of the project? Between communities contributing their own funds and the sustainability of the project? Why or why not? 9. What has happened since the project has ended? Are activities continuing? Why or why not? 43 10. What type of benefits or changes have occurred, if any, beyond the direct goals of the project? PART II: SOCIAL NETWORK ANALYSIS [See SNA Tool] PART III: CONTEXT (20 MINUTES) 11. Is World Connect similar or different to other development projects in this community? Why? 12. How does this community identify what its development needs are? What formal and informal processes exist? What are the results of these processes? Who makes the decisions? PART IV: WRAP UP (5 MINUTES) 13. Do you have any other reflections and recommendations that you’d like to share with us? Thank you for your time. We look forward to continuing our evaluation and sharing the results with you. As I mentioned, if you have any questions in the meantime, you can contact Dr. Wales Singini. 44 Annex D: Social Network Analysis Interview Guide PART A—DEVELOP THE NETWORK: 20 MINUTES I’d like to understand what different members of the community you engaged with during the project. This could include people who participated directly in the project, people who provided support to make the project successful, people who emerged as important partners or stakeholders in the project, etc. For now, don’t include people who benefited from the project, unless they were also active in the design and/or implementation. On this piece of paper, please write your name in the center. First, we will make a list of the people who were involved in the project. Then, I will ask you a few questions about these people. Let’s start. Who did you engage with during the design and implementation of this project? Please make a list and tell me what type of organization or group they are from. It is ok to list more than one person from an organization. • G=government (district, regional, national) • LG=local government (chiefs) • LS=local structure (VDCs, ADCs) • LN=local NGO • IN=international NGO • A=school, university • P=business, private sector • F=church • Z=other citizen, neighbor, parent, etc. not affiliated with a specific group Who else? [Repeat as needed.] Now, I’m going to ask you a couple questions about your relationship with each person. Starting from the top of your list: 1. How frequently was this person involved in the project? o If they were frequently engaged, write their name in a circle close to yours. o If they were not frequently engaged, write their name in a circle farther away. 2. Was this a new relationship? Meaning, you didn’t know or collaborate with this person until the project. o If yes, use this RED marker to draw a line from your name to their name. o If no, use this BLACK marker to draw a line from your name to their name. [Repeat.] PART B—IDENTIFY UNHEARD VOICES: 5 MINUTES 3. Now, let’s look at the map as a whole. Is there anyone on this map who does NOT usually have an opportunity to participate in community development? If so, please circle their names. PART C—REFLECTION: 5 MINUTES 4. What does this map mean to you? 5. What is the significance of these relationships going forward? 45 Annex E: Community Workshop Outline 1. INTRODUCTION AND CONSENT: (10 minutes) 2. ICE BREAKER (5 minutes) 3. SMALL GROUP WORK ON DEVELOPMENT ISSUES (15 minutes) [FLIPCHART 1] 1. What are the most important development issues in your community? 4. SMALL GROUP WORK ON ADDRESSING DEVELOPMENT ISSUES (30 minutes) [FLIPCHART 2] 2. What is needed to address these issues? From OUTSIDE the community From INSIDE the community 5. SUMMARIZE RESULTS AS A GROUP (15 minutes) 6. COFFEE BREAK (15 minutes) 7. ASSEMBLE PRIORITY THEMES AND VOTE ON EACH THEME (10 minutes) [FLIPCHART 3] Theme: Getting Better About the Same 46 Getting Worse Why is this theme getting better / about the same / getting worse? 8. PARTICIPANTS CHOOSE A THEME FOR DISCUSSION (30 minutes) 9. WRAP UP (20 minutes)