EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 i USAID/DRC INTEGRATED YOUTH DEVELOPMENT ACTIVITY (IYDA) FINAL PROGAM PERFORMANCE EVALUATION DO Number and Name: Cooperative Agreement 720066078CA00003 Activity Start and End Dates: April 6, 2018 to September 30, 2022 Total Estimated Cost: $27.5 million AOR: Pascal Tshimanga, USAID/DRC/Education Implemented by: Education Development Center, Inc. Partners: FHI 360, Catholic Relief Services EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 0 Table of Contents ACRONYMS........................................................................................................................................1 FIGURES………………………………………………………………………………………………………………………………………………2 TABLES………………………………………………………………………………………………………………………………………………..2 PART I. INTRODUCTION .................................................................................................................3 1.1. Positive Youth Development Framework and Establishing a Foundation for Resilience ..............................3 1.2. Consortium Composition and Roles in Implementation.............................................................................5 PART II. METHODOLOGY.................................................................................................................5 2.1 The Evaluation Criteria ............................................................................................................................5 2.2 Data Tools & Collection ...........................................................................................................................6 2.2.1 Primary Data Collection .............................................................................................................................. 6 2.2.2 Data Entry and Processing .......................................................................................................................... 7 2.2.3 Secondary Data Review .............................................................................................................................. 8 2.3 Limitations..............................................................................................................................................8 PART III. EVALUATION RESULTS........................................................................................................8 3.1. OECD Evaluation Criteria Applied to All Intermediate Results....................................................................9 3.1.1. Relevance – Is the Intervention Doing the Right Things?........................................................................... 9 3.1.2. Coherence and Cohesiveness – How Well Does the Intervention Fit? .................................................... 10 3.1.3. Effectiveness – Is the Intervention Achieving Its Objectives? .................................................................. 11 3.1.4. Efficiency – Limitations and Timeliness – How Well Were the Resources Used?.................................... 13 3.1.5. Impact – What Difference Is the Intervention Making? ........................................................................... 14 3.1.6. Sustainability – Will the Benefits Last? ..................................................................................................... 16 PART IV. CONCLUSIONS AND ADDITIONAL RECOMMENDATIONS ....................................................18 REFERENCES ....................................................................................................................................20 EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 1 Acronyms ALP Accelerated Learning Program BYOB Be your Own Boss CAP Centres d’Apprentissage Professionnel CDCS Country Development Cooperation Strategy CLA Collaborating, Learning and Adapting COP Chief of Party CRS Catholic Relief Services CYM Community Youth Mapping DCOP Deputy Chief of Party DIVAS Division des Affaires Sociales (Division of Social Affairs) DIVIJEUNESSE Division de la Jeunesse (Division of Youth) DNH Do No Harm DRC Democratic Republic of Congo ECCN Education in Crisis and Conflict Network EDC Education Development Center, Inc. ERSA Education Recovery Support Activity GBV Gender-based violence GIS Geographic Information System GRDC Gouvernement de la République Démocratique du Congo ICT Information Communications Technology IYDA Integrated Youth Development Activity M&E Monitoring and Evaluation MEL Monitoring, Evaluation and Learning MIS Management Information Systems MoE Ministry of Education OECD Organization for Economic Co-operation and Development PIRS Performance Indicator Reference Sheet PMP Performance Management Plan PYD Positive Youth Development RCT Randomized control trial RERA Rapid Education and Risk Analysis RISD Research Initiatives for Social Development SILC Savings and Internal Lending Communities SPSS Statistical Package for Social Sciences VC Value chain WBL Work-Based Learning WRN Work Ready Now YDA Youth Development Alliances EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 2 Figures Figure 1: USAID PYD Measurement Toolkit...................................................................................................4 Figure 2: OECD Better Criteria for Better Evaluation.....................................................................................6 Figure 3: Female participants vocational tracks pursued............................................................................15 Figure 4: Male participant vocational tracks pursued.................................................................................15 Tables Table 1: Distribution of actors in individual and group interviews................................................................7 Table 2: Distribution of focus groups by site and by actor ............................................................................7 EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 3 Part I. Introduction The USAID/DRC Integrated Youth Development Activity (IYDA) was a three-year U.S. Agency for International Development (USAID)-funded program that was extended to four years. IYDA was implemented by Education Development Center, Inc. (EDC), and a consortium of partners: Catholic Relief Services (CRS), Family Health International 360 (FHI 360), and for a time, Souktel. The primary objective of IYDA was to support vulnerable youth in developing basic assets for resilience, as well as to capitalize on emerging socioeconomic opportunities. IYDA’s strategy was to create inclusive learning and livelihoods pathways for vulnerable youth and to develop youth-serving networks to support those pathways and inclusive economic opportunities. These learning pathways and economic opportunities were reinforced by systems actors and support services, such as tutoring and mentoring. The activity was implemented in the Provinces of North Kivu (NK) and South Kivu (SK) of the Democratic Republic of the Congo (DRC), specifically in the urban cities of Goma, Beni and Butembo (NK), and Bukavu (SK), and in the rural South Kivu Territories of Kalehe, Kabare, Walungu, and Idjwi. The project worked with existing structures—education and vocational training centers, government officials, the private sector, and local youth-serving organizations. In particular, IYDA worked with accelerated learning programs (ALPs or Centres de Rattrapages Scolaires [CRSs]), vocational training centers (Centres d’Apprentissage Professionnel [CAPs]), community literacy programs, and youth-led and youth-serving organizations (YLSOs). The purpose of this study is to provide an evaluation of the overall program performance. This performance evaluation seeks to (1) determine if IYDA succeeded in obtaining its program objectives and (2) make some recommendations for future programming. 1.1. Positive Youth Development Framework and Establishing a Foundation for Resilience The overarching activities in the activity were designed to help foster resilient youth who could join the workforce and/or contribute to their community’s well-being. Youth enrolled in the activity were provided with theoretical and practical skills to improve their own lives, but they also engaged in a series of community-based activities that positioned them to contribute to their community and (trickle up) toward the national development objectives. The intervention was designed to support USAID’s Positive Youth Development (PYD) Framework as a means to guide the intervention to work with and for youth to “improve their Assets, Agency, Contribution and Enabling Environment.” Figure 1 illustrates the framework. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 4 Figure 1: USAID PYD Measurement Toolkit1 The IYDA’s theory of change stated the following: “IF youth gain relevant skills and have opportunities for social and economic engagement, and IF community stakeholders are supportive of youth efforts, THEN youth will be more resilient to the impact of conflict and violence.” 2 This theory of change reflects alignment with the PYD Framework, addressing the transformation of the structures and enabling environment needed to foster and encourage youth resilience. IYDA objectives were achieved by pursuing the following three intermediate results (IR): IR1: Youth gain and apply relevant skills for improved educational and social outcomes. Under IR1, the consortium strengthened and provided the principal nonformal education service providers in the Kivus with teaching and learning materials as well as professional development investments. These were divided into three pathways: Pathway 1 – Accelerated learning programs CRS/ALPs Pathway 2 – Vocational training schools or professional training centers (CAP) Pathway 3 – Community-based literacy programs and community-based organizations (CBOs) IR2: Youth are positively engaged in community activities. Interventions under IR2 strengthened youth participants’ social-emotional and life skills by affording them safe spaces, opportunities for belonging and agency, social networks, and supportive relationships with adults to help them become drivers of and actors in community-building initiatives. 1 Youth Power Learning, Hinson, L, Kapungu, C., Jessee, C., Skinner, M., Bardini, M., Evans-Whipp, T. (2017). Power youth development measurement toolkit (p. 22). https://www.youthpower.org/positive-youth-development-pyd-framework 2 IYDA Program Document, Cooperative Agreement between USAID and EDC, Inc. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 5 IR3: Youth are engaged in economic activities. Interventions under IR3 focused on livelihoods training and the transition to entrepreneurship or the workforce to provide youth with basic resilience, adaptation, and market-driven skills, all of which can assist youth in times of stress; to identify immediate opportunities; and to provide a foundation for future prosperity. At the end of Year 3 of IYDA, which had a total budget of $22.5 million USD, a 12-month cost and time extension was allocated to the activity, providing an additional funding of $5 million USD. The total duration of the activity, which also benefited from a six-month no-cost extension before the cost extension, was therefore four years and six months with a budget of $27.5 million USD. The extensions were granted in large part to the project as a means to mitigate some of the negative impacts that the COVID-19 pandemic (2020-2022) and the Mount Nyragongo volcano eruption (May 2022) had on the activity and to provide additional resources to focus on promoting sustainability through the creation of lasting relationships among the youth, nonformal education institutions, the private sector, and the community. 1.2. Consortium Composition and Roles in Implementation IYDA was implemented by a consortium with EDC as the project prime (the lead). FHI 360 and Catholic Relief Services (CRS) were the international subawardees with technical roles and implementation responsibilities. Souktel was also a subawardee but closed as an organization around the time that their project tasks were completed. While AVSI was noted in design documents as a back-up solution to work with the CRS and livelihood activities, they were not engaged in the project. RISD, a Congolese data collection firm, was identified as a local resource partner and supported the IYDA with monitoring and evaluation tasks. With regards to the roles and responsibilities on the technical aspects of the project, EDC was largely responsible for IR1 and IR3. FHI 360 drove IR2 interventions, and CRS supported IR3 interventions through the creation of savings and internal lending community groups (SILCs). Part II. Methodology This performance evaluation used a light touch approach and focused on the macro level of IYDA’s performance and achievements to qualify its overall performance over the four-year activity period. 2.1 The Evaluation Criteria This evaluation used the criteria of the Organization for Economic Co-operation and Development (OECD) to examine IYDA’s overall performance. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 6 Figure 2 : OECD Better Criteria for Better Evaluation3 The OECD criteria was applied from different perspectives to evaluate the overall success of the consortium. Specific focus was paid to how the project was able to apply the PYD Framework to the IYDA design, implementation, and impact and results. Using the criteria, the evaluation asked, “Was the EDC program and activities able to help youth achieve resilience or self-reliance?” 2.2 Data Tools & Collection Two approaches were used to carry out this evaluation: a literature review of the activity documents and a qualitative data collection, conducted in the form of focus groups and semi-structured interviews with participants, staff, stakeholders, and the donor. Data was collected in South Kivu, including Bukavu and the rural areas of Walungu, Katana, Kalehe, and Minova. In North Kivu, the study areas were Beni, Butembo, and Goma. The literature review was conducted reviewing both project documents and studies. Part of the literature review also included conducting general research about other education programming (formal and nonformal) and Congolese rules and national objectives around education and regional objectives for economic development. The project shared the activity design, baselines, evaluations, and the curriculum with the consultant team. These documents were reviewed by the lead consultant, discussed with the field team, and integrated into the analysis and reporting. 2.2.1 Primary Data Collection The tools developed to collect data from evaluation participants included qualitative interview and focus group questionnaires. There tool design was informed by EDC HQ staff (the Monitoring and Evaluation [M&E] Lead and the Program Director), but they were designed and finalized by the consultant team, led by consulting firm Green Action Development. The score system using the OECD criteria was developed by Green Action Development based on the OECD approach. 3 Organisation for Economic Co-operation and Development. (2020, February). Better criteria for better evaluation. https://www.oecd.org/dac/evaluation/evaluation-criteria-flyer-2020.pdf EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 7 During the field visits, information was gathered via the following methods: • Individual Interviews with different groups of actors: school principals and teachers, mobilizers, SILC supervisors, project partners (non-governmental organizations), donors, young entrepreneurs, parents of young people, young employees, etc. (Table 1). In total, 62 interviews were conducted. • Focus groups were held with youth from SILC, CRS, and CAP pathways who had completed a pathway that supported transition to the workforce. They were graduates from Cohort 1 and Cohort 2. For each discussion, 10 youth were gathered each time. Table 2 shows the distribution. Table 1: Distribution of actors in individual and group interviews Sites School Director s Teachers EDC Staff IR1, IR2 & IR3 FHI Staff & Part ners EDC Staff for IR2 Self￾Employ ed Youth Employ ed Youth EDC Partn ers for IR2 EDC Partner IR2 Mobili zers & Superv isors Youth Parents & Employ ers Public Service DIVAS Donor USAID Total Bukavu 3 4 2 4 2 2 3 3 2 25 Goma 2 3 2 1 1 2 2 1 14 Katana Kalehe 1 1 2 2 6 Walungu 1 1 Minova 1 1 1 3 Beni (phone) 1 1 3 5 Phone 3 4 1 8 Total 7 9 8 4 1 4 3 3 4 8 7 3 1 62 Table 2: Distribution of focus groups by site and by actor Sites Youth CAP Youth CRS Youth ONG Youth SILC Youth YCA Total Bukavu 1 1 2 Goma 1 1 1 3 Kalehe Katana (Kabare) 1 1 Walungu 1 1 Total 2 1 2 1 1 7 In addition, the evaluation team engaged in discussions with FHI partner associations under the IR2 component, which included the following: Action Humanitarian pour la Sante et le Developpement (AHUSADEC), Action pour le Bien etre de la Femme et L’Enfant au Congo (ABFEC), Reseau des Associations autochthones Pygmes (RAAPY), and Generation Epanoui (GEP). The team also engaged with three community mobilizers in Bukavu. 2.2.2 Data Entry and Processing A data entry grid was provided in Word for each interviewer to record information noted in the field. The information recorded in the field was supplemented by audio recordings so that the interviewer could be sure that all of the information provided by the respondent had been accurately recorded on the grid. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 8 The content analysis was carried out to identify the main trends according to region, gender, route, and pre-established sampling criteria. Data processing was carried out by combining youth data (focus groups and interviews) as appropriate and information from the literature review. 2.2.3 Secondary Data Review As the evaluation budget and time frame were limited, and as the study was designed to be qualitative, much of the quantitative information and data were provided by IYDA and the project Monitoring, Evaluation, and Learning (MEL) Director. The data provided by the project were cleaned and treated, and at times disaggregated, by year or gender etc. Upon request, some of the raw data was shared on Excel spreadsheets. This permitted the evaluation team to cross check quantitative indicators or targets (number of youth employed, etc.) and to make visuals, graphs, and tables presented in this report. 2.3 Limitations The greatest challenge to conducting and completing this evaluation was the limited availability of time. During the first week after the contract signing in Goma, Beni, and Butembo, the towns experienced massive disruptions and violence by the population toward MONUSCO, forcing the shutdown of the cities. This disruption impacted the project’s ability to hit the ground running regarding data collection. Nonetheless, by the Thursday of that same week, data collection had started. These constraints, however, resulted in the decision to no longer physically visit Beni and Butembo, and thus interviews were conducted by telephone. It is important to also note that interviews and data collection on the ground are often written in French (translations and reading take more time). It should also be noted that the interviews with Beni and Butembo only occurred during the week of August 15–18 (when the actual report was due) as the local partners requested that we interview them after the manuals, kits, and some activities were completed with EDC. Their feedback was limited in some of the questions—especially those on the scorecard- as they did not feel like they had enough time to evaluate the activity or its impact on them yet. For Year 4 participants in Beni and Butembo, kit distributions occurred after most of the data collection for this study was completed. Thus the timing of the interviews was not ideal. The quality of this evaluation has been impacted by the initial explanation of the type of report desired, that is “a light touch overall program evaluation,” and by not building in sufficient time for an external consultant to absorb all of the details. IYDA was a complex multi-year intervention with multiple moving parts, players, and locations as well as layered interventions. To truly grasp the activity, and to fully evaluate the nuanced impact (as opposed to general overall impact) of the activity, more time should have been given. Part III. Evaluation Results IYDA accomplished the majority of its objectives in part or in full within the life of the activity. The partners and populations where the activity was implemented appreciated the original design and flexibility to make changes in the activity as it advanced or encountered obstacles. For example, such flexibility allowed the activity to improve the curriculum by translating it not only into French but also into Swahili and even adding curriculum when the need was highlighted. The IYDA also demonstrated flexibility and a desire to improve with each revision, as shown in the following example: After the first market study and assessment, the project’s recognized that the results did not show sound livelihood options for the youth. Coupling this with the recognition of youth capacity and ability to conduct their EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 9 own labor market analyses, the project successfully piloted and then scaled a youth-driven local labor market analysis that produced tangible results in the form of a business catalogue for interested youth. In this vein, the project was successful as it continually molded itself to meet the demands of youth and the evolving opportunities that accompanied a project that was able to invest multiple years of work with key youth service providers. 3.1. OECD Evaluation Criteria Applied to All Intermediate Results 3.1.1. Relevance – Is the Intervention Doing the Right Things? Overall Design IYDA was designed within USAID’s objectives and is found to have responded the strongest to youth beneficiary needs through curriculum and job transition support. The activity aligned with policy for the Government of Congo and the donor. Its contribution to curriculum for Level 3 ALPs was recognized and validated by the Government of the DRC. However, the adoption of the workforce preparedness program curriculum was not within IYDA’s control. IYDA documents and assessments could have done more to assess the power dynamics within the structures and communities in which they were working to mitigate negative externalities that could have been foreseen, especially surrounding teachers. This is important to highlight because teachers are marginalized and exploited within the existing education system, which is fragile and disjointed, especially within the alternative education system. While the activity did provide training to the teachers, it did not provide stipends to teachers or subsidize school fees in any manner. At the same time, the activity trained and provided a stipend to volunteer tutors, youth who provided after-school learning support to youth. However, as the tutors took on an increasing amount of responsibility— especially during the COVID-19 pandemic when program staff could not visit field sites—and given that the stipend was two to three times higher than what teachers and trainers earned, tension arose between the teachers and the volunteer tutors. While engaging youth as tutors contributed to the activity objectives, this setup undermined the social hierarchy that existed in these areas. However, the project did recognize the issue and addressed it in the last year of the project when it transitioned to a system of community mobilizers whose greater role and responsibilities were more clearly communicated to the community. IR1: The life skills curriculum Work Ready Now (WRN) and entrepreneurship training Be Your Own Boss (BYOB) were relevant and complementary to the technical curriculum for CAPs. WRN was relevant and contributed to CRS students’ final exam preparation and transition to the formal education system. The SILCs were highly relevant and contributed to financial literacy and entrepreneurship for all youth who were members. With regards to the material support IYDA provided to the CRS and CAP schools, while it was aligned with a School Improvement Planning (SIP) exercise, future projects should do more in guiding the schools to select materials and equipment that support priority vocational tracks. For IYDA, this was a challenge for a number of reasons, including poor basic infrastructure of the schools and outdated teaching approaches. While resources that the project was able to provide—furniture, basic teaching materials, and limited equipment—did have a positive impact on the learning facilities, the value, the manner of procurement, and the disbursement mechanisms did not allow the activity to support what could have actually been more relevant to these schools, which was equipment for their vocational EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 10 trainings. It is extremely important to help the students truly master their vocational track. As a note, however, the provision of more modern equipment would need to be accompanied by updated course curriculum and training for service providers. The project responded in part to the needs of the nonformal education sector by helping to build a foundation of basic knowledge and life skills to enhance the current courses. However, the actual national education system (both formal and nonformal) from Kinshasa outwards and the trickle down of budgets, policies, and support remained an obstacle that IYDA could not address through this program, but which did negatively impact its overall success and sustainability. IR2: Civic engagement and community development activities organized through the activity effectively developed and strengthened community ties and solidified youth’s positive role in the community. Although these activities were needed and highly relevant, they did not provide solutions for income generation for youth participants. While the project did endeavor to layer the approach with other IR areas, there was unrealized opportunity to better leverage the link between community interventions that supported income generation for youth and youth graduating from one of the project’s pathways. IR3: The project guided youth in the selection of their professional track or trade. In other words, it did not impose choice on participants, something that built their sense of independence and drive. The income-generating activities and professional tracks chosen by the young people were based on their preferences and their understanding of the job opportunities in the community. Supporting youth in selecting and pursuing their own pathways contributed to the project’s relevance. 3.1.2. Coherence and Cohesiveness – How Well Does the Intervention Fit? Overall Design The activity was designed so that the curriculum would be adopted by the national education system for integration into the nonformal education system and then eventually positioned to be used at scale. The project succeeded in getting the curriculum adopted. It was developed in coordination with the DIVAS and DIVIJEUNESS4 regionally and validated by the relevant ministries at the national level. At the provincial level, division inspectors were trained to become master trainers, positioning them to provide continued support to schools who adopted the curriculum. National scale is thus possible, but rollout requires additional resources in a stronger education system, something that is outside of IYDA’s purview. IR1: Curriculum was complementary to the existing curriculum and/or improved on existing curriculum. Evaluation participants—especially those in the education system—indicated that they would have liked all teachers and school administrators at each participating school to have benefited from the course materials and training. This finding brings into question how well the cascade model worked, as normally trained school directors and teachers should have been able to share their knowledge and materials with colleagues. There was also mention of frustration on how training and tutors, volunteers, and mentors fit into the existing education system strategy. IR2: Youth community interventions that promoted culture and community, such as dance and music, were supported by IYDA and seen as contributing factors to building cohesion in an inclusive manner 4 DIVAS stands for Division des Affaires Sociales (Social Affairs Division), and DIVIJEUNESS stands for Division de la Jeunesse (Youth Division). EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 11 that resonated with a diverse group of stakeholders. The activity’s ability to invest in cultural events and support more artistic endeavors that celebrated community and tradition was considered to have added value to the intervention. IR3: With the very high rates of unemployment, especially among the youth population, support for transitioning to the workforce was very relevant and needed. However, in the first labor market assessment, the jobs opportunities and private sector partnerships identified did not match the vocational tracks pursued by their beneficiary population. Furthermore, the salaries were on the lower end. A more appropriate analysis of the regional and national economies and markets could have directed the activity, learning facilities, and students toward sectors that (1) had significant amounts of public and private funding (e.g., energy, electrification, and solar installation); (2) had a high need for technical skills that these schools could, or should have offered; and (3) could have contributed to community development and both community and individual resilience. However, supporting local CAPs and CBOs in offering these skills training would have taken more resources and a larger mandate than what was allocated to IYDA. Modifications, Adaptions, and Recommendations While the curriculum was adapted for each group (CRS, CAPS, etc.) and for accelerated learning, it was too accelerated. One respondent speaking about CAP students noted, “For the young people with great capacities, it was adapted, but for the average and weak young people the duration was too short to allow them to understand the training. The older youth (around 22 years of age) were more adapted to learn the material than the younger youth (around 16 years of age.” Over the course of the project lifetime, some modifications and revisions were made that contributed to uptake. These included providing materials in Swahili, extending the training time, working with CRS and CAPs to sequence the training, and providing refresher trainings to teachers and trainers, all of which contributed to improved uptake and understanding. The addition of tutors available to provide support for the workforce development curriculum and the availability of business mentors to support business planning and start-up also allowed for increased uptake (understanding) for the students, although it also created some duplication in roles between the teachers and tutors. Overall, the activity was very coherent within the USAID-funded education activities. 3.1.3. Effectiveness – Is the Intervention Achieving Its Objectives? Overall Design IYDA responded to the nonformal education system’s needs in terms of improved competency-based curriculum, professional development, and the distribution of teaching and learning materials to traditionally underserved nonformal schools. The sole focus on CRS Level 3 was felt by many respondents to be a weakness, who recommended that the modules should be provided at all levels. Thus the link between the ACCELERE investments in Levels 1 and 2 could not be established in this review. The activity was effective in obtaining its objectives. The intervention results were largely achieved across groups. According to the IYDA’s indicator tracking table, out of 19–20 targets, the activity surpassed the majority of them. This does not, however, mean that the target was sufficient or adequate to meet the needs of their beneficiaries. For example, IYDA had a target of distributing 19,022 teaching and learning materials, but by 2021, IYDA had distributed more than 30,312 textbooks, surpassing their target and achieving 157%. However, these numbers do not actually illustrate a EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 12 comprehensive consideration of effectiveness, which would have included the question, Does every student and admin have all the textbooks that they need to teach or learn? While the project distributed tens of thousands of textbooks, learning conditions were not conducive to the conservation of materials over long periods, and thus schools reported a continued need for textbooks during the evaluation. The project notes that as a result of the school closures during the pandemic and the displacement from the volcanic eruption in Goma, a significant number of teaching and learning materials were lost. IR1: IYDA added a number of teacher’s manuals to the schools. This access for the teachers has led to good quality of teaching, as well as to improvements in the school climate and environment. The activity encountered initial delays in the distribution of manuals due to the time it took to adapt the curriculum. However, the manuals and these adaptations were highly appreciated and impactful on the testing and learning uptake. This IR was largely considered a success and effective by both the participants and the activity. The in-kind materials provided to schools to support their improvement plans was a solution to help address some of the technical or organizational needs for each school or learning facility. IYDA realized that basic equipment, such as a computer and teaching materials (besides the manuals) like chalk pens etc., was often lacking in the supported structures. IYDA used small grants as a solution to reduce some of these ongoing small burdens that the administration faced. The small grants program could not, however, address bigger systemic issues, such as staff salaries. IR2: According to the IYDA’s indicators targets table, the activity evaluated that they were the least effective in this IR, achieving a nominal percentage of many of the IR2 targets. They were most effective on improving youth perception. While they helped establish or strengthen youth development alliances, they did not seem to work as effectively in achieving their targets for strengthening systems that served vulnerable populations. There were also a number of hiccups in the material distribution that was designed to support youth community activities, such as providing great quality musical equipment, like an electric guitar, but then not providing the energy source. IR3: The materials provided to learners and grants given to schools were appropriate as based on our participants own requests. However, the effectiveness of the material support provided to youth starting a business depended on the geographic areas. Youth from disadvantaged areas were more successful with their entrepreneurship endeavors because they generally seemed to face less obstacles in carrying out their activities. On the other hand, young people from the city had more business￾associated expenses, including higher rent and taxes, which reduced the effectiveness of the kits. Modifications, Adaptions, and Recommendations Procurement appears to be the largest obstacle to achieving effectiveness encountered IYDA. The procurement and distribution of the manuals was delayed in Year 1 due to the adaptation process. While the activity managed to distribute the manuals in a timely way in the following years, the CRS students, who had previously been given manuals to keep, were no longer provided these manuals as the manuals were allocated to schools rather than students. Procurement of the business start-up kits was also extremely challenging. The activity attempted both direct procurement and the use of vouchers, as well as direct cash distributions (for SILC members). Each approach presented its own opportunities and challenges. From a program perspective, the use of vouchers was the most efficient. Surveys indicated that the communication around the kit distributions was not always well understood by youth. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 13 3.1.4. Efficiency – Limitations and Timeliness – How Well Were the Resources Used? Overall Design The activity as designed was efficient. The consortium’s approach was relatively flexible and adaptable, pivoting to meet the constraints of the COVID-19 pandemic and the May 22, 2021, volcano eruption in Goma, and benefiting from the extension periods. However, the duration of the 12-month cost extension, during which time the project was to expand programming both geographically and thematically (introduction to English and the YSYLO Advocacy manual), was insufficient to ensure quality rollout, implementation, and follow-up of the new initiatives. IR1: The duration of the learning cycles was short. Both students and teachers at the CAP and the CRS could have benefited from longer training cycles. While the ALP curriculum was strong, some of the topics were new to both teachers and students and required more time for uptake. To support teachers’ adoption of the materials, the project held regular refresher trainings, but these could have been longer, especially for a population with high levels of mobility, which resulted in the participation of many previously untrained teachers. Despite IYDA support, schools’ poor infrastructures were challenging. Site visits by the evaluation team noted the CRS and CAPs as marginalized run-down institutions, especially when run by the State. For example, the CRS in Minova is housed in a primary school where the CRS director is given a small room behind the primary school director’s office to store his computer and school material (provided by the IYDA). The CRS teaching and learning hours take place after the primary school classes have finished for the day, and they are in the same classrooms. The CRS pays rent to the primary school for the use of space, creating a kind of hierarchy between the two systems, with neither of them willing to take responsibility for property and classroom maintenance. In another CRS visited, the building was made entirely of wood and consisted of two floors, the second of which felt extremely unstable. Here the classrooms were dusty and filled with used paper and trash. The existing infrastructure, such as desks and posters, were provided by the USAID/UKAID UNICEF/IRC Program. The assistant director was able to clearly identify which items and materials were provided by IYDA versus UNICEF, and there was no duplication as IYDA provided curriculum and materials that were complementary to the other project’s donations. Photos 1, 2, 3, 4 (left to right): CRS Minova, an oven at Caps Minova, a sewing machine, and literacy classes at CRS/CAPS Badirika IR2: This was the most complex IR to evaluate as the impact was not only on the individual but also on the community. Additionally, the extension period provided opportunity to build out activities under this IR. The results of FHI’s intervention seem somewhat effective, but may not have been considered efficient by EDC as they elected to develop their own material over continuing with FHI’s system in Beni and Butembo during the extension. EDC designed its own approach and materials to support YSYLO in community advocacy and activism efforts. The evaluation found that EDC’s materials were appreciated EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 14 for their quality. However, as noted above, IYDA should have been provided more time to implement all of the activities planned for the costed extension period. The Young Women and Adolescent Girls Mentoring Pilot was effective and highly appreciated by all participants, but it was time consuming for the mentors, who volunteered their time and donated their resources (transport, participation and calls) to the pilot. While this reduced efficiency, IYDA succeeded in implementing a mentorship program with vulnerable populations and leaders on a purely voluntary basis. As the goal was to provide support to the mentees, and when the mentors were willing to give this time, the pilot was a value added irrelevant of the inefficiency. IR3: The youth-led local labor market assessment piloted was more efficient and effective than the labor market assessment conducted by the international consultant. The start-up kit procurement and distribution to youth on the entrepreneurship track suffered from inefficiency during the last year when EDC shifted to a centralized regional procurement approach (vs the use of a voucher system that supported youth choice and developing local markets). Future programming should carefully consider how business plans are approved. Approval of the business plans—a pre-requisite to receiving the start-up kit—was decentralized to mentors, tutors, and program representatives during the last year. This approach provided the project with limited oversight of the process and product, and contributed to potentially negative power dynamics between the youth and the mentor. 3.1.5. Impact – What Difference Is the Intervention Making? Overall Design The activity did have a positive impact on its participants, as well as future students passing through programs offered by CRS, CAP, and CBOs, who partnered with IYDA and who have decided to continue to integrate the curriculum into their teachings. As the overall impact of the activity has been the primary topic of discussion in the majority of this document, this section will be brief. IR1: The consortium and EDC were particularly impactful with IR1 and the activities therein. The IYDA beneficiary feedback, gathered throughout the life of the project and then during the data collection for this study, indicated an overall high level of satisfaction with IYDA contributions to both the learning facilities and the regional entities that supported them (i.e., DIVAS and DIVIJEUNESS). Trainers who were interviewed indicated that they not only taught the students the lessons that they were trained on, but they also applied the Be Your Own Boss and Work Ready Now in their daily lives and activities. Investing in the DIVAs and DIVIJEUNESS created the opportunity to scale the intervention to new schools. IR2: This IR was impactful in material and subject matter, especially with youth and especially with regards to supporting sports and dance. Musical activities had some issues because of the equipment; however, the sports improved community interaction and contributed to greater social cohesion between youth and elders. IR3: For those who received the complete start-up kit, IYDA was impactful. For those who did not, they still learned how to write a business plan and were given the life and basic business skills necessary to manage a business or be an optimal employee. These are skills that are not traditionally taught in either formal or informal education. The SILCs were successful in helping youth become financially literate. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 15 During the period that the evaluation data was collected, 42.8% of CAP students (of a total of 1,838 students) were on one vocational track—sewing. While sewing creates economic opportunity, it can be extremely limiting, especially when the market is saturated, and the negative impact can have a ripple effect. Future projects should do more in terms of orienting CAPs and CBOs on the type of trade courses offered and in supporting youth to invest in more promising (or less saturated) sectors. While sewing and tailoring were the top vocation pursued by females, it was a positive sign to see a higher percentage of females engaged in vocations that were considered to be more traditional male vocations, such as welding (21%) and carpentry (9%). It was not clear from the data provided what type of livelihoods activities those youth engaged in the SILCS pursued, but the situation should be the same. Figure 3 : Female participants vocational tracks pursued There were a number of results in this study indicating declining rates of participation among males. This is not only in the proportion of males to females attending the technical schools and learning facilities but also in the number of males versus females on job retention. Figure 4: Male participant vocational tracks pursued The chart illustrates the vocational tracks taken by females attending CAPS/technical schools. At 35%, sewing remains a top vocation for females. However, the chart also shows growing interest and investment by females in higher paid vocations. This study highlights a few trends that are also reflected globally, like higher numbers of females versus males being educated and increasing their technical diversity. While it is very positive to see female participation in other vocations, it is important to also see increasing numbers of males engaged in higher￾skilled and higher￾paying vocations. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 16 Given the limited value of the start-up kit (up to $150), IYDA could have considered encouraging youth to form groups to invest in joint business ventures as a means to leverage resources. Such ventures could be encouraged especially for trades that have more expensive start-up costs due to the price of basic materials and equipment. This community-based approach can create an added layer of complexity, but it can also create positive learning experiences, such as sharing a workshop, or promote specialization of trades among the group members. Modifications and Adaptions The transition of the Lead in IR2 from FHI to EDC in Beni and Butembo during Year 4 allowed for additional quality curriculum to be developed. However, the 12-month extension period did not provide enough time to implement the a comprehensive civic engagement intervention that included accompaniment and would contribute to achieving the maximum impact. 3.1.6. Sustainability – Will the Benefits Last? Overall Design IYDA’s aim was that sustainability would be obtained by having the national Ministry of Education formally adopt the curriculum and integrate it into the national and regional nonformal education systems. But this did not occur due to the Government’s consistent failings at managing the education system. IYDA’s lasting or overarching objectives of promoting youth learning to mitigate violence should be sustainable, but it can only be measured over time. IR1: The curriculum developed is sustainable and scalable. IYDA left some manuals and a pool of master trainers. By leaving soft copies, they can ensure that these institutions have access to the curriculum into the future, which can be supported by master trainers. Some of the learning facilities visited indicated that they plan to maintain their use of the IYDA curriculum. For example, CAP Anna Micheli decided to integrate the IYDA curriculum into its national program, so that the curriculum would become an integral part of its basic training program. It should also be noted that currently all the centers throughout the country belonging to the congregation of Anna Micheli’s followers have decided to integrate these trainings into their respective programs. At the moment, other IYDA partners (CAPs, CRS, CBOs) are calling on the expertise of the Anna Micheli staff to provide the same training. It is not clear if actual safe spaces were created during the life of the activity. The learning facilities could be considered safe spaces and were improved through professional development, increased collaboration with local institutional inspectors, and the receipt of equipment and materials through in￾kind grants. However, site visits showed many continued to lack actual clean and safe environments with sufficient materials and equipment. IR2: Where FHI and EDC worked with the same local partners, such as La Colombe, interviews suggested that partners felt they received adequate support. However, a multi-pronged approach was not always feasible given competing priorities and limited time. In some cases, diversifying the partner pool between the two partners expanded the projects investments. IR3: Youth who become entrepreneurs had a high level of sustainability—continuing their businesses and generating revenue. It appears that those who entered the workforce via the SILCs had more capacity to continue to strengthen their entrepreneurship activities, but all avenues pursued that offered income-generating opportunities were appreciated by the participants. Job placements and using the activity’s network were less effective and less sustainable than the kits and the SILCS. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 17 Early on, EDC was able to identify potential programs, enterprises, industries, and value chains where youth could find economic opportunities. On paper, these types of collaborations sounded easy and seemed a natural fit, but in reality, with conflict and shifting program dynamics all with their own constraints, they often became difficult-to-accomplish activities. Furthermore, many of the activities identified were focused on agricultural value chains, which were not really an ideal partnership as most of the skills that the students and youth were pursuing were technical trades and not related to agriculture. Only 82 youth out of more than 12,000 youth received agriculture-related start-up kits, according to the data provided by the activity. This small number could be because IYDA was able to offer entrepreneurship and employment that promoted trades or because this population had very little interest in agriculture. Supporting wage employment efforts was not successful given the informal nature of the market and economy. In general, in the DRC, wage employment is poorly regulated and paid. As a result, and as the interviews often indicated, it is easier, especially when paid so little, to pursue entrepreneurship if the means for initial investment exists. The discussion with the young people employed in the companies revealed that despite their presence in the employment structures, they were still motivated by the desire to work in their own companies. They said that BYOB and WRN prepared them to take care of themselves in an autonomous way (Photo 5). Photo 5: CAP graduate and recently employed carpenter Modifications and Adaptions While there was limited time to use the new YSYLO Advocacy and Activism curriculum, it can be used for future programming. Interview feedback highlighted that IYDA support varied depending on the institution. Some claimed that the IYDA modernized schools by providing them with school kits and equipment. The activity also provided plenty of teaching materials (chalk, paper, etc.), which improved the educational system for teachers and students. Others were able to add vocational tracks—namely the sewing and tailoring course—which was not in their CAPs prior to IYDA. They stated that after they added this track, the number of new students increased compared to the previous years. Other CAPS also highlighted how they had 20 students in cutting and sewing, but during IYDA, the number of students increased to 80. While this is a sign of increasing attendance and retention—at least during the life of the activity—it is the vocational tracks that had start-up kits readily available, as opposed to tracks that would pay better salaries or fees for the services but did not come with quality kits. Thus, the influence of the kit on vocational track selection and long-term retention should be examined post-project for final impact and sustainability. I had too many misconceptions about entrepreneurship, but at the end of my training, I understood that to become a great entrepreneur you need a lot of courage and patience. Today I am an employee, but my dream is to become my own boss. To achieve this, I have to work hard, have a good relationship with my boss, and prioritize his work more than money. On the professional side, this project helped me to discover and develop my intellectual abilities in terms of know-how and creativity. Personally, this project has changed my interpersonal character positively in terms of how to live with others and in terms of hygiene at work and personal hygiene. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 18 Part IV. Conclusions and Additional Recommendations Despite the many obstacles faced during the life of the activity, IYDA did for the most part manage to help build the assets and agency of youth—through the curriculum-driven programming along multiple pathways, through youth-driven community engagement, and through the entrepreneurship support. The activity’s greatest impact from the perspectives of many IYDA graduates was learning to become more independent, autonomous, and accountable. The life-skills curriculum Work Ready Now and the job opportunities and entrepreneurship support contributed to youth’s development of positive norms, expectations, and perceptions, and youth highlighted these outcomes in the interviews. Youth reported having a better idea about general health and hygiene and highlighted newly adopted life skills as stemming from the WRN curriculum. IYDA also contributed to improving the social standing of youth through youth contribution and engagement. The activity helped the youth build healthy self-images and relationships through its interventions but especially through the support it provided via its network of mentors and volunteers. The Young Women and Adolescent Girls Mentoring Pilot—during which vulnerable adolescent girls were matched to very successful professional women, such as lawyers, NGO workers, and professional photographers—received extremely positive feedback from participants. IYDA also promoted belonging and membership through activities such as sports and music with the Pygmies and the Bantu. Project constraints stemmed from an operating environment that inhibited socioeconomic development, especially for entrepreneurs who faced formal and informal taxation that curbed their growth potential. The external environment and structural weaknesses in the DRC negatively impacted program delivery, uptake, and sustainability. The whole system, and especially the education system, required significant systemic improvements to maximize investments such as IYDA. However, in the context, the activity would have benefited from a greater examination of the socio-cultural context and how the power dynamics impact the capacity to do business and/or operate within the market. The three main recommendations to strengthen IYDA interventions are as follows: 1. Provide all students with a manual. Although an expensive commitment, providing a student with a manual and a workbook would allow them to study at home and even share their knowledge with others. WRN and BYOB were of interest to all stakeholders, from students to teachers to employers. 2. Improve the technical curriculum. WRN and BYOB are highly relevant curriculum for improving life skills and initial business management skills, but they do not provide technical or vocational skills training nor did IYDA look at how to reinforce the technical curriculum. The activity provided some complementary and relevant skills, but it did not address the existing issues regarding the quality of the technical curriculum—including looking at how to diversify the types of vocations that were taught in the learning facilities they collaborated with. Trades should be forward looking and green. While this recommendation was outside of the project’s mandate, it should be included in future programming. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 19 3. Improve the efficiency of start-up kits. This recommendation includes both improving delivery mechanisms and leveraging investments with SILC group membership, as well as encouraging joint business ventures. In addition to the three IYDA recommendations, following are larger programmatic recommendations for future interventions: 1. Ensure activities do not exacerbate or perpetuate gender and other social disparities, and work to reduce disparities where possible. 2. Empower women, girls, and members of traditionally disadvantaged groups to access education and training opportunities that lead to social inclusion and economic growth. 3. Continue to promote positive youth engagement, and invest in shared community initiatives that celebrate culture and tradition (such as music and dance). 4. Improve access to and integration among age-appropriate and youth friendly services, engage youth and their families, and connect and integrate health and social services so there is a continuum of care and support at the community level. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 20 References ADIJ. (2019). Accompagnement et Apprentissage en Milieu Professionnel, guide de formation adapté au cohorte 2. U.S. Agency for International Development. CRS. (2019). Presentation on LMA and IPSE. Ppt presented in the coordination meeting. CRS, & ADIJ. (2019). Gender analysis report. report CRS. EDC. (2018) IYDA attachment B: Project description. EDC. (2018). Annual progress report: FY 2018 FINAL. U.S. Agency for International Development EDC. (2018, July). FY18 quarterly report, April–June 2018. EDC. (2018). Workplan FY19: Workplan narrative, March 2018–February 2021. EDC. (2019). Annual progress report: Year 2, October 2018–September 2019. EDC. (2019, January). FY 2019, Q1 quarterly report, October to December 2018. EDC. (2020, January). FY 2020, Q1 quarterly report, September to December 2019. EDC. (2019, June). Baseline evaluation report:, Measuring youth employment, perceptions, and skills, Cohort 1. EDC. (2019, April). FY 2019, Q2 quarterly report, January to March 2019. EDC. (2019, June). FY 2019, Q3 quarterly report, April to June 2019. EDC. (2019). Workplan: FY20, Year 2 work plan, October 2019-September 2020. EDC. (2020). Workplan: FY21 IYDA Y3 work plan narrative final, October1, 2020–September 30, 2021. EDC. (2020). Labor market assessment report. EDC. (2020). Annual Progress Report Year 2, fiscal report 2020, October 2019 – September 2020. EDC. (2020). Annual Progress Report Year 3, October 2020 – September 2021. EDC. (2020, April). FY2020 Q2 Quarterly Report, January to March 2020. EDC. (2020, July). FY2020 Q3 Quarterly Report, April to June 2020. EDC. (2021).Workplan_FY22_IYDA Y4 Work Plan Narrative_USAID, October 2021 – September 2022. EDC. (2021, February). FY2021 Q1 Quarterly Report, October to December 2020. EDC. (2021, April). FY2021 Q2 Quarterly Report, January to Mach 2021. EDC. (2021, July). FY2021 Q3 Quarterly Report, April to June 2021. EDC. (2021, October). USAID/DRC Integrated Youth Development Activity (IYDA), Endline Cohort I Outcome Evaluation Report. EDC. (2021, April). USAID/DRC Integrated Youth Development Activity (IYDA), Midline Cohort I Outcome Evaluation Report. EDC. (2021, July). USAID/DRC Integrated Youth Development Activity (IYDA), Endline Cohort 2 Evaluation Report: Outcomes of Youth Employment Survey. EDC. (2021, July). USAID/DRC Integrated Youth Development Activity (IYDA), Cohort 2, Midline, Outcome Evaluation Report. EDC. (2022, January). Quarterly Report, fiscal year 2022, quarter 1, October 1, 2021 to December 31, 2021. EDC | USAID/DRC IYDA Cooperative Agreement No. 72066018CA00003 21 EDC. (2022, April). Quarterly Report, fiscal year 2022, quarter 2, January 1, 2022, to March 31, 2022, 2021. EDC. (2022). Agency and Action: Strengthening the Leadership and Assets of Youth-Led and Youth￾Serving Organizations, report. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Français, Lecture/Ecriture, Guide de l’éducateur, CRS 3, Tome 1. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Français, Lecture/Ecriture, Guide de l’éducateur, CRS 3, Tome 2. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Français, Lecture/Ecriture, Manuel de l’apprenant(e), CRS 3. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Mathématique, Guide de l’Educateur (trice), Niveau 3 CRS, Tome 1. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Mathématique, Guide de l’Educateur (trice), Niveau 3 CRS, Tome 2. Ministère des Affaires Sociales (Direction Générale de l’Education Non-Formelle). (2019, Septembre). Mathématique, Manuel de l’apprenant(e), Niveau 3 CRS. Ministère de la jeunesse et Initiation à la nouvelle citoyenneté (Centre d’Apprentissage Professionnel CAP). (2019, Septembre). Mathématique en Kiswahili : Tome 1, Guide de l’éducateur. Ministère de la jeunesse et Initiation à la nouvelle citoyenneté (Centre d’Apprentissage Professionnel CAP). (2019, Septembre). Mathématique en Kiswahili : Tome 2, Guide de l’éducateur. Ministère de la jeunesse et Initiation à la nouvelle citoyenneté (Centre d’Apprentissage Professionnel CAP). (2019, Septembre). Mathématique en Kiswahili : Manuel de l’apprenant. Ujeneza, N. (2020). Investment and Private Sector Engagement Scoping Mission, IYDA, report EDC, USAID.