1 
EX-POST EVALUATION OF THE USAID/MALI
OUT OF SCHOOL YOUTH PROJECT (PAJE￾NIETA)
FINAL EVALUATION REPORT
December 2022
DISCLAIMER 
This report is made possible by the support of the American People through the United States Agency for International Development (USAID). It was 
prepared by the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) activity, a partnership of the United States Department of 
Agriculture/Foreign Agriculture Service (USDA/FAS), the University of Rhode Island (URI), and Kwame Nkrumah University of Science and Technology 
(KNUST). The content of this report does not necessarily reflect the views of USAID or the United States Government.
i
ACKNOWLEDGEMENTS
This ex-post evaluation of the USAID/Mali Out of School youth project (PAJE-NIETA) Final Report was 
prepared for the USAID Africa Bureau and USAID/ Mali Economic Growth Team under the Analytical 
Support Services and Evaluations for Sustainable Systems (ASSESS) Project (Project No. 624-P-14-00004). 
It was developed by Jindra Cekan, (PhD), Sandra Basgall (PhD), Mouhamadou Gueye (PhD) and 
Mohammed Diarra (PhD) with support from the evaluation managers: Alphonse Bigirimana (Africa Bureau 
Senior MEL Advisor), Mohamed Traore (USAID/Mali Senior MEL Specialist) and Harry Francois 
(USAID/Mali Senior MEL Specialist).
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) ii
TABLE OF CONTENTS
LIST OF TABLES iii
LIST OF FIGURES iii
ACRONYMS iv
ABSTRACT vi
EXECUTIVE SUMMARY vii
BACKGROUND vii
EVALUATION QUESTIONS viii
EVALUATION METHODOLOGY viii
FINDINGS AND CONCLUSIONS viii
RECOMMENDATIONS x
INTRODUCTION 1
MALI OUT OF SCHOOL YOUTH PROJECT BACKGROUND 1
INTRODUCTION 1
PROJECT OBJECTIVES/INTERMEDIATE RESULTS 1
PROJECT IMPLEMENTATION APPROACH AND SUSTAINABILITY STRATEGY 2
EVALUATION PURPOSE AND USE 5
EVALUATION PURPOSE 5
AUDIENCE AND USES OF THE EVALUATION 6
EVALUATION QUESTIONS 6
EVALUATION DESIGN AND METHODOLOGY 7
CONCEPTUAL FRAMEWORK 7
SAMPLING CONSIDERATIONS 8
DATA COLLECTION METHODS AND TOOLS 9
DATA ANALYSIS 11
EVALUATION LIMITATIONS AND MITIGATION STRATEGIES 12
FINDINGS 14
EVALUATION QUESTION 1 14
EVALUATION QUESTION 2 19
CONCLUSIONS 24
RECOMMENDATIONS 26
ANNEXES 28
ANNEX A: SCOPE OF WORK 28
ANNEX B: EVALUATION DESIGN MATRIX 43
ANNEX C: SELECTED VILLAGES BY REGION, COHORT, GENDER, AND ROAD DISTANCES 
ON-OFF TARMAC 45
ANNEX D: REFERENCES 46
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) iii
ANNEX E: FOCUS GROUP DISCUSSION GUIDES 48
ANNEX F: KEY INFORMANT INTERVIEW GUIDES 53
ANNEX G: KEY INFORMANTS INTERVIEWED 61
ANNEX H: PROFILE OF EVALUATION TEAM 65
LIST OF TABLES
TABLE 1: COHORT, PERIOD OF IMPLEMENTATION, TARGET, NUMBER OF VILLAGES AND 
REGIONS COVERED....................................................................................................................................................... 4
TABLE 2: FGD PARTICIPANTS BY RESPONDENT GROUP AND GENDER..............................................10
TABLE 3: KII RESPONDENTS BY RESPONDENT CATEGORY AND GENDER.........................................11
TABLE 4: SIKASSO RANKINGS OF MOST VALUABLE PAJE-NIETA INPUTS FOR SUSTAINABILITY 19
TABLE 5: : CURRENT LIVELIHOOD PRACTICED BY RESPONDENTS........................................................20
LIST OF FIGURES
FIGURE 1: MALIAN YOUTH LEARNING PATHWAY.......................................................................................... 2
FIGURE 2: MALI OUT OF SCHOOL YOUTH PROJECT HOLISTIC INTEGRATED MODEL....................3
FIGURE 3: SUSTAINABILITY AND EXIT STRATEGIES CONCEPTUAL FRAMEWORK ............................8
FIGURE 4: GEOGRAPHICAL LOCATIONS OF SELECTED VILLAGES............................................................ 9
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) iv
ACRONYMS
ACEFOR Amélioration de la Compétitivité des Entreprises par la Formation 
AFD Agence Française de Développement 
AFR USAID / Africa Bureau
AJA Association Jeunesse Action 
AMSS Association Malienne pour la Survie au Sahel 
ANPE l’Agence Nationale Pour l’Emploi
APEJ Agence pour la Promotion de l’Emploi des Jeunes
APTE Amélioration des Performances de Travail et d'Entrepreneuriat
ASSESS Analytical Support Services and Evaluations for Sustainable Systems 
CED Centers for Educational Development 
CRS Catholic Relief Services 
CVE Countering Violent Extremism 
DRC Democratic Republic of Congo (DRC)
EDC Education Development Center
EQ1 Evaluation Question 1
EQ2 Evaluation Question 2
ERSA Education Recovery Support Activity 
FACEJ Fonds d'Appui À la Création d'Entreprise par les Jeunes
FAFPA Le Fonds d’Appui à la Formation Professionnelle et à L’Apprentissage
FAMOC Fonds d'Appui aux Moteurs du Changement
FGD Focus Group Discussions 
FIER Formation Professionnelle, Et Appui À L’entreprenariat des Jeunes Ruraux
GII Gender Inequality Index 
GLEE Girls Leadership and Empowerment through Education
HDI Human Development Index 
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) v
IDT Interdisciplinary Team
IFAD International Fund for Agricultural Development
IGA Income Generating Activities
ILO International Labor Organization
INCLUSIF The Inclusive Financing Project for Agricultural Sectors 
INGO International Non-Governmental Organization
IYDA Integrated Youth Development Activity 
M&E Monitoring and Evaluation
NGOs Non-Governmental Organizations
PAJE-NIETA Project d’Appui Aux Jeunes Entrepreneurs
OIM L'Organisation Internationale Pour Les Migrations
PAEFFK Project for the Economic Empowerment of Women in the Shea Butter 
Subsector
PAFP Programme d'Appui À La Formation Professionnelle
PROCEJ Project For the Development of Youth Skills and Employment
PRODEJ Programme de Développement de l'Emploi des Jeunes
PSP Private Sector Partners 
RFP Request For Proposal
SILC Savings and Internal Lending Communities
SOW Scope of Work
UN United Nations
UNDP United Nations Development Programme
UNESCO United Nations Educational, Scientific and Cultural Organization
USAID United States Agency for International Development
USDA United States Department of Agriculture
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) vi
ABSTRACT
This ex-post evaluation examined the sustainability of the USAID/Mali Out of School Youth (PAJE-NIETA) 
outcomes, six years after project completion in 2015, and answers two questions: 
1. To what extent and in what ways did PAJE-NIETA sustainability strategy contribute to maintaining 
jobs and enterprises for youth and still supports the creation of jobs and youth enterprises in 
Mali?
2. To what extent and in what ways were PAJE-NIETA basic education, work readiness, financial and 
business skills programs effective in empowering youth to create their own enterprises and 
become more civically engaged in their communities? To what extent and in what context are 
these programs replicable? 
The evaluation employed a qualitative approach, conducting 72 key informant interviews with project 
stakeholders knowledgeable about the project in Mali, 27 focus group discussions with members of village 
youth associations, field observations of youth enterprises, and a review of project documents. 
The evaluation concluded that project success at the time of exit does not necessarily imply sustained 
benefit over time. Partnership, resources, capacity, and motivation were critical to achieve sustainability. 
Sustainability assumed that the results achieved at the village-level are self-sustainable. Apart from the 
village associations that existed, none of the sustainability strategies was working. The groups identified 
basic education, entrepreneurial training, and SILC as the most effective elements in empowering youth.
USAID is encouraged to adjust the solicitation and application review process for future projects to 
account for sustainability.
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) vii
EXECUTIVE SUMMARY
The United States Agency for International Development (USAID)/Mali and USAID/Africa Bureau (AFR) 
commissioned the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) activity 
managed by USAID/West Africa and implemented by the United States Department of Agriculture 
(USDA) to conduct an ex-post evaluation of the USAID/Mali Out of School Youth Project, also known as 
PAJE-NIETA (Projet d’Appui aux Jeunes Entrepreneurs). The purpose of the ex-post evaluation is to 
examine the sustainability of the PAJE-NIETA outcomes, six years after project completion. This report 
presents the findings, conclusions, and recommendations from the ex-post evaluation.
BACKGROUND
With half of Mali’s population under the age of 18 and youth unemployment over 15 percent1, Malian 
youth have few opportunities for meaningful work. Elevated levels of youth illiteracy and unemployment 
lead to migration and general disaffection. To address this challenge, USAID/Mali designed the Mali Out 
of School Youth Project (also known as Projet d’Appui aux Jeunes Entrepreneurs, PAJE-NIETA) to help rural 
youth in Mali change their own lives. PAJE-NIETA aimed to enable 10,000 rural youth between the ages 
of 14 and 25 who had either never been to school or who had dropped out in the early grades to become 
better educated, more economically productive, civically engaged and empowered to improve their lives 
and those of their families and communities. PAJE-NIETA was a five-year, $25 Million project implemented 
from 2010-2015 by the Education Development Center, Inc. (EDC) in collaboration with Catholic Relief 
Services (CRS), Swisscontact, Association Jeunesse Action (AJA) and for the Timbuktu region, Association 
Malienne pour la Survie au Sahel (AMSS). The Education, Economic Growth, and Democracy and 
Governance Teams of USAID/Mali jointly funded the project. 
EDC and its partners developed an integrated holistic model of the youth development cycle which was 
used as the main guide for project implementation. The integrated holistic model included providing youth 
with basic education, entrepreneurship training, and training in agro-pastoral and service industries. The 
project trained volunteers to deliver basic education and entrepreneurship courses and to accompany 
program youth on their path to becoming entrepreneurs. PAJE-NIETA also trained youth to use mobile 
technology to access basic education lessons via Stepping Stone, an application developed by EDC, and 
further trained the youth to establish savings and internal lending groups to support both business and 
social loans. The project also strengthened the organizational capacities of existing youth associations to 
manage and implement community projects. At the end of the project, a total of 10,951 youth completed 
technical training in an income-generating activity of their choice. Of the 6,806 youth in the third and final 
cohort of youth, 95 percent successfully launched micro-enterprises.
To promote sustainability of project activities, PAJE-NIETA employed strategies including capacity 
development of existing local youth associations, integration of youth volunteers, development of a local 
training network, and the implementation of the Private Service Provider (PSP) approach designed by CRS 
to foster sustainability of savings groups. Furthermore, the project collaborated with the Government of 
Mali (GOM) to help ensure materials developed and lessons cultivated under the project benefit the 
GOM’s efforts in education and youth development.
1 Mali Youth Assessment Report (2010). https://pdf.usaid.gov/pdf_docs/pnadz230.pdf. Accessed on June 3, 2022.
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) viii
EVALUATION QUESTIONS
The ex-post evaluation answers the following evaluation questions (EQs): 
1. To what extent and in what ways did PAJE-NIETA sustainability strategy contribute to 
maintaining jobs and enterprises for youth and still supports the creation of jobs and youth 
enterprises in Mali? 
2. To what extent and in what ways were PAJE-NIETA basic education, work readiness, financial 
and business skills programs effective in empowering youth to create their own enterprises and 
become more civically engaged in their communities? To what extent and in what context are 
these programs replicable? 
EVALUATION METHODOLOGY
The ex-post evaluation employed a qualitative approach, conducting 72 key informant interviews (KIIs), 
27 focus group discussions (FGDs), and a desk review of key project documents. The evaluation team 
conducted key informant interviews with PAJE-NIETA stakeholders, including USAID/Mali staff with direct 
knowledge of the project, PAJE-NIETA implementing partner staff, government representatives, 
representatives of youth associations and youth volunteers, project agents, as well as a non-representative 
sample of youth beneficiaries who live in the communities that were targeted by the project and are 
knowledgeable about the project. The key informants were selected using a mix of purposive and 
convenience sampling. The evaluation team also conducted a scoping visit from August 23-28, 2022, to 
identify the key informants. This research included five weeks (October 5 – November 6, 2022) of primary 
data collection in Mali, focused on the project regions of Koulikoro, Sikasso, and Kayes, eliminating 
Timbuktu due to insecurity. The team used pattern/content analysis to analyze the data and developed 
findings and conclusions for each evaluation question. The evaluation team identified the following five 
limitations inherent to any ex-post evaluation: (1) Challenges with attribution2 of outcomes that 
materialized after the project closure; (2) Inability to locate project beneficiaries and key stakeholders 
who are knowledgeable about the project; (3) Difficulty in finding key project documentation; (4) Poor 
quality of the data collected by the project; and (5) Recall biases resulting from respondents’ poor 
recollection of past events. The mitigation strategies used by the team to ensure the findings are 
dependable and valid are presented under the section Evaluation Limitations and Mitigations Strategies. 
FINDINGS AND CONCLUSIONS 
Evaluation Question 1: To what extent and in what ways did PAJE-NIETA sustainability strategy contribute to 
maintaining jobs and enterprises for youth and still supports the creation of jobs and youth enterprises in Mali?
Through interviews with stakeholders, the evaluation team concluded:
● PAJE-NIETA did not have an explicit sustainability strategic document that was 
shared with both the partners and the Government of Mali. However, to promote 
sustainability of project activities, the project employed several strategies including the capacity 
development of existing local youth associations, the integration of youth volunteers, the 
development of a local training network, and the implementation of the private service provider 
(PSP) approach designed by CRS to foster sustainability of savings groups. There were loose ends 
regarding the sustainability strategy employed by PAJE-NIETA. The project, at closure, was not 
2 Attribution is the extent to which the observed change is the result of the intervention, considering all other factors which 
may also affect the outcome(s) of interest, while contribution refers to the extent to which an intervention is assessed to have 
contributed to particular outcomes. Due to the challenge with attribution for ex-post evaluations, contribution is preferred. 
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) ix
handed over to the Government of Mali as a whole nor to the respective ministries or any 
partners in their sectoral areas.
● Implementing partners such as AJA, and EDC, continue to use PAJE-NIETA support 
structures, tools, and training material. EDC replicated PAJE-NIETA’s training and support 
programming model in Mali, Senegal, and Democratic Republic of Congo. UN-bilateral projects 
such as Formation Professionnelle, et Appui à L’entrepreneuriat des Jeunes Ruraux (FIER) 
continued PAJE-NIETA capacity development by funding the manual for youth vocational training. 
The curriculum for the manual used PAJE-NIETA’s manual as a guide.
● Political and civil unrest during project implementation hampered government 
collaboration. Due to political and civil unrest, the project could not integrate the Malian 
government in its activities. Additionally, the upheaval did not foster continued long-term donor 
commitments to education and entrepreneurship, as USAID funding to PAJE-NIETA ceased.
● With the exception of the village associations that existed, none of the sustainability 
strategies employed by PAJE-NIETA was operational. Community members self￾trained each other. Linkages to employment networks and market information, including via 
telecom service providers, were nonexistent because the equipment became obsolete. 
Collaboration with government institutions did not work well due to political and civil unrest 
during project implementation and lack of financial resources from the government. Private sector 
mentors and PSPs were not sustained in the eight villages visited. Youth members who received 
training on specific livelihoods trained others. 
Evaluation Question 2: To what extent and in what ways were PAJE-NIETA basic education, work readiness, 
financial and business skills programs effective in empowering youth to create their own enterprises and become 
more civically engaged in their communities? To what extent and in what context are these programs replicable? 
The evaluation team reached the following conclusions: 
● Basic education was most effective in strengthening the capacity of youth. The 
evaluation found that among all the key elements of PAJE-NIETA, project participants ranked basic 
education as the most essential element for the sustainability of their livelihoods. This was 
followed by entrepreneurial training, and then IGA kits. SILC ranked least among the most 
effective elements of the model to strengthen their capacity and maintain their business overtime. 
Majority of the women were still practicing the livelihood they were trained in. Most were using 
their PAJE-NIETA livelihoods to generate secondary or tertiary income, supplementing their 
subsistence agriculture. 
● Several projects have replicated elements of PAJE-NIETA’s holistic and integrated 
learning model, but none replicated the full model. Elements of PAJE-NIETA’s model such 
as basic education, vocational training, and SILC have been replicated by the following activities: 
❖ Project For the Development of Youth Skills and Employment (PROCEJ), 
❖ Fonds d'Appui À la Création d'Entreprise par les Jeunes (FACEJ), 
❖ Formation Professionnelle, Et Appui À L’entreprenariat des Jeunes Ruraux (FIER), and
❖ Agence pour la Promotion de l’Emploi des Jeunes (APEJ)
FIER project adopted the training materials for literacy and numeracy but lacked the financial 
resources to continue providing IGA kits, SILC, and other capacity building inputs. 
● Community members learned from one another to diversify their income sources in 
the face of these shocks. PAJE-NIETA trained youth in livelihoods with diversified incomes. 
The Sahel experienced a series of shocks during the project and after project closure. The shocks 
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) x 
included coup d’état in 2012, COVID-19, and inflation which negatively impacted livelihoods. 
Because the project trained youth associations in different livelihood., Community members 
learned from one another to diversify their income sources in the face of these shocks.
RECOMMENDATIONS
Based on the findings and conclusions, the evaluation team proposes the following recommendations to 
USAID, future implementing partners, and the Government of Mali:
USAID
1. USAID should require implementing partners to submit a sustainability framework with every 
proposal. USAID should ensure the sustainability framework details a sustainability 
implementation pathway within the intervention design with clear sustainability timelines and 
benchmarks and plans to document all original data from the intervention. The data should be 
made accessible to USAID. 
2. USAID should ensure in measuring project impact; evaluations must incorporate indicators of 
sustainability. This implies, implementing partners need to demonstrate evidence of resources, 
capacity, partnership, and motivation continuing to sustain benefits at the end of the project.
3. USAID should consider adjusting its evaluative processes and extending future youth projects 
beyond 5 years when there is evidence of progress toward sustainable impacts and indications of 
potential for sustainability.
4. USAID and its partners should strengthen their capacities, as necessary, to institutionalize 
sustainability in programming through training and improved knowledge management, as well as 
strengthened organizational commitment to look beyond immediate impact to sustainability.
5. USAID should ensure continued and consistent use of a system whereby implementing partners 
archive all baseline and evaluation reports including accessible and documented original data in 
order to enable post-project evaluations.
6. USAID should design future projects such that exit is gradual, with a phased transfer of 
responsibility to the appropriate stakeholders; exit should follow a phase of incrementally 
independent operation, and project beneficiaries and beneficiary communities should be engaged 
in plans for sustainability and exit from the beginning of the project cycle.
7. Projects should be designed with the local context (economic, political, and social/ cultural 
systems) in mind, should take account of the need for resilience in the face of climate or other 
shocks and stressors, and should be updated in response to changes in the local context.
8. Project design should incorporate strategies for sustaining beneficiary demand as well as
supply of services.
Implementing Partners
1. Implementing partners should base their sustainability plans and related exit strategies on clearly 
articulated theories of change. Implementing partners need to assess carefully and realistically the 
assumptions underlying their sustainability plans and reassess them continually to account for 
change in the external environment.
2. Implementing partners should conduct surveys to assess the durability and sustainability of 
enterprises during project implementation.
Government of Mali
1. The Government of Mali should work with donors to integrate policies and programming, such 
as how projects feed into youth development policies. Further benefits would come from 
Final Report: Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) xi 
government investments in universal education being taught throughout the country and that basic 
education including financial literacy, vocational training, and support – the three pillars of the 
program–are included. But given the political climate in Mali, such as requesting the French 
government’s aid program to leave, conditions may not be conducive now.
2. If funding is available, the government should trace the use of PAJE-NIETA skills gained by former 
national staff to assess continued use and investment back into national growth, including any 
benefits of having government representatives.
3. Mobilize more financial resources for the promotion of basic and vocational education to offer 
more applied courses to out-of-school youth in areas where education is still lagging behind. Also 
ensure that the complementary training components and all inputs are ready for startup. 
1 
INTRODUCTION
The United States Agency for International Development (USAID)/Mali and USAID/Africa Bureau (AFR) 
requested the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) activity 
managed by USAID/West Africa and implemented by the United States Department of Agriculture 
(USDA) to conduct an ex-post evaluation of the USAID/Mali Out of School Youth Project, also known as 
PAJE-NIETA (Projet d’Appui aux Jeunes Entrepreneurs). The evaluation examined the sustainability of the 
PAJE-NIETA outcomes, six years after project completion. USAID’s Statement of Work (SOW) for this 
evaluation is included in Annex A. 
This report comprehensively addresses each of the evaluation questions listed in the SOW and presents 
the findings, conclusions, and recommendations. First, it gives a background to the PAJE-NIETA project, 
the implementation approach and the sustainability strategy adopted by the project to put the evaluation 
in context. Then, it presents the evaluation purpose and use as well as the evaluation questions. It then 
lays out the evaluation design, findings, and conclusions. The report ends with recommendations drawn 
from specific findings and conclusions. 
MALI OUT OF SCHOOL YOUTH PROJECT BACKGROUND
INTRODUCTION
PAJE-NIETA was a five-year, $25 Million project implemented from 2010-2015 by the Education 
Development Center, Inc. (EDC) in collaboration with Catholic Relief Services (CRS), Swisscontact, 
Association Jeunesse Action (AJA) and for the Timbuktu region, Association Malienne pour la Survie au 
Sahel (AMSS). The Education, Economic Growth, and Democracy and Governance Teams of USAID/Mali 
jointly funded the project. 
The project began with a Rapid Assessment of Malian youth conducted in April and May 2010, which 
recommended that primary attention be given to youth living in rural villages and peri-urban areas in an
attempt to reduce migration from rural areas to urban centers. The Rapid Assessment revealed that 
youth wanted to earn a living and achieve successful livelihoods in their home villages. It also revealed 
that youth are entrepreneurial and able to identify the inputs and knowledge they need to reach their 
goals, which, for rural youth, are in the agro-pastoral sector.
In order to address that need, USAID/Mali designed the Mali Out of School Youth Project with the goal 
to enable 10,000 rural youth between the ages of 14 and 25 who had either never been to school or who 
had dropped out in the early grades to become better educated, more economically productive, civically 
engaged and empowered to improve their lives and those of their families and communities. 
PROJECT OBJECTIVES/INTERMEDIATE RESULTS
The project addressed four intermediate results of USAID/Mali’s strategic plan:
IR 1: Increased access to relevant basic education: The first project Intermediate Result (IR) was to enable 
youth to acquire basic academic skills, knowledge and attitudes needed to enter the job market. Youth 
were provided with work readiness, entrepreneurship and basic education training focused on literacy 
and math skills in local language as well as a functional level of literacy in French to negotiate the 
marketplace. This also included information needed to become better educated, more economically 
productive and civically engaged through training in life skills, leadership, and conflict resolution. It was 
anticipated that at the end of the project, targeted youth would be better equipped with basic 
education and work readiness skills needed to earn a livelihood.
IR 2: Increased access to relevant skills training: The project’s second intermediate result was to enable 
youth to acquire technical skills needed to earn a living within rural and peri-urban settings. Rural youth 
2 
would engage primarily in agro-enterprise development training while peri-urban youth may focus more 
on ancillary vocational education skills training related to the agricultural sector, such as fabrication of 
agricultural tools. Related to this, the project sought to build the capacity of rural vocational training 
providers, particularly with respect to their ability to train out-of-school and illiterate youth. It was 
anticipated that at the end of the project, targeted youth would be better equipped with vocational 
education and technical skills needed to earn a livelihood while local skills providers have improved 
capacity to prepare youth for work.
IR 3: Increased entrepreneurship opportunities and job networking: Under this Intermediate Result, PAJE￾NIETA was tasked to enable youth to acquire entrepreneurship and work readiness skills needed for self￾employment and job networking. The project provided youth with entrepreneurship and work readiness 
skills training and consulted with micro-finance institutions, local businesses, producers, and ateliers to 
increase economic opportunities for youth via self-employment or jobs. At the end of the project, it was 
expected that targeted youth would gain viable employment opportunities (self-employment or 
other) in the agricultural and vocational education sectors.
IR 4: Increased civic engagement: The project’s fourth intermediate result was to increase civic 
engagement at community, regional and national levels. Youth development activities reinforced the notion 
of community and regional and national level civic engagement. The project supported youth in building 
the capacity of existing youth associations. Youth associations were guided to collect data, analyze 
community needs, and plan and execute community service projects with assistance from the project in 
the form of small grants.
Figure 1 depicts the Mali Youth learning pathway which implicitly conveys the project’s Theory of Change.
FIGURE 1: MALIAN YOUTH LEARNING PATHWAY
PROJECT IMPLEMENTATION APPROACH AND SUSTAINABILITY STRATEGY 
Implementation Approach
EDC and its partners developed an integrated holistic model of the youth development cycle which was 
used as the main guide for project implementation3. The integrated holistic model included providing basic 
3 Following the rapid youth assessment in 2010, EDC conducted extensive consultations with the Ministry of National Education, Ministry of 
Employment and Professional Training, the Ministry of Youth and Sports, the Ministry of National Education, USAID, and other key 
3
education training for youth covering literacy, numeracy, and life skills, upgrading the quality of technical 
skills for stakeholders involved in youth vocational training, providing accompaniment in the form of work 
readiness, financial, and business skills training as well as links to credit and savings programs to facilitate 
the purchase of start-up equipment. The project integrated information technology to support various 
interventions. The model is summarized in Figure 2. The frontline implementers of the project were 
Youth Corp Volunteers (unemployed university or professional school graduates who committed to 
living in the target villages), dispensing the basic education and entrepreneurship courses, and 
accompanying the youth as they launched their microenterprises. In addition, the volunteers collaborated 
with the Village Youth Associations and assisted in community service activities.
FIGURE 2: MALI OUT OF SCHOOL YOUTH PROJECT HOLISTIC INTEGRATED MODEL
The project targeted a total of 10,000 rural and peri-urban youth, an equal number of children aged 14-
25 who were unschooled or out-of-school and had little or no literacy skills. It covered the regions of 
Kayes, Koulikoro, Sikasso, and Timbuktu, and followed a cohort-model consisting of three consecutive 
youth cohorts. Lessons learned from each cohort were used to improve interventions for the next 
cohorts. Each cohort and its period of implementation as well as the target and regions covered is 
presented in Table 1. 
stakeholders to develop the project model. Based on the results of these consultations, data from the Mali youth assessment and lessons 
learned through other EQUIP3 youth projects, EDC developed an integrated holistic model for the youth development cycle.
4 
TABLE 1: COHORT, PERIOD OF IMPLEMENTATION, TARGET, NUMBER OF VILLAGES AND REGIONS COVERED
COHORTS PERIOD TARGET NUMBER OF 
VILLAGES
REGIONS 
COVERED
Cohort 1 June 2011 – June 2013 5,740 youths 
(54 percent women)
100 Kayes, 
Koulikoro, 
Sikasso
Cohort 2 February 2013 – July 2014 1,255 youths
(50 percent women)
21 Sikasso
Cohort 3 June 2014 – November 2015 7,858 youths
(60 percent women)
107 Kayes, 
Sikasso, 
Timbuktu
For the first two cohorts, youth were trained in market gardening, grain cultivation, livestock-fattening, 
poultry-raising, the transformation of agricultural products, the production and repair of agricultural 
equipment and material, soapmaking, and establishing a small restaurant or bakery. With the third cohort, 
more service-oriented income-generating activities (IGAs) were added. This included carpentry, masonry, 
sewing, hairdressing, photography, and cellphone repair. For each IGA, the project identified local 
professionals living in reasonable proximity to the youth to serve as trainers, so that training could be 
provided at the village level and to facilitate post-training follow-up visits by the trainers to each youth 
individually.
Shortly after the youth completed technical training, they received starter kits for their IGA. These kits 
included the minimum set of equipment and material needed to launch their microenterprise. For example, 
for market gardening and grain cultivation, the youth received a variety of improved seeds, a plow, 
watering can and other essentials. For soap-making, the youth received basic equipment, a customized 
soap-cutting table, protective gear, and essential supplies. For establishing a small restaurant, youth 
received a table and benches, cooking and serving utensils, and a small local stove. The average value of 
the kits was US$130.
Youth were also trained by the volunteers and assisted in forming and managing savings and internal 
lending communities (SILC), which gave them access to cash, based on an internationally-used model 
developed by implementing partner CRS.4 Members committed to saving a specific sum each month and 
learned the principles of saving and lending, charging small amounts of interest on loans and delivering 
penalties for late payments. Toward the end of each youth development cycle, youth were introduced to 
micro-finance institutions within their region. Representatives shared the procedures and requirements 
for securing loans with the youth. 
Sustainability Strategy 
The project was designed with sustainability in mind. Key elements of the PAJE-NIETA sustainability plan 
included: 
a. Strengthening the capacity of youth serving institutions and a local training network: The 
project provided organizational capacity building training to members of youth associations, all 
serving as volunteers. Of particular note, the project incorporated in its design an innovative Mali 
Youth Volunteer Corps activity under which 200 educated, unemployed youth who were recent 
4 Each community established two funds: a general fund that provided loans for microenterprise development and a social fund that provided 
loans for personal needs such as illness, marriage, and baptisms.
5 
graduates, were recruited and trained to serve as community-based literacy, work readiness, and 
youth leadership trainers. Youth were also mobilized to design and implement community service 
projects in their villages. 
b. Introducing the Private Sector Provider (PSP) Approach: The project initiated a private 
sector provider approach to enhance the sustainability of the project in its last year (between 
January to June 2015). Under this approach, communities selected a representative to serve as
PSP candidate who received intensive training and got certified and prepared to provide services 
to community members after the project ends. PSPs receive payments for these services. 
c. Partnering with government institutions including:
● Ministry of Employment and Vocational Training
● The Fonds d’Appui a la Formation Professionnelle et L’Apprentissage (FAFPA/ 
Professional Training and Support Fund) in order to ensure the stability and sustainability 
of the Mali Youth Corps.
● The Ministry of Youth, Ministry of Agriculture, and the Ministry of Education.
● At the commune level: The mayor’s office, ministry representatives, community 
development councils, elders, and youth associations 
d. Youth linkages to the private sector: 
● Support to ateliers for additional and improved skills standards, training modules, and 
implication in the accreditation process. Accreditation and certification will motivate 
youth learners in agro-enterprise and in ateliers to continue to use these local systems 
for skills training in the future. 
● Linkages to employment networks and market information system via mobile phones; 
fostering linkages between local telecoms and service providers (agricultural market data, 
employment services) to ensure that these services will remain available after the close 
of the project.
e. Linkages created for agro-entrepreneurs, such as improved access to new markets (through 
opportunity identification analysis), improved access to agricultural market data and new 
processing and marketing methods will allow producers to be sustainable after the project 
intervention cycle is complete.
EVALUATION PURPOSE AND USE
EVALUATION PURPOSE 
The purpose of this ex-post evaluation is to assess the sustainability of project-supported interventions 
six years after the project ended. 
USAID ADS 201 defines sustainability as “The ability of a local system, network, or institutions to produce 
desired outcomes over time. Programs contribute to sustainability when they strengthen the ability to 
produce valued results and to be both resilient and adaptive in the face of changing circumstances” (ADS 
201, see Definitions).
The evaluation seeks to understand the extent to which the PAJE-NIETA sustainability strategy described 
above was effective and is still relevant in addressing the education and business needs of out-of-school 
youth. Specifically, and consistent with the definition of sustainability above, the evaluation will examine 
the extent to which local structures, networks, linkages, and systems established or strengthened by PAJE-
6 
NIETA continue to function and contribute to out-of-school youth education, enterprise creation, and 
civic engagement. In addition, the evaluation will examine the extent to which the holistic and integrated 
out-of-school youth learning model empowered youth in creating and sustaining their own enterprises 
and civic engagement initiatives and how it can be replicated in similar activities. Finally, the evaluation will 
identify the factors behind the effectiveness and sustainability of PAJE-NIETA interventions as well as 
lessons learned and best practices that can be broadly shared and replicated to improve the sustainability 
of future out of school programs. 
AUDIENCE AND USES OF THE EVALUATION
The primary intended audience for this evaluation is USAID/Mali, which will use the results to inform the 
design of an upcoming youth activity as well as to plan for future programs. The findings of the evaluation 
will also be beneficial for the Government of Mali, other donors, education stakeholders, USDA, the Africa 
Bureau, and other USAID/Washington Bureaus which will acquire a better understanding of the 
sustainability of alternative education interventions catering to youth and use findings and lessons learned 
to improve the support and increase impact of future interventions. The ex-post evaluation findings will 
also contribute to the overall Agency’s learning agenda questions focusing on program sustainability which 
is one of the key USAID programming principles. Other Agency and Africa Bureau programming priorities 
and approaches to the evaluation contribute to include resilience and multisectoral programming.
EVALUATION QUESTIONS
In light of the above purposes and uses of the evaluation, AFR and USAID/Mali would like to investigate 
and understand the following key questions and lines of inquiry related to the effectiveness of the 
sustainability strategy and implementation approach employed by PAJE-NIETA. 
Evaluation Question 1: To what extent and in what ways did PAJE-NIETA sustainability strategy contributed to 
maintaining jobs and enterprises for youth and still supports the creation of jobs and youth enterprises in Mali? 
In addressing this question, the evaluation team considered the local and national structures, systems, 
networks, and linkages established or strengthened by the project and whether they continue to support 
and promote youth literacy and business skills, a favorable economic environment for youth (access to 
credit, savings programs, and employment opportunities). The evaluation team also documented instances 
where the project-supported structures, tools, training materials, and curricula have been replicated. The 
evaluation team also analyzed what aspects of the sustainability strategy worked or did not work well and 
why.
Evaluation Question 2: To what extent and in what ways were PAJE-NIETA basic education, work readiness, 
financial and business skills programs effective in empowering youth to create their own enterprises and become 
more civically engaged in their communities? To what extent and in what context are these programs replicable? 
In answering this question, the evaluation team examined the Mali Out of School Holistic and Integrated 
Learning Model to understand which elements of the model were most effective in strengthening the 
capacity of youth to create and overtime maintain their businesses and civic engagement, and which ones 
are likely to be replicated in similar activities as best approaches for sustaining of out-of-school youth 
programs. 
7 
EVALUATION DESIGN AND METHODOLOGY
This ex-post evaluation employed a qualitative approach, conducting key informant interviews (KIIs), focus 
group discussions (FGDs), observation, and review of project documents. The evaluation design and 
methodology section begin with a conceptual framework, followed by the sampling considerations, data 
collection methods and tools, and data analysis. It concludes with related challenges or limitations 
encountered during the evaluation and mitigation strategies employed.
CONCEPTUAL FRAMEWORK
During the evaluation design phase, the evaluation team adopted a sustainability and exit strategies 
conceptual framework developed by Rogers and Coates (2015)5 as shown in Figure 3. The framework 
identified four factors critical to sustainability: (1) a sustained source of resources; (2) sustained technical 
and managerial capacity, so that service providers can operate independently of the project; (3) sustained 
motivation and incentives (of beneficiaries and service providers) that do not rely on program inputs; and 
often (4) sustained linkages to government organizations and/or other entities (partnerships) that can 
augment resources, refresh capacity, and motivate frontline service providers and beneficiaries to provide 
and make use of services and continue project-promoted practices. Thus, this evaluation focused not only 
on the ‘structures, networks, and linkages’ (partnerships) that produced desired outcome over time, but also 
the resources used to fuel such investments, the on-going youth capacity development, and whether 
project-related activities have continued or ceased.
The evaluation team also looked at the exit process which is critical to sustainability. A final factor 
underlying the study of sustainability is external shocks, such as periodic drought, or political crisis. 
COVID-19 as well as key contextual factors, such as government structure, other projects operating in 
the area, and/or cultural beliefs, could threaten the sustainability of outcomes and impacts achieved during 
the project unless recognized and managed from project conception by incorporating resilience strategies 
and other contingencies into the sustainability plan. The evaluation team assessed the role of each of these 
factors as it relates to project observed sustainability to respond to the two evaluation questions. 
As part of the evaluation design phase, the evaluation team also developed an Evaluation Design Matrix 
summarizing the data sources, data collection and analysis methods, and sampling approaches for each 
evaluation question and sub-questions. Annex B presents details of the Evaluation Design Matrix. Brief 
profiles of the evaluation team members are presented in Annex H.
5 Rogers, Beatrice Lorge and Jennifer Coates. 2015. “Sustaining Development: A Synthesis of Results from a Four-Country Study of 
Sustainability and Exit Strategies among Development Food Assistance Projects. For USAID/ FANTA Project III
8 
FIGURE 3: SUSTAINABILITY AND EXIT STRATEGIES CONCEPTUAL FRAMEWORK
Source: Rogers and Coates (2015). Adapted from Coates and Kegode. 2012. ‘Kenya Exit Strategies Study Round 2 Report’.
SAMPLING CONSIDERATIONS
Village Selection
To ensure the right stakeholders and youth beneficiaries are selected to participate in the KIIs and FGDs, 
the evaluation team used purposive sampling for selecting youth beneficiaries and Snowball technique for 
the partners. Villages with 1,000-2,000 inhabitants were preferred to those with less population. The 
evaluation team assumed that for an ex-post evaluation, the likelihood of finding project participants still 
living in a populated village is higher than a less populated village. With the limited time for project 
evaluation and maximization of fieldwork travel efficiency, preference was given to villages closer to the 
regional capital, Bamako, while balancing villages that were on and off tarmac. Using a sample of villages 
from PAJE-NIETA Mid-term Evaluation obtained from Centre d’Etude et de Recherche sur l’Information en 
Population et Santé (CERIPS)6 as sampling frame, the evaluation team purposively selected three project 
villages in the Sikasso Region (Mena, Sogola7 and M’pegnesso), three villages in Kayes Region (Katakoto, 
Horongo, and Baliani) and two in Koulikoro Region (Mofa, Kalassa). The selected villages cover all the 
three cohorts of the project. The selected villages by region, cohort, population, gender, and distances 
from tarmac are presented in Annex C. Figure 4 shows the geographical locations of the eight selected 
villages. Due to security concerns, Timbuktu Region was not selected as part of this ex-post evaluation. A 
scoping exercise was conducted to confirm participants availability (as no complete participant lists 
were available, nor was a complete final evaluation with disaggregated data). The scoping exercise showed 
a high ‘survival rate’ of PAJE-NIETA participants in the selected villages, an average of 81 percent. 
6 CERIPS is a consultancy firm founded in May 2012 and based in Bamako, Mali with specialization in population research.
7 At the time of project implementation, both Mena and Sogola were administratively villages under the cercle Bougouni, which was under 
Sikasso Region. During the ex-post evaluation, Bougouni was a region carved out of the Sikasso Region.
9 
FIGURE 4: GEOGRAPHICAL LOCATIONS OF SELECTED VILLAGES
Kayes Region Koulikoro Region Sikasso Region Pretest Village
Respondent Selection
The selection of respondents for the evaluation involved stratification by the type of evaluation question 
to be addressed and respondent category, although there was significant overlap for the interviews. The 
evaluation team used the following strata: Evaluation Question 1 (EQ1) for partners and Evaluation 
Question 2 (EQ2) for youth beneficiaries. Project partners include USAID/Mali, Government Ministries, 
EDC, CRS, Swisscontact, AJA, Project Agents, and Youth Volunteers. Youth beneficiaries are members of 
the youth associations created by the project in the selected villages. Within strata, the evaluation applied 
purposive and snowball sampling, seeking to identify individuals who are knowledgeable about aspects of 
PAJE-NIETA or its stakeholder community to speak credibly about project sustainability, and other 
related EQ topics. USAID/Mali shared with the evaluation team contact details of potential key partners 
to interview.
DATA COLLECTION METHODS AND TOOLS
The evaluation team employed four data collection methods: (1) Desk review of performance reports and 
other project documents and data; (2) Key Informant Interviews; (3) Focus Group Discussions; and (4) 
Observations. 
Desk Review: The evaluation team reviewed project-related documents received from USAID/Mali. The 
review provided direction to the evaluation. For example, the review showed that there was no explicit 
project sustainability strategy as well as project final evaluation report before the project ended. This 
informed the team to purely use qualitative data for the ex-post evaluation. 
A list of the documents received include: 
● Mali Youth Assessment Report
● Project Description, Approval, and Modification Documents 
10
● Performance Monitoring Plan
● Quarterly and Annual Reports
● Project Baseline and Endline Surveys
● PAJE NIETA Mid-Term Evaluation
● Mali Out of School Final Project Report
● PAJE-NIETA Final Evaluation Design (Power Point Slides)
Additional documents reviewed by the evaluation team are presented in Annex D. 
Focus Group Discussions (FGDs): The evaluation team conducted 27 FGDs with the eight village 
youth associations (one to three FGDs and two livelihoods and employment FGDs per village). Table 2 
shows the total number of participants and the number of those participants who were females (70 
percent females). The evaluation team developed two different FGD Guides–the Youth Associations FGD 
Guide focused on youth current livelihoods, PAJE-NIETA trainings received, projects after PAJE-NIETA, 
outmigration, civic engagement, and emerging outcomes while the livelihood and employment FGD Guide 
focused on respondents’ livelihood and employment in detail (See Annex E for the FGD Guides). Due to 
respondent recall bias, the evaluation team allowed available youth members in a group to participate in 
youth association FGDs to capture responses as accurately as possible. Participants in the youth 
association FGDs ranged from 13 to 56, and 3 to 32 group members participated in the livelihoods and 
employment FGDs. 
TABLE 2: FGD PARTICIPANTS BY RESPONDENT GROUP AND GENDER
RESPONDENT 
CATEGORY 
NUMBER OF 
FGDS
TOTAL # OF 
PARTICIPANTS 
# FEMALE REGIONS 
COVERED
Youth Associations 11 280 223 Koulikoro (2)
Sikasso (6)
Kayes (3)
Livelihoods and 
Employment 
16 168 132 Koulikoro (4)
Sikasso (6)
Kayes (6)
Total 27 457 355
Key Informant Interviews (KIIs): The evaluation team conducted 72 semi-structured KIIs with PAJE￾NIETA stakeholders including USAID/Mali staff with direct knowledge of the project, PAJE-NIETA 
implementing partner staff, government representatives, representatives of youth associations and youth 
volunteers, youth employers, as well as a non-representative sample of youth beneficiaries who live in 
the communities that were targeted by the project and are knowledgeable about the project. Table 3 
shows the number of KIIs the team conducted by respondent category and number of respondents who 
were female (i.e., gender consideration in the team’s evaluation approach). Annex F provides the 
instruments the team used for these interviews. KII questions differed based on the respondent’s 
relationship to USAID and PAJE-NIETA; however, most key informants were asked about their 
relationship to PAJE-NIETA or USAID activities, their perceptions of these activities, challenges and 
opportunities during project implementation, and sustainability of the project results. Youth beneficiaries 
11
KIIs targeted their work and income then-and-now, emerging ways results were sustained at individual 
level and gender issues.
TABLE 3: KII RESPONDENTS BY RESPONDENT CATEGORY AND GENDER
RESPONDENT CATEGORY # INTERVIEWED # FEMALE
Project Partners
USAID/Mali 2 0
Government Ministries 5 0
EDC 5 5
CRS 1 1
Swisscontact 1 1
AJA 1 0
Field/Project Agents 25 3
Youth Volunteers 10 2
Total 50 12
Project Beneficiaries
Individual youth 22 10
Total 72 22
Observations: The evaluation team conducted on-site observations of youth enterprises who 
participated in the KIIs and FGDs. Videos and photographs of observable facilities and structures were 
taken and integrated in this report to validate findings. 
The evaluation team collected data in two phases. Phase 1, from October 5-22, 2022, focused on 
interviewing project beneficiaries, project agents and youth volunteers. Phase II, which took place from 
October 24 – November 6, 2022, targeted USAID, government ministries and implementing partners. 
This was to allow the project partners to put into perspective the findings from the project beneficiaries, 
project agents and youth volunteers.
DATA ANALYSIS 
The evaluation team used Pattern/Content Analysis to identify common themes and trends in the 
qualitative data from the KIIs and FGDs. First, the team captured all the FGDs and KIIs data into Microsoft 
Excel. The Microsoft Excel files were cleaned up, stripping off non-essential words and simultaneously 
assigning each participant comment or quote a separate line on the page as well as each new thought or 
idea therein. Each row was then labeled with the participant and the community. For example, a comment 
from participant three in youth group in community A was assigned the number A3Y. The team coded all 
KIIs and FGDs, and then examined patterns in the data to highlight changes over time (e.g., changes in 
income) and track themes that may not be apparent from a review of the notes or transcripts. Pattern 
analysis provides another tool for assessing what ideas, phrases, and words are repeated. 
GENDER CONSIDERATION
The evaluation team used a gender-sensitive approach in the design, methodology, data collection, data 
analysis and reporting of the results. More than 77 percent of the FGDs were women. The qualitative data 
collection and analysis devoted particular attention to investigating post-project outcomes by gender and 
12
capturing situations, experiences, and outcomes of both male and female youth who participated in and 
benefited from PAJE-NIETA activities.
EVALUATION LIMITATIONS AND MITIGATION STRATEGIES
This section discusses the risks and limitations that affect the reliability and validity of the evaluation 
findings, and the mitigation strategies adopted by the evaluation team. The evaluation team identified the 
following five limitations inherent to any ex-post evaluation: (1) Challenges with attribution of outcomes 
that materialized after the project closure; (2) Inability to locate project beneficiaries and key stakeholders 
who are knowledgeable about the project; (3) Difficulty in finding key project documentation; (4) Poor 
quality of the data collected by the project; and (5) Recall biases resulting from respondents’ poor 
recollection of past events. Details of the evaluation limitations and mitigation strategies adopted by the 
team are presented. 
● Challenges with attribution of outcomes that materialized after project closure: A 
challenge in ex-post evaluations is how to assess the extent to which an intervention impacted
particular outcomes that materialized years after project closure. The difficulty is that the 
beneficiary population do not live in isolation. It is quite possible that project beneficiaries will be 
exposed to other literacy, vocational training, and job creation projects or any intervention at the 
end of PAJE NIETA. Separating the contribution of other projects on PAJE NIETA’s impact will 
be incredibly challenging. The evaluation team employed contribution analysis to assess the 
contribution of PAJE-NIETA to documented results. In determining how USAID’s efforts may have 
contributed to the results of PAJE NIETA’s intervention, the evaluation team sought information 
about other similar interventions implemented by other actors such as the Malian government, 
other donors, international or local NGOs and the private sector.
● Inability to locate project beneficiaries and key stakeholders who are knowledgeable 
about the project: A common challenge in ex-post evaluations is finding participants and project 
staff after the project or activity that implemented the intervention has ended. USAID/Mali shared 
contact details of project staff with the evaluation team, but all the project staff had left the 
implementing partner organizations and could not be reached at the contact details provided by 
USAID/Mali. However, with the names, the evaluation team used their professional networks to 
track them. 
● Recall biases resulting from respondents’ poor recollection of past events: Given that 
the project has ended, the evaluation team agreed that key informants, including the youth 
beneficiaries would not accurately recall the details of the project interventions. Furthermore, it 
is possible that other projects have provided similar support, which could lead to confusion in the 
minds of respondents. The evaluation team relied heavily on existing documentation on the 
project and used precise and concise field data collection tools as well as experienced data 
collection staff for the evaluation. Project training manuals, names of volunteers, flip charts with 
timelines were used to aid youth to recollect events related to PAJE-NIETA. 
● Difficulty in finding key project documentation: Accessing project related documents after 
the project has ended and project staff are no longer available is a challenge associated with ex￾post evaluations. The evaluation team relied on USAID/Mali and AFR to obtain all available 
documentation on the project as specified in the SOW8. USAID shared the contact list of key 
8 Although USAID shared with the evaluation team all the documents mentioned in the SOW for review, it should be noted that there was no 
End of Project Evaluation. This document is different from the Project Final Report. Typically, ex-post evaluations compare project outcomes at 
the end of the project with outcomes years later. In the absence of the End of Project Evaluation, the evaluation team adopted a modified 
13
project personnel, other stakeholders, and list of youth groups in each region and by cohort to 
guide the sampling. The lack of end of project evaluation report meant lack of data on literacy, 
skills or employment at individual, cohort, and regional level for before-and-after analysis. 
● Unavailability and Incompleteness of data collected by the project: Reviewing project 
documents received from USAID/Mali and AFR, the evaluation team noticed that due to privacy 
concerns, there were no youth association member names to track project beneficiaries, except 
village names and total number of males and females in each group and cohort which were duly 
provided. The evaluation team used the PAJE-NIETA mid-term evaluation data conducted by 
CERIPS to recreate the beneficiaries. With the list, a scoping exercise was conducted with the 
objective of determining youth’s availability and readiness to participate in the focus group 
discussions and key informant interviews.
version of the Sustained and Emerging Impacts Evaluation to examine the resources, partnerships, capacities (and even ownership) which 
emerged after project closure to sustain project outcomes.
14
FINDINGS
EVALUATION QUESTION 1
1a. To what extent and in what ways did PAJE-NIETA sustainability strategy contributed to 
maintaining jobs and enterprises for youth and still supports the creation of jobs and youth 
enterprises in Mali?
To examine how structures and systems put in place by donor-aided projects led to local sustainability, 
the evaluation team looked for sustainability planning by donors and implementers which included 
partnerships with the Government of Mali (GOM) and other external stakeholders continuing activities 
for local success in entrepreneurship abilities due to the livelihoods element of the project. Project 
partners linked in networks were asked for their knowledge about effectiveness of PAJE-NIETA’s 
sustainability strategy and continuation of the project’s direct activities. The evaluation team also examined 
direct results being sustained and/or emerging ways the project continued in different forms at local levels. 
Do local and national structures, systems, networks, and linkages established or strengthened by 
the project exist; do they continue to support and promote youth literacy and business skills, a 
favorable economic environment for youth (access to credit, savings programs, and employment 
opportunities)?
PAJE-NIETA did not have an explicit sustainability strategic document that was shared with both the 
partners and the Government of Mali. However, to promote sustainability of project activities, the 
project employed several strategies, including capacity development of existing local youth associations, 
integration of youth volunteers, development of a local training network, and implementation of the 
private service provider (PSP) approach designed by CRS to foster sustainability of savings groups. 
Furthermore, the project collaborated with the Government of Mali (GoM) to help ensure that materials 
developed, and lessons cultivated under the project benefit the GoM’s efforts in education and youth 
development.
There were loose ends regarding the sustainability strategy employed by PAJE-NIETA. First, the project, 
at closure, was not handed over to the GoM as a whole. Neither was it handed over to 
the respective ministries nor any partners in the relevant sectors. For example, the basic education 
component could have been handed over to the Ministry of Education, the entrepreneurial training to the 
Ministry of National Entrepreneurship and Employment, and the SILC component to the Ministry of 
Finance. Secondly, the lack of continuation of collaboration weakened the project's long-term 
sustainability or at least the continuation of its results at the local level. From project documents, there 
were in-country peer partners supporting the project during implementation and after its closure these 
partnerships were missing. Thus, no institution exists to continue to assist unemployed youth at any level 
of government viz. local (commune), mid-level (cercle), upper mid-level (region) or high-level (national). 
Therefore, the lack of a sustainability strategy plan geared toward each of these four layers of authority 
has been a missing link in the process. This created a vacuum that hampered the conditions of the success 
of the project after its closure.
While trainers and accompaniment have ceased, community members continue to cross-train others. 
This is a new emergence of on-going training. The Deputy Mayor of Kita said: 
“PAJE-NIETA trained only one tailor in the village of Katakoto, but he has trained 18 other people who continue 
to sew. And poultry farming has generated enthusiasm among other young people.” 
Despite these local efforts, as one of the Kita government interviewees said, “PAJE-NIETA has done its 
best, but nothing has been done for sustainability because it required continuous training for beneficiaries and 
young unemployed people.” This was lacking post-project and was one of the expectations of USAID’s 
sustainability strategy – that the GoM would take over the continued support.
15
Although PAJE NIETA developed a local trainers’ network, it did not last. A former trainer lamented:
“The trainers of the different specialties deplore the fact that the PAJE-NIETA project has not put them in a 
network. This network could have led and facilitated initiatives to vocational training of unemployed young people.”
SILC financial resources created favorable conditions for sustainability after the project resources ended. 
But SILC modules were incomplete, especially in Cohorts 1 and 2, and links to microfinance institutions
were incomplete. SILC/microcredit can buffer risk when shocks at the family level can weaken the income￾generating activity and SILC training components were variable. Some cohort 1 communities were trained 
in only savings, while later cohorts received savings and lending. SILC groups in the eight villages are 
operational, though not using all the elements. 
In Koulikoro, some participants reported that while SILC was useful, not everyone honored its rules. 
“Everyone could borrow and pay it back with 10 percent interest, but some members stopped doing SILC because 
the lenders did not follow the rules and pay the money they had borrowed. People had become annoyed and were 
discouraged from continuing. They had two bags in which the put money, one for social problems (25 francs) every 
week and one to lend to each other and pay back with interest received shared.”
In Kayes Region, the range of SILC technical elements shows the variability of how well they can serve 
participants or not. For example, participants in Horongo have maintained their original SILC activities. 
They are still active and have two groups of 25 and each member deposits 25 CFA weekly. Their 
borrowing interest rate is 10 percent, and the groups distribute their funds every June, just before the 
rainy season and start the next cycle immediately. Their only wish is that the project had better link them 
to microlending projects so they could make better investments with their funds and have access to more 
loans to help grow their businesses. Both FIER and INCLUSIF9 are microlending in Horongo since PAJE￾NIETA ended. Some PAJE-NIETA participants have tried these projects’ services, but all confirmed that 
PAJE-NIETA was better at supporting the youth to access credit. Horongo youth expressed…”these 
organizations wanted to take our money, open a bank account, and manage the account using their mobile phone. 
We are skeptical of the process but if these projects were co-implemented with PAJE-NIETA, most youth would 
have subscribed to the services.” 
1b. Have PAJE-NIETA support structures, tools, training materials and curricula been replicated?
In order to ensure that youth beneficiaries had the necessary tools to become better educated, more 
economically productive and civically engaged, the Mali Out-of-School Youth project developed a series 
of products covering various technical areas that were used as training tools. The products included the 
following:
1. Basic Education: The basic education and curriculum manuals developed is a two-year 
curriculum (Year 1 and Year 2) consisting of teacher and youth guides in three subjects: i) reading 
and writing in native language (Bamanankan for the south and Songhay for the Timbuktu region in 
the north), ii) math and iii) functional French (oral and written).
2. Technical Training: PAJE-NIETA under the leadership of Swisscontact , developed technical 
training modules and materials to assist the youth to acquire minimum technical skills necessary 
to launch and manage a microbusiness in a trade of their choice. The Mali Out-of School Youth 
project designed training modules (trainer manuals and user manuals) as well as accompanying 
materials for 14 agro-enterprises and service income generating activities: poultry raising, baking, 
grain cultivation and market gardening, hairdressing, sewing, livestock fattening, masonry, 
carpentry, photography, telephone repair, repair of agricultural material, setting up a small 
restaurant, soap making, and processing of agricultural products.
9 The Inclusive Financing Project for Agricultural Sectors
16
3. Savings and Internal Lending Communities (SILC): PAJE-NIETA established SILC groups 
using a methodology that implementing partner CRS has successfully used around the world. 
The project adapted the already existing SILC curricula for use with rural youth in the Malian 
context.
4. Civic Engagement: As part of the project’s civic engagement activities, the project developed 
a civic engagement training manual consisting of four modules aimed at strengthening the capacity 
of existing youth associations to plan and implement community service projects.
5. Citizenship, Conflict Resolution and Promoting a Culture of Peace: A module developed 
to ensure the youth had a clear understanding of the concepts and strategies of citizenship, conflict 
resolution and promoting a culture of peace.
The evaluation team found that some UN-bilateral projects, such as Formation Professionnelle, et Appui 
à L’entrepreneuriat des Jeunes Ruraux (FIER), funded by a combination of IFAD with French/German aid 
directly continued PAJE-NIETA’s capacity development dissemination by funding the production of the 
manual for youth vocational training. The manuals were taken up by the FIER project via GRET Mali.
AJA, an implementing partner of PAJE-NIETA continues its vocational training of the youth in Mali. Its 
training manual was inspired by PAJE-NIETA’s vocational training manual. While CRS continues to do 
SILC, the evaluation team found no integration of PAJE-NIETA’s structures after it ended. On the other 
hand, EDC replicated PAJE-NIETA’s training and support programming model in three other projects. In 
Mali, the Education Recovery Support Activity (ERSA) funded by USAID in Gao and Menaka Regions built 
youth capacity until it closed in 2020. EDC also used USAID funding to replicate PAJE-NIETA in the 
Democratic Republic of Congo (DRC) through the Integrated Youth Development Activity
(IYDA). Finally, EDC partnered with the Mastercard Foundation to replicate similar programming in 
neighboring Senegal in the Improvement of Work Performance and Entrepreneurship Program (APTE 
Project). 
1c. What aspect of PAJE-NIETA sustainability strategy 
worked or did not work well and why?
During implementation, the project supported the capacities 
of youth associations, including incorporating volunteers 
such as the Mali Youth Volunteer Corps, developing a local 
training network, and implementing the Private Service 
Provider network linked to CRS’ SILC groups to promote 
sustainability of project activities. Additionally, the project 
collaborated with the Government of Mali to ensure that the 
materials developed, and lessons cultivated under the 
project will benefit the GOM’s efforts in education and 
youth development. The subsequent text describes what 
aspects of PAJE-NIETA’s sustainability strategy worked or 
did not work well and why.
Except for the pre-existing youth associations, which 
functioned at variable levels, none of the sustainability 
strategies were found to be sustained. For example, linkages 
to employment networks and market information systems 
including via telecom service providers were nonexistent 7-
9 years10 as the mobile devices went obsolete. Private sector 
mentors and PSPs were not sustained in the eight villages 
10 The project was implemented in cohorts. Cohort 1 ended in 2013 while the final cohort, cohort 3 ended in 2015. See Table 1 for details.
Amadou BAH, former EDC staff, currently with GRET 
(French). Employment expert who is directing a training 
program similar to PAJE-NIETA
17
visited. Private sector employment and private sector support did not materialize, possibly because it was 
mostly in Cohort 3 and in part because the project did not meet private sector needs by industry. 
Employment creation outside of communities by the project failed, in part because of limited opportunities 
in rural areas and limited absorptive capacity in peri-urban in what participants were trained in for a brief 
time. This leaves participants to generate jobs for others in their community, e.g., peers, children, other 
members of the community. No PSPs were found to function to generate jobs or mentors and no IT 
livelihoods were found working.
Partner interviews at both local and regional levels confirmed the same. This was due to the noticeably 
short duration of any one cohort (1 year). EDC, CRS, Swisscontact and AJA could not ensure all project 
training materials and inputs were ready for the project launch was weak. Many communities reported 
the IGA kits and SILC preceded the basic education training, and SILC training was incomplete in entire 
communities (e.g., savings without lending) in earlier cohorts. All the partners interviewed confirmed that 
a successful project relies on efficient resources and partnerships leveraged on the ground.
Collaboration with government institutions did not work well. Political and civil unrest made continuation 
and scale-up very unlikely. EDC could not collaborate closely with these national institutions because of 
political sanctions due to the coup d’état in 2012. At the project exit, there was no sustainability plan to 
work with the government on logistics, financial resources or other issues needed to be resolved to 
ensure project outcomes are sustained. After the project closure, resource provision changed 
dramatically. USAID financial support ended and the Malian government budgetary support to sustain the 
project was extremely limited. While other projects were implemented and PAJE-NIETA was replicated 
in northeast Mali and outside of Mali with other funding, no sustained resources supported PAJE-NIETA 
participants through new long-term investments. Given the departure of the Government of France from 
Mali in 2022, and the partial reduction of USAID’s mission staff during this evaluation, the security situation 
does not make more funding likely, at least in the near future.
While the private sector and trainer/ mentor volunteers disappeared as supportive networks, and m￾Learning stopped when mobile phones died, and no other partners helped in sustaining capacities or 
augmenting any training in the eight project villages, local individuals emerged as powerhouses. For 
example, community members cross-trained and lent to each other to pursue their livelihoods. Some 
women passed on their soapmaking skills to other community members before leaving their villages to 
marry elsewhere. 
1d. What are the factors behind the effectiveness or sustainability of PAJE-NIETA intervention?
USAID, in its sustainability definition, expected that both national and foreign aid stakeholders would 
commit resources to sustain PAJE-NIETA results by investing in the education system, local educational 
capacities, and that communities would self-generate livelihoods, having been “educated, economically 
productive, have been trained to perform a job and civically engaged.” 
There was an implicit assumption of handover of projects to national governments that assumed their 
uptake of programming, but it was not explicit in the Request for Proposal (RFP). While respondents in 
the GOM’s education sector deeply appreciated PAJE-NIETA, they confirmed that the partnership was 
weak, their resources were so limited that their ability to own and scale-up youth capacity development 
and support was impossible. 
All the assumptions with the exception of village-level self-employment were not sustained. Ministry and 
other donor support was assumed based on its potential to lead to sustained outcomes of employment, 
engagement in communities. The model’s theory also assumed partners’ continued investments in 
sustaining such employment and training. Although appreciated by the Malian government, especially those 
in basic education and training, the political and economic situation in the country has not facilitated 
investments since closeout.
18
From the perspective of the local partners and the village, the sustainability strategy meant training 
communities in literacy and numeracy, work readiness and savings and internal lending (SILC) skills that 
would last long and that would help them grow their businesses. PAJE NIETA did that, but implementing 
partners expected the project to bring in new funding that they could use after the project–with a view 
of replicating it in other regions of the country or elsewhere.
At the local level, the project faced challenges during initiation for youth to join. The second Deputy 
Mayor of Kita noted: 
“My role was to sensitize young people and women from the villages to join the project, because some were 
reluctant.” 
However, he thinks PAJE-NIETA created both short- and long-term employment for young people in his 
community. He added: 
“My wife continues to sell soap because of PAJE-NIETA. Today, all those who do poultry farming know how to 
vaccinate their poultry without the help of a veterinarian and the tailors trained by PAJE-NIETA have their own 
sewing workshops in the village.”
Emerging sustainability focuses on the ability and capability of the project beneficiaries to sustain results 
on their own (motivation), but in diverse ways. Project beneficiaries continue to invest their 
resources, time, and labor in livelihoods they were trained years after project closure. Innovations such 
as spontaneous training one another, lending to one another and such unplanned new pathways to 
sustainability, changing how the project was implemented, but being built on how locals redesign to 
sustain results is one of the key innovations.
Participants have taken the financial gains from poultry raising, livestock fattening and market gardening 
and invested back in new businesses or have taken the place of donors as lenders to guarantee a long￾term sustainability as well as returns on investment. In many PAJE-NIETA villages, lending, and training 
to one another has become pervasive. One woman the evaluation team interviewed in the Koulikoro’s 
pretest Kassela village, used income from her market gardening livelihood to start a different business in 
reselling textiles from Burkina Faso. 
PAJE-NIETA catalyzed enterprises that some new youth access via their 
families and communities. Yet in a context characterized by high 
unemployment in a country like Mali, any job or enterprises created no 
matter how limited their number is a positive action but there was no 
emerging (locally created initiatives) by the Malian government or other 
partners but only locally led.
Moussa Taunkara of Baliani in Kayes Region learned how to raise poultry 
and overcame the challenge associated with his first PAJE-NIETA 
venture. He did not receive the promised vaccines, and therefore all his 
chicks died. Yet with the training received independently he has 
sustained and grown a healthy poultry business. He helps others by giving 
chickens, loans, and advice, and repaid when new owners sell their first 
chickens. Although he is still involved in agriculture and has taken up 
animal fattening, his net income has increased three-fold from 600,000 
CFA at project end to closer to 2,000,000 CFA. 
On civic engagement, PAJE-NIETA strengthened the capacity of existing 
youth associations in the respective implementation villages, to carry out 
community development activities. In Sikasso’s three villages, the youth 
reported during the implementation of PAJE-NIETA, they engaged in 
reforestation, trash collection activities, school and health centers 
Moussa Taunkara with his rooster in 
Baliani, Kayes Region. 
19
construction including the Sogola Youth center, the Mena Health Center, and the M’Pegnesso Market 
Hangar. However, post-project closure, civic activites has been dependent on local inputs to refurbish
buildings and clean villages. Among the communities, the frequency of civic activities was highly variable.
In Koulikoro and Kayes Regions, respondents said that there was nothing done since building homes for 
teachers and school desks during the implementation of PAJE-NIETA. This section demonstrated that 
resources, capacity, partnership, and motivation are interrelated and synergistic to achieve sustainability.
EVALUATION QUESTION 2
To what extent and in what ways were PAJE-NIETA basic education, work readiness, 
financial and business skills programs effective in empowering youth to create their own 
enterprises and become more civically engaged in their communities? To what extent are 
these programs replicable in similar activities? 
2a. Which elements of the Mali Out of School Holistic and Integrated Learning Model were most 
effective in strengthening the capacity of youth to create and overtime maintain their businesses 
and civic engagement?
In addition to providing basic education, technical training and start-up kits, a principal input from the 
project along the microenterprise start-up pathway, was support for youth to create savings and internal 
lending communities (SILC). The evaluation found that among all the key elements of PAJE-NIETA, 
project participants ranked basic education as the most essential element for the sustainability of their 
livelihoods. This was followed by entrepreneurial training, and then IGA kits. SILC ranked least among 
the most effective elements of the model to strengthen their capacity and maintain their business 
overtime. See Table 4 for Sikasso rankings of most valuable PAJE-NIETA element. These rankings were 
similar for the other regions.
TABLE 4: SIKASSO RANKINGS OF MOST VALUABLE PAJE-NIETA INPUTS FOR SUSTAINABILITY
DOMAIN/ VILLAGES SOGOLA MENA MPEGNESSO TOTAL
Literacy/Numeracy 47 80 245 372 = 49%
Vocational 
Training/IGA
23 79 114 216= 28%
SILC 20 62 89 171= 23%
Total 90 221 448 759
In all the regions, the three key project components of basic education and vocational training were all 
instrumental in equipping the project participants with continued success. Most participants remarked 
that knowing profit and loss was one of the most important things they learned and that they could use 
it in other areas of their lives. The literacy and numeracy skills combined with job competencies and 
providing some of them with in-kind (IGA kits) and access to financial resources (via SILC groups) were 
necessary for short-term and sustained success. The combination was praised by one female interviewee
in Kalassa, Koulikoro Region: 
“There has never been a project like PAJE-NIETA.”
Nonetheless, the project trained participants in different livelihood activities that provided participants 
with secondary and tertiary income sources. While agricultural activities improved through literacy and 
numeracy, most respondents (especially women) reported improvements in their non-agriculture 
income. Participants praised basic education’s positive effect (especially financial literacy) on their 
20
livelihoods, and parents commended PAJE-NIETA for helping them to assist their children with their 
school assignments. 
One such parent is Mariam Soucko from Katakoto, Kayes 
Region. Mariam has subsistence agriculture as her primary 
livelihood and she integrated PAJE-NIETA soap making 
livelihood activity during the rainy season. This has increased 
her income by 40 percent. She praised PAJE-NIETA’s basic 
education because it has helped her to support her children’s 
school assignment. She still has a good relationship with her 
education mentor.
The majority of participants during the interview were women 
and they were still self-employed in the livelihood in which they 
were trained. These participants were using their PAJE-NIETA 
livelihoods to generate secondary or tertiary income, to 
supplement their subsistence agriculture. Table 5 depicts the 
current livelihood practiced by respondents. The most 
common livelihood practiced by respondents was agriculture 
(which in later cohorts were separated into market gardening 
and other activities). This is followed by poultry raising, 
tailoring/sewing, soap making, animal fattening, food processing, 
and restaurant/ food sales. Accompaniment or connection to 
past trainers was exceedingly rare
TABLE 5: : CURRENT LIVELIHOOD PRACTICED BY RESPONDENTS
LIVELIHOOD REGION TOTAL
KOULIKORO SIKASSO KAYES
Agriculture 0 24 37 61
Market Gardening 3 31 16 50
Animal Fattening 4 10 6 20
Poultry 20 16 10 46
Welding 3 0 2 5
Food Processing/ Baking 2 1 3 6
Restaurant 2 7 2 11
Tailoring/ Sewing 0 0 8 8
Soap making 7 17 7 31
Total 41 106 91 238
Djénéba Thiam, Research Enumerator, conducting an 
interview with Mariam Soueko about her soapmaking 
skills she learned from the PAJE-NIETA Project in 
Katakoto, Kayes.
21
In Sikasso, an overwhelming majority of the participants 
(about 95 percent) in the three villages interviewed have 
agriculture as their primary activity and still practice their 
PAJE-NIETA livelihoods for which they received training. 
Some who received diverse types of livelihoods’ trainings 
were especially successful. One of the few male 
participants, Garanke Mady Nomogo of Kayes, had two 
livelihoods: agriculture and men’s tailoring that he learned 
from his father. He joined PAJE-NIETA to learn to sew 
women’s clothing, which has become his primary livelihood 
with a shop in the center of the village. He is one of the 
few who continued to be linked to the capacity 
development mentor. He said: 
“After the program ended, I could call the sewing teacher about 
problems I encountered.” 
His annual income is five times higher than it was when he 
joined the project, from 1 million CFA at the end of 
project in 2015 to 5 million CFA in 2022. 
The entrepreneurial training was highly diversified, 
providing women with income from multiple sources. The project’s training and support, according to 
the beneficiaries led to increase in women incomes and assets, for some more than five-fold. While 
PAJE-NIETA was designed and implemented before the current USAID gender policy was in force, the 
project reflected good gender practices, including facilitating women’s participation in project activities. 
This included an arrangement for nannies to take care of women with children in order to allow them 
to participate fully in the training.
For those who finished the basic education and vocational training, the training’s impact on production 
and productivity was noticeable. In Kalassa, Koulikoro region, one participant said,
“Education was useful for her because she can read, write and calculate.”
Another said that: 
“She had training in soapmaking, but education was useful for her because she can now 
do many things that she could not do before PAJE-NIETA; writing was not easy for her, but 
she can calculate and use her phone.”
While Income Generating Activities (IGA) were to boost employment prospects, most IGAs were not 
sustained. The overwhelming majority of beneficiaries complained that many roosters and poultry 
distributed to them died from fowl diseases. 
2b. Which elements of the Mali Out of School Holistic and Integrated Learning Model are likely 
to be replicated in similar activities?
The project’s sustainability strategy assumed that other projects would spontaneously adopt the most 
promising elements. Several projects have replicated PAJE-NIETA’s element of education, vocation, and 
microcredit since it ended in 2015. These include Project for the Development of Youth Skills and 
Employment (PROCEJ), Fonds d'Appui À la Création d'Entreprise par les Jeunes (FACEJ), Formation 
Professionnelle, Et Appui À L’entreprenariat des Jeunes Ruraux (FIER), and Agence pour la Promotion de 
l’Emploi des Jeunes (APEJ). However, none of these projects replicated the full model. For instance, the 
Garanke Mady Nomogo in his tailor shop in Katakoto, Kayes 
Region 
22
FIER project accepted the training materials for literacy and numeracy, but without the financial resources 
to continue providing IGA kits, SILC, and other capacity building inputs. 
Following PAJE-NIETA, Project for the Economic Empowerment of Women in the Shea Butter Subsector 
(PAEFKK) trained women in Kayes in shea butter production, a fine addition to soapmaking. Shea butter 
has become too expensive an input to buy for soapmaking, so they now use their own shea butter. This 
has built on PAJE-NIETA success, to the degree that they resell the shea butter soap to PAEFFK which 
gives them an easy market of their products. Marian mentioned above has also trained one woman in 
soapmaking. .
Kita’s Deputy Mayor also confirmed that elements of PAJE-NIETA continued through projects such as 
FIER, and through the Department of Agriculture’s new project which includes market gardening, one of 
the livelihood components under PAJE-NIETA. He was quick to add that the projects are temporary, and 
many beneficiaries lament the lack of an integrated strategy for the unemployed youth. While there is a 
national youth development and employment strategy, resources and partnerships are needed to 
implement it. Findings in section 1b discussed some elements of the model which are being replicated by 
other projects.
There were additional limitations related to research and programming which affect ability to draw 
conclusions about replicability of the project. This includes factors such as intensity, duration, and quality 
of training and support as well as changes in targeted youth and scope of livelihoods. According to CRS
and EDC, PAJE-NIETA was envisioned as a three-year project, but political unrest and its effect on 
incoming funding extended the project duration. The 2012 coup hampered the implementation of Cohort 
2 and 3 because funding was frozen for Mali. However, new USDA funding became available for Cohort 
2 focusing on a smaller agricultural project in Sikasso region, and new USG Democracy and Governance 
funding came on board to support r Cohort 3, adding peacebuilding modules to the training of the youth 
and including Timbuktu region, and more peri-urban livelihoods. 
2c. What were some of the lessons learned and best practices for implementing out of school youth 
programs that can be broadly shared? 
Effects of political and civil unrest on activity programming
Political and civil society upheavals within Mali have not fostered continued long-term donor commitments 
to education and entrepreneurship. PAJE-NIETA had to receive additional funds from the USG Democracy 
and Governance Team to continue the activity. Future programming should consider alternative sources 
to fund educational activities in case funding ceases during program design.
Programming should address new shocks but also emerging outcomes
Across Mali from 2012-2022, there have been a series of shocks during the project and ex-post affecting 
livelihoods across the project areas visited. This began with the coup d’état in 2012 affecting both Cohort 
1 and 2 and had financial implications for PAJE-NIETA programming. From 2019-2020, COVID affected 
the whole country and from 2021-22 a global embargo due to the international reaction to the coup d’état 
takeover led to 300 percent inflation. The extent of inflation recently faced by communities is illustrated 
by women soap makers in Katakoto, Kayes Region:
“In 2021, we could not make soap as the price of shea butter rose from 12,000 CFA11 to 16,000 CFA. Firewood 
became scarce and the price rose threefold from 5,000 CFA to 15,000 CFA”.
The communities have been exposed to climatic, political and security hazards which negatively impacted 
livelihoods. Due to these series of shocks in the sahel, livelihood training targeted at out-of-school youth 
should have diversified incomes. While all PAJE-NIETA participants have agriculture as their primary 
11 1 USD = 619 CFA
23
livelihoods and the project trained people in secondary or tertiary livelihoods, such secondary livelihoods 
are vital.
The project had no effect on its ultimate goal of stopping migration
Migration continues to be an important source of income for the villages. Migrants, especially men, send 
money to help their family in difficult moments, thus migration serves as an effective economic coping 
strategy. Few men participated in the project and the short year-long duration of each cohort made it 
impossible for them to join later. Men engage in agriculture during the rainy season and seek better 
opportunities during the dry season in the capital or other large cities or neighboring countries. Although 
the project could not stem outmigration, men’s migration remains seasonal and is region dependent. For 
example, gold mining in Kayes drew many men, but few out-migrate from Koulikoro. In Sikasso, migration 
is limited. 
Vocational Training IGA starter kits and skills were transferred and helpful but incomplete 
Several participants 7-9 years later are still using the IGA kits, but many did not receive their kits. Others 
too have their kits damaged when they receive them, which speaks to logistics issues of an inexperienced 
prime implementer. Also, very few mentioned vocational training mentors still being accessible and there 
were no thought support items would need to be replaced, so it was not planned for either in the short 
or long-term. EDC’s logistical skills, especially in the vocational training kits, were insufficient and CRS had 
logistics skills that were not used, which EDC needed. The launch year was very much delayed, leading 50
percent of those enrolled to drop out. 
Conduct localized market research on livelihoods for vocational training and cap the number of 
participants per livelihood
An element of the holistic model was the training of the youth in vocational training. However, the projects 
many livelihoods did not always match market demand with supply to help participants find and continue 
to practice their livelihoods. If a vocational area has a limited saturation point, such as soap making, there 
is a need to limit the number trained in each site or near each other if the market is similar.
For example, in Koulikoro region, interviewees said soap became glutted in the community making it 
difficult to sell. Some participants still do soapmaking as secondary or tertiary livelihood but only during 
the rainy season. The main problem was the high number of females who were trained in soapmaking. 
One of the women said, “If all of them produce soap, they all have a problem selling their soap in their village 
and in the surrounding villages. Some women, therefore, make only liquid soap now. I make soap twice a month.” 
EDC echoed this as a problem, as the project found it difficult when many youths wanted to pursue the 
same livelihood, e.g., tailoring. The project market research could have informed them better via local 
market demand studies about how to cap training in some livelihoods over others, rather than relying on 
regional market research the project did which did not get such local nuances.
Plan for obsolescence of IT Resources
PAJE-NIETA’s mLearning component was intended to reinforce the learning that occurred in class. Each 
cell phone provided youth with a set of lessons in literacy, numeracy, functional French, and 
entrepreneurship that were linked to the courses taught in class.
In addition, an audio drama of 85 episodes was provided that chronicled the efforts of two youth to launch 
a poultry-raising business (including their problems and challenges and how they resolved them). During 
the evaluation team’s visit, most participants do have their phones since they have become obsolete. They 
could no longer refresh their mind on the training received with their mobile phones. The phones were 
given free of charge which could also impact sustainability. Participants were not motivated to get their 
own mobile phone to access the m-Learning platform.
24
CONCLUSIONS
The evaluation reached seven conclusions based on the findings:
● PAJE-NIETA did not have an explicit sustainability strategic document that was 
shared with both the partners and the Government of Mali. However, to promote 
sustainability of project activities, the project employed several strategies including the capacity 
development of existing local youth associations, the integration of youth volunteers, the 
development of a local training network, and the implementation of the private service provider 
(PSP) approach designed by CRS to foster sustainability of savings groups. There were loose ends 
regarding the sustainability strategy employed by PAJE-NIETA. The project, at closure, was not 
handed over to the Government of Mali as a whole nor to the respective ministries or any 
partners in their sectoral areas.
● Implementing partners such as AJA, and EDC, continue to use PAJE-NIETA support 
structures, tools, and training material. EDC replicated PAJE-NIETA’s training and support 
programming model in Mali, Senegal, and Democratic Republic of Congo. UN-bilateral projects 
such as Formation Professionnelle, et Appui à L’entrepreneuriat des Jeunes Ruraux (FIER) 
continued PAJE-NIETA capacity development by funding the manual for youth vocational training. 
The curriculum for the manual used PAJE-NIETA’s manual as a guide.
● With the exception of the village associations that existed, none of the sustainability 
strategies employed by PAJE-NIETA was operational. Community members self￾trained each other. Linkages to employment networks and market information, including via 
telecom service providers, were nonexistent because the equipment became obsolete. 
Collaboration with government institutions did not work well due to political and civil unrest 
during project implementation and lack of financial resources from the government. Private sector 
mentors and PSPs were not sustained in the eight villages visited. Youth members who received 
training on specific livelihoods trained others. 
● Basic education was most effective in strengthening the capacity of youth. The 
evaluation found that among all the key elements of PAJE-NIETA, project participants ranked basic 
education as the most essential element for the sustainability of their livelihoods. This was 
followed by entrepreneurial training, and then IGA kits. SILC ranked least among the most 
effective elements of the model to strengthen their capacity and maintain their business overtime. 
Majority of the women were still practicing the livelihood they were trained in. Most were using 
their PAJE-NIETA livelihoods to generate secondary or tertiary income, supplementing their 
subsistence agriculture. 
● Several projects have replicated elements of PAJE-NIETA’s holistic and integrated 
learning model, but none replicated the full model. Elements of PAJE-NIETA’s model such 
as basic education, vocational training, and SILC have been replicated by the following activities: 
❖ Project For the Development of Youth Skills and Employment (PROCEJ), 
❖ Fonds d'Appui À la Création d'Entreprise par les Jeunes (FACEJ), 
❖ Formation Professionnelle, Et Appui À L’entreprenariat des Jeunes Ruraux (FIER), and
❖ Agence pour la Promotion de l’Emploi des Jeunes (APEJ)
25
FIER project adopted the training materials for literacy and numeracy but lacked the financial 
resources to continue providing IGA kits, SILC, and other capacity building inputs. 
● Political and civil unrest during project implementation hampered government 
collaboration. Due to political and civil unrest, the project could not integrate the Malian 
government in its activities. Additionally, the upheaval did not foster continued long-term donor 
commitments to education and entrepreneurship, as USAID funding to PAJE-NIETA ceased.
● Community members learned from one another to diversify their income sources in 
the face of these shocks. PAJE-NIETA trained youth in livelihoods with diversified incomes. 
The Sahel experienced a series of shocks during the project and after project closure. The shocks 
included coup d’état in 2012, COVID-19, and inflation which negatively impacted livelihoods. 
Because the project trained youth associations in different livelihood., Community members 
learned from one another to diversify their income sources in the face of these shocks.
26
RECOMMENDATIONS
Based on the findings and conclusions, the evaluation team proposes the following recommendations to 
USAID, future implementing partners, and the Government of Mali:
USAID
1. USAID should require implementing partners to submit a sustainability framework with every 
proposal. USAID should ensure the sustainability framework details a sustainability 
implementation pathway within the intervention design with clear sustainability timelines and 
benchmarks and plans to document all original data from the intervention. The data should be 
made accessible to USAID. 
2. USAID should ensure in measuring project impact; evaluations must incorporate indicators of 
sustainability. This implies, implementing partners need to demonstrate evidence of resources, 
capacity, partnership, and motivation continuing to sustain benefits at the end of the project.
3. USAID should consider adjusting its evaluative processes and extending future youth projects 
beyond 5 years when there is evidence of progress toward sustainable impacts and indications of 
potential for sustainability.
4. USAID and its partners should strengthen their capacities, as necessary, to institutionalize 
sustainability in programming through training and improved knowledge management, as well as 
strengthened organizational commitment to look beyond immediate impact to sustainability.
5. USAID should ensure continued and consistent use of a system whereby implementing partners 
archive all baseline and evaluation reports including accessible and documented original data in 
order to enable post-project evaluations.
6. USAID should design future projects such that exit is gradual, with a phased transfer of 
responsibility to the appropriate stakeholders; exit should follow a phase of incrementally 
independent operation, and project beneficiaries and beneficiary communities should be engaged 
in plans for sustainability and exit from the beginning of the project cycle.
7. Projects should be designed with the local context (economic, political, and social/ cultural 
systems) in mind, should take account of the need for resilience in the face of climate or other 
shocks and stressors, and should be updated in response to changes in the local context.
8. Project design should incorporate strategies for sustaining beneficiary demand as well as
supply of services.
Implementing Partners
1. Implementing partners should base their sustainability plans and related exit strategies on clearly 
articulated theories of change. Implementing partners need to assess carefully and realistically the 
assumptions underlying their sustainability plans and reassess them continually to account for 
change in the external environment.
2. Implementing partners should conduct surveys to assess the durability and sustainability of 
enterprises during project implementation.
Government of Mali
1. The Government of Mali should work with donors to integrate policies and programming, such 
as how projects feed into youth development policies. Further benefits would come from 
government investments in universal education being taught throughout the country and that basic 
27
education including financial literacy, vocational training, and support – the three pillars of the 
program–are included. But given the political climate in Mali, such as requesting the French 
government’s aid program to leave, conditions may not be conducive now.
2. If funding is available, the government should trace the use of PAJE-NIETA skills gained by former 
national staff to assess continued use and investment back into national growth, including any 
benefits of having government representatives.
3. Mobilize more financial resources for the promotion of basic and vocational education to offer 
more applied courses to out-of-school youth in areas where education is still lagging behind. Also 
ensure that the complementary training components and all inputs are ready for startup. 
28
ANNEXES
ANNEX A: SCOPE OF WORK
Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA)
Evaluation Statement of Work (SOW)
USAID/Mali and USAID/Africa Bureau seek to buy into the Analytical Support Services and Evaluations 
for Sustainable Systems (ASSESS) activity managed by USAID/West Africa and implemented by the United 
States Department of Agriculture (USDA) to conduct an ex-post evaluation of the USAID/Mali Out of 
School Youth Project, also known as PAJE-NIETA (Project d’Appui aux Jeunes Entrepreneurs), The 
evaluation will examine the sustainability of the USAID/Mali Out of School Youth Project outcomes, six 
years after project completion. The agro-pastoral nature of the entrepreneurial activities engaged in by 
the youth is of particular interest to USDA. The evaluation will be conducted by USDA partners- Kwame 
Nkrumah University of Science and Technology (KNUST) and the University of Rhode Island (URI). 
Funding for this activity will be made available under a PASA agreement USDA has with the Africa Bureau. 
However, 40 percent of the budget will be funded by USAID/Mali through the ASSESS PASA. The 
anticipated period of performance is from January 15, 2022 - June 30, 2022. 
I. Mali Out of School Youth Project Background
a) Introduction
PAJE-NIETA was a five-year, $25 Million project implemented from 2010-2015 by the Education 
Development Center, Inc. (EDC) in collaboration with Catholic Relief Services (CRS), Swisscontact, 
Association Jeunesse Action (AJA) and for the Timbuktu region, Association Malienne pour la Survie au 
Sahel (AMSS). The Education, Economic Growth, and Democracy and Governance Teams of USAID/Mali 
jointly funded the project. 
The project began with a Rapid Assessment of Malian youth conducted in April and May 2010 which
recommended that primary attention be given to youth living in rural villages and peri-urban areas in an 
attempt to reduce migration from rural areas to urban centers. The Rapid Assessment revealed that youth 
wanted to earn a living and achieve successful livelihoods in their home villages. It also revealed that youth 
are entrepreneurial and able to identify the inputs and knowledge they need to reach their goals, which, 
for rural youth, are in the agro-pastoral sector.
In order to address that need, USAID/Mali designed the Mali Out of School Youth Project with the goal 
to enable 10,000 rural youth between the ages of 14 and 25 who had either never been to school or who 
had dropped out in the early grades to become better educated, more economically productive, civically 
engaged and empowered to improve their lives and those of their families and communities. 
b) Project Objectives/Intermediate Results
The project addressed four intermediate results of USAID/Mali’s strategic plan:
IR 1: Increased access to relevant basic education: The first project Intermediate Result was to enable 
youth to acquire basic academic skills, knowledge and attitudes needed to enter the job market. Youth 
were provided with work readiness, entrepreneurship and basic education training focused on literacy 
and math skills in local language as well as a functional level of literacy in French to negotiate the 
marketplace. This also included information needed to become better educated, more economically 
productive and civically engaged through training in life skills, leadership, and conflict resolution. It was 
29
anticipated that at the end of the project, targeted youth would be better equipped with basic 
education and work readiness skills needed to earn a livelihood.
IR 2: Increased access to relevant skills training: The project’s second intermediate result was to enable 
youth to acquire technical skills needed to earn a living within rural and peri-urban settings. Rural youth 
would engage primarily in agro-enterprise development training while peri-urban youth may focus more 
on ancillary vocational education skills training related to the agricultural sector, such as fabrication of 
agricultural tools. Related to this, the project sought to build the capacity of rural vocational training 
providers, particularly with respect to their ability to train out-of-school and illiterate youth. It was 
anticipated that at the end of the project, targeted youth would be better equipped with vocational 
education and technical skills needed to earn a livelihood while local skills providers have improved 
capacity to prepare youth for work. 
IR 3: Increased entrepreneurship opportunities and job networking: Under this Intermediate Result, PAJE￾NIETA was tasked to enable youth to acquire entrepreneurship and work readiness skills needed for self￾employment and job networking. The project provided youth with entrepreneurship and work readiness 
skills training and consulted with micro-finance institutions, local businesses, producers, and ateliers to 
increase economic opportunities for youth via self-employment or jobs. At the end of the project, it was 
expected that targeted youth would gain viable employment opportunities (self-employment or 
other) in the agricultural and vocational education sectors. 
IR 4: Increased civic engagement: The project’s fourth intermediate result was to increase civic 
engagement at community, regional and national levels. Youth development activities reinforced the notion 
of community and regional and national level civic engagement. The project supported youth in building 
the capacity of existing youth associations. Youth associations were guided to collect data, analyze 
community needs, and plan and execute community service projects with assistance from the project in 
the form of small grants.
c) Project implementation approach and Sustainability Strategy 
Implementation Approach
EDC and its partners developed an integrated holistic model of the youth development cycle to be used 
as the main guide for project implementation. The model is summarized in the diagram below: 
30
The following figure depicts the Mali Youth learning pathway which implicitly conveys the project’s 
Theory of Change. 
31
The project targeted a total of 10,000 rural and peri-urban youth, an equal number of children aged 14-
25 who were unschooled or out-of-school and had little or no literacy skills. It covered the regions of 
Kayes, Koulikoro, Sikasso, and Timbuktu, and followed a cohort-model consisting of three consecutive 
youth cohorts. Lessons learned from each cohort were used to improve interventions for the next 
cohorts. 
● Cohort 1 (June 2011-June 2013) targeting 5,740 in 100 villages in the regions of Kayes, Koulikoro, 
and Sikasso 
● Cohort 2 (February 2013-July 2014) targeting 1,255 youth in 21 villages (26 centers) in the Sikasso 
region
● Cohort 3 (June 2014-November 2015) targeting 7,858 youth in 107 villages (178 centers) in the 
regions of Kayes, Sikasso, and Timbuktu. 
Key project interventions included providing basic education training for youth covering literacy, 
numeracy, and life skills, upgrading the quality of technical skills for stakeholders involved in youth 
vocational training, providing accompaniment in the form of work readiness, financial, and business skills 
training as well as links to credit and savings programs to facilitate the purchase of start-up equipment. 
The project integrated information technology to support various interventions. 
Sustainability Strategy 
The project was designed with sustainability in mind. Key elements of PAJE-NIETA sustainability plan 
included: 
32
a. Strengthening the capacity of youth serving institutions and a local training network: The project 
provided organizational capacity building training to members of youth associations, all serving as 
volunteers. Of particular note, the project incorporated in its design an innovative Mali Youth 
Volunteer Corps activity under which 200 educated, unemployed youth who were recent 
graduates, were recruited and trained to serve as community-based literacy, work readiness, and 
youth leadership trainers. Youth were also mobilized to design and implement community service 
projects in their villages. 
b. Introducing the Private Sector Provider (PSP) Approach: The project initiated in its last year a 
private sector provider approach to enhance the sustainability of the project. Under this approach, 
communities selected a representative to serve as PSP candidate who received intensive training 
and got certified and prepared to provide services to community members after the project ends. 
PSPs receive payments for these services. 
c. Partnering with government institutions including:
● Ministry of Employment and Vocational Training
● The Fonds d’Appui a la Formation Professionnelle et L’Apprentissage (FAFPA/ 
Professional Training and Support Fund) in order to ensure the stability and sustainability 
of the Mali Youth Corps.
● The Ministry of Youth, Ministry of Agriculture, and the Ministry of Education.
● At the commune level: The mayor’s office, ministry representatives, community 
development councils, elders, and youth associations 
d. Youth linkages to the private sector:
● Support to ateliers for additional and improved skills standards, training modules, and 
implication in the accreditation process. Accreditation and certification will motivate 
youth learners in agro-enterprise and in ateliers to continue to use these local systems 
for skills training in the future.
● Linkages to employment networks and market information system via mobile phones; 
fostering linkages between local telecoms and service providers (agricultural market data, 
employment services) to ensure that these services will remain available after the close 
of the project.
e. Linkages created for agro-entrepreneurs, such as improved access to new markets (through 
opportunity identification analysis), improved access to agricultural market data and new 
processing and marketing methods will allow producers to be sustainable after the project 
intervention cycle is complete.
II. Existing Project Information and Data Sources
The following are the existing project information documents that might be useful in answering the key 
evaluation questions described in Section V below. USAID/Mali will share these and any additional 
documents with the evaluation team before the implementation of the evaluation.
a) Mali Youth Assessment Report
b) Project Description, Approval, and Modification Documents
c) Performance Monitoring Plan
33
d) Quarterly and Annual Reports
e) Project Baseline and Endline Surveys
f) PAJE NIETA Mid-Term Evaluation
g) Mali Out of School Final Project Report
h) PAJE-NIETA Final Evaluation Design (Power Point Slides)
III. Evaluation Purpose and Use
a) Evaluation Purpose 
The purpose of this ex-post evaluation is to assess the sustainability of project-supported interventions 6 
years after the project ended. 
ADS defines sustainability as “The ability of a local system, network, or institutions to produce desired 
outcomes over time. Programs contribute to sustainability when they strengthen the ability to produce 
valued results and to be both resilient and adaptive in the face of changing circumstances” (ADS 201, see 
Definitions).
The evaluation seeks to understand the extent to which PAJE-NIETA sustainability strategy described 
above was effective and is still relevant in addressing the education and business needs of out-of-school 
youth. Specifically, and consistent with the definition of sustainability above, the evaluation will examine 
the extent to which local structures, networks, linkages, and systems established or strengthened by PAJE￾NIETA continue to function and contribute to out-of-school youth education, enterprise creation, and 
civic engagement. In addition, the evaluation will examine the extent to which the holistic and integrated 
out-of-school youth learning model empowered youth in creating and sustaining their own enterprises 
and civic engagement initiatives and how it can be replicated in similar activities. Finally, the evaluation will 
identify the factors behind the effectiveness and sustainability PAJE-NIETA interventions as well as lessons 
learned and best practices that can be broadly shared and replicated to improve the sustainability of future 
out of school programs. 
b) Audience and Uses of the Evaluation
The primary intended audience for this evaluation is USAID/Mali, which will use the results to inform the 
design of an upcoming youth activity as well as to plan for future programs. The findings of the evaluation 
will also be beneficial for the Government of Mali, other donors, education stakeholders, USDA, the Africa 
Bureau, and other USAID/Washington Bureaus which will acquire a better understanding of the 
sustainability of alternative education interventions catering to youth and use findings and lessons learned 
to improve the support and increase impact of future interventions. The ex-post evaluation findings will 
also contribute to the overall Agency’s learning agenda questions focusing on program sustainability which 
is one of the key USAID programming principles. Other Agency and Africa Bureau programming priorities 
and approaches to the evaluation contribute to include resilience and multisectoral programming.
IV. Evaluation Questions
In light of the above purposes and uses of the evaluation, AFR and USAID/Mali would like to investigate 
and understand the following key questions and lines of inquiry related to the effectiveness of the 
sustainability strategy and implementation approach employed by PAJE-NIETA. 
Evaluation Question 1: To what extent and in what ways did PAJE-NIETA sustainability strategy contribute to 
maintaining jobs and enterprises for youth and still supports the creation of jobs and youth enterprises in Mali? 
34
In addressing this question, the evaluation team should consider the local and national structures, systems, 
networks, and linkages established or strengthened by the project and whether they continue to support 
and promote youth literacy and business skills, a favorable economic environment for youth (access to 
credit, savings programs, and employment opportunities). This question will also document any instances 
where the project-supported structures, tools, training materials, and curricula have been replicated. The 
evaluation team should also analyze what aspects of the sustainability strategy worked or did not work 
well and why.
Evaluation Question 2: To what extent and in what ways were PAJE-NIETA basic education, work readiness, 
financial and business skills programs effective in empowering youth to create their own enterprises and become 
more civically engaged in their communities? To what extent and in what context are these programs replicable? 
In answering this question, the evaluation team should examine the Mali Out of School Holistic and 
Integrated Learning Model to understand which elements of the model were most effective in 
strengthening the capacity of youth to create and overtime maintain their businesses and civic engagement, 
and which ones are likely to be replicated in similar activities as best approaches for sustaining of out-of￾school youth programs. 
V. Evaluation Design and Methodology 
1. Evaluation Design and Methods
USAID expects that the evaluation will use qualitative data collection methods and leverage, as feasible, 
existing PAJE NIETA quantitative and qualitative data, including monitoring and research data, other 
project implementation data (including quarterly, annual, and final project reports), internal PAJE-NIETA 
data and data from the mid-term evaluation. The following are illustrative data collection methods which 
will be finalized by the evaluation team during the final evaluation design phase:
Desk Review: The evaluation team will conduct a desk review of available primary and secondary 
documents including background documents, PAJE-NIETA work plans, performance monitoring plans and 
reports, research, evaluation and other assessment reports, relevant government of Mali documents, and 
third-party research and evaluation reports.
Existing performance monitoring data: The evaluation team will compile and integrate, as feasible, and 
appropriate, existing performance monitoring data, into the analysis of the survey described above. 
In-depth, semi-structured interviews: USAID expects the evaluation team to conduct key informant 
interviews with PAJE-NIETA stakeholders, including USAID/Mali staff with direct knowledge of the 
project, PAJE-NIETA implementing partner staff, government representatives, representatives of youth
association and youth volunteers, savings groups, youth employers, as well as a non-representative sample 
of youth beneficiaries that can be located or others who live in the communities that were targeted by 
the project and are knowledgeable about the project. Interviews with other donors funding similar 
activities may also be considered. 
Focus group discussions: The evaluation team will also conduct focus group discussions with 
representatives of structures/entities that were/participate in the project or other youth activities 
including youth associations, youth volunteers, SILCs, community members, government 
agencies/departments, and other relevant stakeholders. The evaluation team should ensure diversity in 
the age, gender, and geographic location of focus group discussion participants.
35
Observations: The evaluation team will integrate on-site observations of youth enterprises in KIIs and 
FGDs. Videos and photographs of observable facilities and structures will be taken and integrated in the 
final evaluation report. 
Sampling Considerations
Given the limitations of any ex-post evaluation, it is anticipated that a non-probability sampling (purposeful, 
convenience, snowball) will be used to ensure the right stakeholders and youth beneficiaries are selected 
to participate in key informant interviews and focus discussion discussions. The evaluation team will 
propose the best sampling strategy for qualitative data collection, taking into consideration the Mali 
operating environment and within the limitations inherent to any ex-post evaluation and qualitative data 
collection method. Despite these limitations, the sampling strategy should be as rigorous as possible to 
minimize bias and ensure the credibility of the findings and usefulness of resulting recommendations. The 
evaluation team is particularly encouraged to include in the sample any youth beneficiaries that can be 
located and/or youth beneficiaries and other project stakeholders that are highly knowledgeable about 
the project and its approaches as well as youth development issues in Mali. 
The evaluation team will consult with AFR and USAID/Mali before finalizing the sample. This will allow 
USAID and former PAJE-NIETA staff to assist in identifying additional or alternative project sites and 
stakeholders to include in data collection. 
2) Data Analysis Methods
In its evaluation design proposal, the evaluation team responding to this SOW will propose specific data 
analysis methods on a question-by-question basis, including the appropriate mix of methods necessary to 
respond to the evaluation questions. The data analysis plan should show how KIIs and FGDs will be 
transcribed, analyzed, and triangulated to draw key findings and conclusions. The plan should be 
comprehensive enough to provide details for data collection and analysis for each and every question. The 
evaluation team should also indicate any software that will be used to perform data analysis. 
3) Gender Considerations
Evaluation design, methodology, data collection, analysis and report should disaggregate the data by gender 
and capture the situations, experiences, and outcomes of both male and female youth who participated in 
and benefited from PAJE-NIETA activities. The evaluation team should consider methods that can help 
identify both positive and negative unintended consequences for men and women. Evaluation data 
collection instruments and protocols should reflect an understanding of gender roles and constraints in a 
particular cultural context, as well as reflect local contexts and norms concerning the conditions under 
which men and women live. 
4) Evaluation Strengths and Limitations
The evaluation team must disclose any strengths and limitations to the evaluation and how they plan to 
mitigate them. Known limitations that are inherent to any ex-post evaluations that occur 5 or more years 
after project closure include the inability to locate project beneficiaries and key stakeholders who are 
knowledgeable about the project, the difficulty in finding key project documentation, inferior quality of the 
data that was collected by the project and recall biases resulting from respondents’ poor recollection of 
past events. Attribution of outcomes that materialized after project closure may also pose challenges. The 
evaluation team will disclose these and other limitations and include approaches for mitigating these 
challenges in order to ensure the quality of the evaluation findings and resulting recommendations. 
36
5) Summary Evaluation Design Matrix
The evaluation team responding to this SOW should include in its proposal an evaluation design matrix 
that summarizes the data sources, collection, and analysis plan for each evaluation question. An illustrative 
sketch of the matrix is included below.
Evaluation 
Questions
Data Source(s) Data Collection 
Methodology
Data Analysis 
Methodology
PAJE-NIETA 
performance 
indicators data or 
other data/evidence 
that is relevant to the 
evaluation question 
VI. Evaluation Deliverables and Other Reporting Requirements
The evaluation team will be responsible for the following deliverables (indicated with an asterisk) and 
other reporting tasks and will provide a final list of proposed deliverables and due dates in its evaluation 
design proposal for USAID’s approval. All documents and reports will be provided electronically to 
USAID. All qualitative and quantitative data will be provided in electronic format to USAID in a format 
consistent with ADS 579 requirements. 
Prior to the submission of the final evaluation design proposal, the evaluation team will discuss with USAID 
whether its preliminary dissemination plan for this evaluation indicates other deliverables that should be 
prepared. Such additions as agreed with USAID will then be included in the final evaluation design proposal.
An illustrative timeline is included in the description of each of the evaluation deliverables. USDA and its 
partners should provide a detailed timeline with their proposal for USAID review.
a) Evaluation Kick-Off Meeting (Virtual)
As soon as the Evaluation team is in place, ASSESS will convene a kick-off meeting with the team, 
USAID/West Africa REGO ASSESS COR, relevant Africa Bureau and USAID/Mali staff, and USDA staff 
managing the ASSESS PASA mechanism. The purpose of this meeting is to clarify evaluation expectations 
and deliverables and make sure everyone is on the same page on key aspects of the evaluation SOW. The 
kick-off helps the team to propose an evaluation design that best responds to the evaluation requirements 
included in the SOW.
b) Draft Workplan/Evaluation Design Proposal** (Virtual)
Fifteen days following receipt of the final evaluation SOW, the evaluation team will deliver a draft workplan 
including an evaluation design proposal as a centerpiece. The evaluation design describes the conceptual 
framework for the evaluation and the justification for selecting that approach. The design proposal must 
at least contain the following: 
37
● Discussion of the overall approach of the evaluation, highlighting the conceptual model(s) adopted 
by evaluation question and demonstrating a clear understanding of the project’s theory of change 
and particularly the Out of School Learning Model and Implementation Approach. Discussion of 
the data collection and data analysis methods that will be used to answer each evaluation question, 
and the limitations for each method. 
● Discussion of how gender analysis will be integrated into the evaluation design.
● Detail key data sources that will be selected to inform the answer to each evaluation question. 
● Discussion of the sampling approach, including area and population to be represented, rationale 
for selection, and limitations of the sample.
● Discussion of risks and limitations that may undermine the reliability and validity of the evaluation 
results, and the proposed mitigation strategies for each.
● Summarized evaluation methodology in a matrix that contains for each evaluation question: 
measure(s) or indicator(s), data collection method(s), data source, sampling approach, and data 
analysis method(s).
● Timeline showing the key evaluation phases (e.g., data collection, data analysis, and reporting) and 
specific deliverables and milestones.
● Responsibilities and qualifications of each evaluation team member 
● Discussion of USAID staff participation in each evaluation phase and their anticipated roles, 
responsibilities, and reporting requirements. 
● Discussion of logistical considerations for conducting the evaluation, including specific assistance 
that will be required from USAID, such as providing arrangements for key contacts within the 
mission or government.
● Detailed estimated budget.
Upon receipt of USAID/Mali and AFR feedback on the draft workplan/evaluation design, the evaluation 
team will make the necessary revisions and convene a meeting with USAID/Mali and AFR to present the 
revised design document and receive any final feedback from USAID/Mali and AFR.
c) Final Workplan/Evaluation Design ** (Virtual)
Within 5 days of receipt of USAID feedback on the revised draft workplan/evaluation design proposal, the 
evaluation team will deliver a final workplan/evaluation design proposal. The final workplan/evaluation 
design proposal will include all the elements listed in the draft evaluation design proposal, the final data 
collection instruments, as well as the final workplan details such as key evaluation tasks, timeline, and data 
collection instruments. 
In-Briefing with USAID/Mali (in-person) and final preparations for field work
Upon arrival in Mali, the evaluation team will meet with USAID/Mali for introductions and discussion of 
any final details related to field work, including meetings with key in-country stakeholders. After the in￾briefing with USAID/Mali, the evaluation team will hold a training for data collectors and pre-test and 
revise the data collection instruments. The team will also fill any gaps in the evaluation design by 
incorporating any changes or updates reflecting added information obtained on the ground and discussions 
38
with Mission and other stakeholder staff who are familiar with the project, and a quick review of additional 
data sources. 
The team will submit an updated final workplan/design that includes those changes and the final, pre-tested
data collection tools/instruments. 
d) Interim Meeting
The evaluation team will organize one virtual briefing session at mid-point of data collection to provide 
USAID/Mali updates on data collection progress and discuss potential challenges and emerging 
opportunities. 
e) Outbrief/Presentation of Preliminary Findings (Virtual)
The team will make a presentation of key preliminary findings of the evaluation to USAID/Mali and 
USAID/Washington at the close of fieldwork and before the team departs Mali. The debriefing must 
include a discussion of preliminary findings, conclusions, and recommendations. The evaluation team 
leader will share the draft PowerPoint slides for USAID’s review prior to the presentation.
f) Draft Evaluation Report** (virtual)
Within 7 days of the presentation of preliminary findings, the evaluation team will submit the draft 
evaluation report to USAID. The report should separately and comprehensively address each of the 
objectives and evaluation questions listed in the Scope of Work as well the findings, conclusions, and 
recommendations which should be clearly supported by the collected and analyzed data. Findings should 
be presented graphically where feasible and appropriate, using graphs, tables, and charts.
The draft evaluation report must contain at least the following:
● An evaluation abstract (not to exceed one paragraph) that discusses the evaluation purpose, key 
questions, and key findings and recommendations.
● Executive Summary: This section should be up to five pages in length and describe the purpose, 
project background, evaluation design and methodology including the evaluation questions, and 
key findings, conclusions, and recommendations. The executive summary should accurately 
represent the report as a whole.
● Background: This section will provide a brief description of PAJE-NIETA project that highlights its 
scope, design features including the project RF and/or development hypothesis, and activities 
undertaken.
● Evaluation Design and Methodology: This section will detail the overall evaluation design and 
methodology and related research protocols undertaken in conducting the evaluation, including 
the relevant data collection and analysis methods, sampling approach, and related challenges or 
limitations encountered during the evaluation and mitigation approaches employed. 
● Findings: This section will present findings collected from the evaluation relevant to each 
evaluation question. The evaluation findings must be presented as analyzed facts, evidence, and 
data and not be based on hearsay. The findings must be specific, concise, and supported by 
quantitative and/or qualitative evidence analyzed through scientifically plausible methodologies. 
● Conclusions: The evaluation report will present evaluation conclusions that are interpretations 
and judgments based on the findings described and must logically follow from the gathered data 
39
and findings and be explicitly justified. If necessary, the evaluation team will state its assumptions, 
judgments, and value premises in presenting a conclusion so that readers can better understand 
and assess them. 
● Recommendations: This section will concisely and clearly present recommendations that are 
drawn from specific findings and conclusions provided in the report. The recommendations must 
be stated in an action-oriented fashion and be practical, specific, and with defined target 
audience(s).
The draft report should not exceed 30 pages in length (not including appendices, lists of contacts). Annexes 
should include the Scope of Work, description of the methodology used, lists of individuals and 
organizations consulted, data collection instruments (i.e., questionnaires and discussion guides), a 
bibliography of documents reviewed, and a table mapping all Findings/Conclusions/Recommendations. 
g) Final Evaluation Report** (Virtual)
Following receipt of all USAID comments on the draft evaluation report, the evaluation team will have 5 
days to prepare and submit a definitive version that incorporates and responds to USAID feedback. A 
translated French version of the report should also be submitted to USAID along with the English version. 
The final evaluation report should contain the same sections as noted above for the draft evaluation report 
and should also include: 
● References: This section should include a list of all documents reviewed, including background 
documentation. 
● Annexes: All the annexes listed in the draft report description above will be included in the final 
evaluation report. 
The evaluation report should be formatted in accordance with USAID’s general branding guidelines and 
meet the requirements described in ADS 201mah, 
USAID Evaluation Report Requirements, the How to Note on Preparing an Evaluation Report ( 
https://usaidlearninglab.org/sites/default/files/resource/files/how￾to_note_preparing_evaluation_reports.pdfand) and Criteria to Ensure the Quality of an Evaluation Report 
(ADS 201maa Criteria to Ensure the Quality of the Evaluation Report A Mandatory Reference for ADS 
Chapter 201 (usaid.gov). All members of the evaluation team should be provided with USAID’s mandatory 
statement of the evaluation standards they are expected to meet, shown in the text box below.
Finally, the evaluation team will upload a copy of the final evaluation report both the English and French 
version, to USAID’s Development Experience Clearinghouse (DEC) within 90 days of COR approval to 
post it on the DEC. 
h) Final Presentation** (Virtual)
The final report is to be accompanied by a PowerPoint presentation that aims to debrief select 
USAID/Washington stakeholders on the findings, conclusions, and recommendations from the ex-post 
evaluation. A draft of the final deck should be submitted to USAID/Washington prior to finalization and 
the virtual presentation.
VI. Evaluation Team Composition and Qualifications 
The evaluation team will consist of the following 4 professionals: A Senior Youth Workforce Development 
Specialist who will function as the Evaluation Team Leader; a Senior Education Specialist, an Agro-
40
Enterprise Development Specialist, and Monitoring and Evaluation Specialist/Logistics Coordinator. The 
following are the desired profiles of the team:
1) Evaluation Team Leader/Senior Youth Workforce Development Specialist 
(International) 
The team leader will be primarily responsible for the quality of the evaluation design and its execution, 
including timely production of quality deliverables associated with this evaluation. He/she will also be 
responsible for managing the team and consulting with USAID/Washington and USAID/Mali. 
● Must have a minimum of a master’s degree in Education or an applicable social science field. 
● Must have extensive knowledge of and 8-10 years of experience in youth workforce development, 
youth entrepreneurship/vocational training, and income generation activities for youth. 
Experience in the agriculture and finance sectors would be a plus. 
● Knowledge of/familiarity with youth employment, and youth enterprise development in Mali or 
similar African contexts
● Exceptional knowledge of evaluation theories and methods. Sound, proven knowledge of USAID 
Evaluation Policy and ADS 201 evaluation quality standards
● At least 7-year experience designing and conducting evaluations of education in Africa and 
particularly those focusing on youth workforce development 
● History of successful oversight, as team leader, of evaluations of complex international technical 
assistance projects
● Demonstrated experience managing multinational teams and producing high-quality and timely 
reports for USAID or similar audiences
● Proven experience leading the drafting/finalization of evaluation reports (report samples will be 
required)
● Extensive knowledge and skills in designing qualitative and quantitative survey research 
instruments and methodologies; excellent data analysis skills, including relevant data analysis 
software
● Strong knowledge of experience in positive youth development and gender integration and 
analysis 
● Excellent analytical and report writing skills
● Fluent in English and French
2) Sr. Evaluation Specialist/Sr. Education Specialist (International)
The Evaluation Specialist/Sr. Education specialist will work in close coordination with the Team Leader 
and will be actively engaged in efforts to oversee and ensure the quality of data collection activities, ensure 
that data codebooks are clearly written, and that all data collected can be properly transferred to USAID. 
He/she will have the following qualifications: 
● Must have a minimum of a master’s degree in Education or a related social science field
● Experience in out of school youth education, capacity building, and vocational training for youth
41
● Knowledge of/familiarity with youth employment, and youth enterprise development in Mali or 
similar African contexts
● Strong knowledge and application of the positive youth development principles 
● Gender knowledge and experience highly desirable 
● Minimum of 7-year experience conducting evaluations of donor-funded out of school youth 
education or youth workforce development activities 
● Excellent quantitative and qualitative research skills and demonstrated ability to undertake content 
analysis and write a full or components of an evaluation report
● Fluent in both English and French
3) Agro-Enterprise Development Specialist (Local) 
The local agro-enterprise development specialist will work under the leadership and supervision of the 
evaluation team leader who also serves as Youth Workforce Development Specialist. He/she will 
contribute to the evaluation design and data collection and analysis, particularly bringing his/her sectoral 
expertise and local knowledge on aspects related to agro-enterprise development, the main source of 
income generation and youth employment in African rural communities. He/she will have the following 
profile: 
● Minimum of bachelor’s degree in Agriculture, Economics, or Business Administration; Master’s 
degree preferred 
● Strong knowledge of and experience designing, managing, and/or conducting research or 
evaluations of agro-enterprise, livelihood, or other income generation activities
● Experience in market linkages and savings and lending schemes in the Mali context 
● Well versed in Mali context and culture; considerable experience collaborating with communities 
and associations in the geographic areas covered by the ex-post evaluation 
● Must have detail knowledge of Mali out of school youth issues and key stakeholders involved in 
addressing these challenges
● At least 5 years’ experience consulting for International Development agencies on Mali youth 
enterprise activities and developing research products based on quantitative and qualitative data 
gathering 
● Fluent in French and English
4) Monitoring and Evaluation Specialist/Research Coordinator (Local)
The local M&E Specialist will support the evaluation team leader and Sr. Education Specialist in all aspects 
of data collection and analysis. He/she will be primarily responsible for reviewing data collection tools and 
ensuring data collection follows standard M&E protocols. He/she will also play a key role in sample design, 
instrument design, and data coding and statistical analysis. In addition, he/she will assist with all data 
collection logistics. He/she will have the following qualifications:
● Bachelor’s degree in the field of Statistics, Sociology, Economics, and/or other relevant fields. 
● Minimum of 7 years of experience in monitoring and evaluation
42
● Strong understanding of data collection and analysis methodologies applied to program evaluation, 
including surveys, key informant interviews, and focus-group discussions
● Substantial experience in designing participating in/or contributing to evaluations of international 
development programs
● Must have good knowledge of USAID programming policies and practices.
● Proficiency in quantitative and/or qualitative analysis software such as SPSS, Stata, In-Vivo.
● Strong experience managing logistics for research and evaluation activities 
VII. Evaluation Management and Logistics
The evaluation will be primarily managed by the Africa Bureau which is issuing this Statement of Work. 
Africa Bureau Sr. M&E Advisor will serve as the COR of the evaluation while USAID/Mali will appoint an 
activity manager who will manage the evaluation team on the ground in close consultation with the Africa 
Bureau COR. USAID/Mali will also provide feedback on the evaluation SOW and key deliverables. USDA 
PASA and ASSESS managers will manage the review and payment of consultant invoices and be responsible 
for submitting all deliverables and other required reports, including financial reports to the Africa Bureau 
for review and approval.
The ASSESS team will be responsible for all logistics, including coordinating all travel throughout Mali, 
lodging, printing, office space, equipment, car rentals). USAID/Mali will provide support to set up initial 
meetings with key stakeholders, including any local government or private sector partners. The evaluation 
team leader will be responsible for managing the team and ensuring the quality and timeliness of the 
deliverables. ASSESS will also be responsible for liaising and leading communications with the USAID/Mali 
activity manager for the ex-post evaluation while in the field. 
43
ANNEX B: EVALUATION DESIGN MATRIX
Evaluation Questions Data Sources Data Collection 
Methods
Sampling or Selection 
Plan
Data Analysis 
Methods
To what extent and in what ways PAJE-NIETA sustainability strategy contributed to maintaining jobs and enterprises for youth and still supports the creation of jobs and 
youth enterprises in Mali?
In what ways do local and national structures, 
systems, networks, and linkages established or 
strengthened by PAJE NIETA continue to support 
and promote youth literacy and business skills, 
provide a favorable economic environment for 
youth (include access to credit, savings programs, 
and employment opportunities) and continue to 
function and contribute to out-of-school youth civic 
engagement?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
Have PAJE-NIETA support structures, tools, 
training materials and curricula been replicated?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
What aspect of PAJE-NIETA sustainability strategy 
worked or did not work well and why? (Work well 
here means it contributed to youth maintaining jobs 
and enterprises for youth and still supports the 
creation of jobs and youth enterprises in Mali)?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
What are the factors behind the effectiveness or 
sustainability of PAJE NIETA intervention? 
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Pattern/Content 
analysis
44
Evaluation Questions Data Sources Data Collection 
Methods
Sampling or Selection 
Plan
Data Analysis 
Methods
Purposive sampling for 
qualitative in-depth 
interviews. 
To what extent and in what ways were PAJE-NIETA basic education, work readiness, financial and business skills programs effective in empowering youth to create their 
own enterprises and become more civically engaged in their communities? To what extent are these programs replicable in similar activities? 
Which elements of the Mali Out of School Holistic 
and Integrated Learning Model were most effective 
in strengthening the capacity of youth to create and 
overtime maintain their businesses and civic 
engagement?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
Which elements of the Mali Out of School Holistic 
and Integrated Learning Model are likely to be 
replicated in similar activities?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
What were some of the lessons learned and best 
practices for implementing out of school youth 
programs that can be broadly shared? 
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
What is the best approach for sustaining out-of￾school youth programs?
Project documents
USAID Staff
PAJE-NIETA Partners
Project beneficiaries
Desk review 
KIIs
FGDs
Documents received 
from USAID/AFR and 
USAID/Mali. 
Online search.
Purposive sampling for 
qualitative in-depth 
interviews. 
Pattern/Content 
analysis
45
ANNEX C: SELECTED VILLAGES BY REGION, COHORT, GENDER, AND ROAD DISTANCES ON-OFF TARMAC
Region Cercle Commune Village Cohort Populatio
n 
(RGPH)
PAJE￾NIETA 
Women
PAJE￾NIETA 
Men
PAJE 
NIETA 
Total 
Road Distance on-off 
tarmac
1. Kayes Kita Tambaga Katakoto Cohort 3 866 72 9 81 3.5 km
2. Kayes Kita Kita Ouest Horongo Cohort 3 1,186 49 12 61 0 km
3. Kayes Kita Kita Nord Baliani Cohort 1 632 24 24 48 10 km from Kita North
4. Koulikoro Kati Baguineda_camp Mofa Cohort 1 1,829 27 26 53 0 km
5. Koulikoro Kati Siby Kalassa Cohort 1 820 38 11 49 0 km
6. Sikasso Bougouni Bougouni Sogola Cohort 1 1,353 13 23 36 0 km
7. Sikasso Sikasso Diomaténé M'pegnesso Cohort 2 1,061 68 19 87 20 km
8. Sikasso Bougouni Koumantou Mena Cohort 3 2,192 48 11 59 15 km
46
ANNEX D: REFERENCES
1. Cekan, Jindra and Laurie Zivetz. “Sustained and Emerging Impacts Evaluation- Better Evaluation 
Theme” https://www.betterevaluation.org/en/themes/SEIE, accessed October 1, 2022.
2. Diallo, H. (2017). “Professional integration of young people who do not attend school or drop out early 
in Mali: the experience of the Support Project for Young Entrepreneurs (PAJE-Nièta).” A Dissertation 
produced as part of the Research-Action Teaching Unit submitted to the Programme of 
Engineering of Training and Jobs Systems. University of Toulouse.
3. Education Development Center, Inc. Sept 30, 2020. “ERSA; Education Recovery Support Activity 
Final Report 7/22/2015-7/31/2020” USAID Boston, MA: USA 
4. Education Development Center, Inc. (undated, except for October 1, 2010-November 30, 2015) 
“USAID Mali Out of School Youth Project (Projet d’Appui aux Jeunes Entrepreneurs / PAJE-Nièta) 
Final Report. Boston, MA: USA 
5. Education Development Center, Inc. October 22, 2013. “FY2014 Annual Report. Annex 
1Performance Monitoring Plan Indicators by Cohort and Entrepreneurship Curriculum” USAID 
Mali Out of School Youth Project (Projet d’Appui aux Jeunes Entrepreneurs / PAJE-Nièta). Boston, 
MA: USA 
6. Education Development Center Inc. EQUIP3 Project. April 30, 2010/ Revised May 18, 2010. “Mali 
Youth Assessment Report” Boston, MA: USA 
7. Education Development Center, Inc. October 20, 2012. “FY12: Year Two Annual Report.” USAID 
Mali Out of School Youth Project (Projet d’Appui aux Jeunes Entrepreneurs / PAJE-Nièta). Boston, 
MA: USA 
8. Education Development Center, Inc. October 22, 2013. “FY2013 Annual Report.” USAID Mali 
Out of School Youth Project (Projet d’Appui aux Jeunes Entrepreneurs / PAJE-Nièta). Boston, 
MA: USA 
9. Gueye, Mouhamadou, Bourma Fane and Sekou Omar Fofana for CERPIS. March 2014. “Evaluation 
mi-Parcours de Programme d’Appui aux Jeunes Entrepreneurs / PAJE-Nièta.” Bamako, Mali.
10. Jones, Abigail and Elizabeth Jordan, Water and Sanitation Specialists, USAID Center for Water 
Security, Sanitation, and Hygiene “Unpacking the Drivers of WASH Sustainability” 
https://www.globalwaters.org/resources/blogs/unpacking-drivers-wash-sustainability
11. SwissContact and Danida. September 2018. “PAFP: Programme D’Apui a la Formation 
Professionne,” brochure
12. USAID “Program Cycle Discussion Note: Ex-post Evaluations.” April 28, 2021, updated June 9, 
2022. Washington, DC USA
13. USAID “Style Guide 2021”. Washington, DC USA 
14. USAID “Gender Equality and Women’s Empowerment Policy, 
2022. https://www.usaid.gov/sites/default/files/documents/USAID_Policy_on_Gender_Equality_a
nd_Womens_Empowerment_8.19.22.pdf 
15. Rogers, Beatrice Lorge and Jennifer Coates. 2015. “Sustaining Development: A Synthesis of Results 
from a Four-Country Study of Sustainability and Exit Strategies among Development Food 
Assistance Projects. For USAID/ FANTA Project III
16. https://www.usaid.gov/sites/default/files/documents/1866/FFP_Sustainability_Exit_Strategies_Stud
y_Synthesis_Report_Dec2015_Final.pdf 
17. Strunk, William, Jr., and E.B. White. 2000. The Elements of Style. 4th ed. Boston: Allyn and Bacon. 
Its in-text citation: (Strunk and White, 2000).
47
ADDITIONAL INTERNET LINKS TO PROJECTS OR REFERENCES MENTIONED:
MALI:
1. PRODEJ and APEJ are Youth Employment Programs designed by the Malian Ministry of National 
Entrepreneurship, Employment and Vocational Training, Fonds d’Appui à la Formation 
Professionnelle et à l’Apprentissage (FAFPA https://pefop.iiep.unesco.org/fr/reseau/acteurs/fonds￾dappui-la-formation-professionnelle-et-lapprentissage ).
2. PRODEJ and APEJ are Youth Employment Programs designed by the Malian Ministry of National 
Entrepreneurship, Employment and Vocational Training, Fonds d’Appui à la Formation 
Professionnelle et à l’Apprentissage (FAFPA https://pefop.iiep.unesco.org/fr/reseau/acteurs/fonds￾dappui-la-formation-professionnelle-et-lapprentissage ), projet de Formation Professionnelle, 
Insertion et Appui à L’Entrepreneuriat des Jeunes Ruraux (FIER https://www.food￾security.net/projet/formation-professionnelle-insertion-et-appui-a-lentrepreneuriat-des-jeunes￾ruraux-fier/), IFAD/French and German aid agencies) and FAFPA of MNEEVE, Swisscontact on 
Youth Vocational Training and CRS on SILC.
3. UK: University College London, OE, UCL's Faculty of Education and Society
https://learning.closer.ac.uk/learning-modules/introduction/types-of-longitudinal￾research/prospective-vs-retrospective-studies/
48
ANNEX E: FOCUS GROUP DISCUSSION GUIDES
ANNEX E1. Focus Group Discussion Guide (Youth Associations)
Instructions: The evaluation team must use this tool to interview all identified PAJE-Nièta youth who received 
training, IGA, and SILC. Include any youth who is not on the scoping list but could remember the support they 
received.
1. Region/ 
Cercle/ 
Commune: 
2. Village:
3. Cohort: ⬜1st ⬜ 2nd ⬜
3rd 
4. Number Participants: Female: Male
Identify and Contextualize projects from 2010-now, and shocks to livelihoods
5. Draw timeline by year from Cohort, e.g., 2011-13 for 1, 2013-14 for 2, 2014-15 for 3
6. Identify PAJE-Nièta by training dates and when participants got IGA & SILC support. Identify 
keywords participants identify the project with.
7. Confirm that all participants were trained and received support from the project. Let 
participants describe the training inputs and supports received.
8. Identify shocks (e.g., coup 2012, COVID 2019-20, economic, social, weather by year) that 
disrupted their livelihoods
9. Have there been any similar projects since PAJE- Nièta in their community?
10. If yes to Question 9, what are the similar projects?
11. If yes to Question 9, What support did the similar projects gave?
12. To what extent did the Project’s livelihoods skills and inputs prevent high outmigration among 
men and women?
14. For how long the project was able to prevent outmigration among men 
and women?
Civic engagement
49
15. When did the youth association start in your 
village?
16. What projects were done by the youth associations due to PAJE- Nièta in terms of civic 
engagement?
17. What civic engagement activities are being continued, e.g., recreation, renovation and cleaning, 
reforestation, other collective work, or something else?
18. If none, why 
not?
19. Whose resources or partnerships have supported project participants to continue building the 
capacity of new youth to continue planting or caring for plants.?
Emerging Opportunities
20. Whose resources or partnerships ensure results are sustained? Who is helping the participants 
to pursue their livelihoods differently since the end of the project (e.g., help from the diaspora 
as remittances)?
50
ANNEX E2. Focus Group Discussion Guide (Livelihoods and Employment)
Instructions: Group participants by the livelihoods (e.g., agriculture, livestock fattening, poultry raising and soap￾making) and ensure both men and women are well represented. Groups should be no more than 10 
respondents, gathered in a circle, with space to draw on the ground or on flip-chart paper. Have 10 stones for 
ranking. Answer the following questions through direct questions (and ask the rest to raise their hands if they too 
are doing their livelihoods in current employment). For project skills and support, draw itemized lists for 
participants to rank through a consultative process (have one individual rank, but ask the group, on average, if 
this rank represents their experience, and if not, to re- rank as a group). 
1. Region/ 
Cercle/ 
Commune: 
2. Village:
3. Cohort: ⬜1st ⬜ 2nd ⬜ 3rd 4. Number Participants: Female: Male
Current Employment, Tracing to PAJE-NIÈTA 
5. What livelihoods are the youth employed in?
a. Primary:
b. Secondary:
6. What livelihood(s) are generating money?
7. Did PAJE- Nièta train them in the livelihood(s) that 
are generating money?
⬜ Yes ⬜ No If NO, what were they trained in?
8. If participants are no longer doing your PAJE- Nièta livelihood, Probe why?
Project Skills & Support
9. Open-ended question: What PAJE- Nièta skills and supports remembered by respondents are helping them do 
the activity (literacy & numeracy, entrepreneurship trainings and employment support (IGA/ SILC)
51
10. Ask the respondents to collectively rank the most important skills/training required for the livelihoods in 
the past when PAJE- Nièta was being implemented and now can be done adding more stones for more 
important skills or support).
Current Skills & Support
Ask anyone who is willing to respond to answer and then ask the rest to raise their hands if they agree. Ask 
individual respondents to share their experiences and opinions on whether the support and skills they received 
have been sustained.
11. Ask if Income Generating Activities (IGA) kits have been retained? (Including 
rooster)
⬜ Yes ⬜ No
If NO, why not?
12. Has the SILC group re-formed since the project closed? ⬜ Yes ⬜ No If NO, why not?
13. Is SILC group working now? ⬜ Yes ⬜ No If NO, why not?
14. Are you still a member of the SILC group and are using it for your
livelihood? 
⬜ Yes ⬜ No
If YES, how is it helping you?
15. Have you received any training after PAJE- Nièta ended? ⬜ Yes ⬜ No
If YES, what?
16. Probe in open discussion (or KII if the topic of money is sensitive) if the project’s support is still accessible 
and used? Why or why not?
Emerging
52
17. Have you changed anything in how you practice your livelihood or use the skills?
18. Have any of your partners or others who provide resources or capacity development changed (e.g., mentors)?
53
ANNEX F: KEY INFORMANT INTERVIEW GUIDES
ANNEX F1: KEY INFORMANT INTERVIEW GUIDE - PARTNERS 
1. Name:
2. Region/ 
Cercle/ 
Commune:
3. Village:
4. Your organization during PAJE-Nièta implementation (2010-2015)
5. Position then: 6. Position now:
7. Your organization now 
(2022)
8. How were you involved in PAJE-Nièta during implementation in skills training, civic engagement, 
or employment to create sustainable livelihoods for unemployed youth?
9. How successful do you think PAJE-Nièta was in employing youth and fostering long-term 
engagement with their communities and/or national partners?
54
10
. 
How well do you think PAJE-Nièta was planned for its results sustainability?
Specifically,
Resources:
11
. 
Did the youth participating in the project continue to have access to training and networking 
resources after the project ended? Yes ___ or no ____,
a) If yes, with what training and networking 
resources? 
b) When was this done?
12
. 
How well are these resources meeting actual needs 
of the youth now?
13
. 
Which entity funds young people with regard to creating a favorable economic environment? 
(Ministries, foreign donors, private sector partners, community members, youth associations, 
individuals).
a. Access to credit?
b. Savings programs?
c. Employment Opportunities?
14
. 
Were in-kind and cash support for youth sustained through other means since closure of the 
project?
⬜ Yes ⬜ No a) If YES, by 
who?
55
Partnership and Local Ownership
15
. 
Have new partnerships or projects arisen to support new unemployed 
youth? 
⬜ Yes ⬜ No
If YES, who are they?
16
. 
Has any entity replicated elements of PAJE-NIETA for newly unemployed youth? (Entity 
refers to ministries, foreign donors, private sector partners, community members, 
youth associations, individuals. PAJE-NIETA elements include basic education and 
entrepreneurship, young volunteers, youth associations, technical training, IGA, and
SILC starter kit)
17
. 
Has any organization retrained PAJE-Nièta youth in new livelihoods?
a. If yes, which one?
18
. 
What have the new partners done better or worse than PAJE-Nièta?
a) Better
? 
__________________________________________________
______
b) Wors
e?
19
. 
Have any local partners or communities offered new ways to support unemployed youth?
⬜ Yes ⬜ No 
a) Describe:
20
. 
How has civic engagement through youth associations been sustained since PAJE-Nièta?
56
21
. 
Who taught you about civic engagement? PAJE-Nièta or 
different entity?
Capacity Building
22
. 
How have youth capacities development continued since the end of PAJE-Nièta, particularly for 
youth literacy and business skills supports provided to unemployed youth: 
a. Youth Literacy:
b. Business Skills:
23
. 
Do you train the youth yourselves or by fellow community members informally train them?
24
. 
Do you receive support 
from any organization to 
train youth? Yes _____ 
No _____
a) If yes, which 
organization?
25
. 
Also, are there new training curricula for unemployed youth in need? ⬜ Yes ⬜ No
26
. 
If yes, how do they differ from those developed by PAJE￾Nièta?
27. If you know of project youth trainers, mentors, or ateliers, have they continued to provide 
support and job networking?
57
28. Are new capacities needed for new agro-economic livelihoods that have emerged in recent 
years?
Yes ____ No ____
a) If yes, what are these new agro-economic 
livelihoods that emerged after PAJE￾Nièta?
29
. 
Are there new private sector jobs that need trained youth? ⬜ Yes ⬜ No
30
. 
Who is providing new training?
31
. 
Have capacity-development in your organization or others you know changed and have youth 
accessed them?
Emerging Sustainability
32
. 
Has any of the PAJE-Nièta’s activities been adapted – but sustained- in diverse ways– after it 
closed? 
(PAJE-NIETA's activities include youth support, which encompasses job readiness, basic education, 
and livelihoods training covering agribusiness, vocational education, entrepreneurship, savings 
groups, ICT; and coaching which includes Technical Training in IGA, practice, internship/coaching, 
IGA starter kit, access to market information via ICT /radio).
33
. 
What innovations have they caused or generated?
Other
34
. 
Is there anyone else knowledgeable about unemployed youth workforce development we should 
speak to?
35
. 
Is there anything else you would like to tell us?
58
59
ANNEX F2. KEY INFORMANT INTERVIEWS (PROJECT BENEFICIARIES)
Instructions: Select individuals who were excellent informants or who were either incredibly positive or extremely 
negative about the project. Do an individual interview with them separately (in their home, a building or away 
from others, outside). Remember, this is a time to ask more sensitive questions.
1. Name:
2. Region/ 
Cercle/ 
Commune
: 
3. Village:
4. Cohort: ⬜1st ⬜ 2nd ⬜ 3rd 5. ⬜ Female: ⬜ Male 6. Age:
7. Status: ⬜ Single ⬜ Married ⬜ Divorced ⬜ Widowed 8. How many Children? 
__
9. Has Spouse Migrated? ⬜ Yes ⬜ No If YES, is money being sent home? ⬜ Yes ⬜ No
10. What services did you receive in PAJE- Nièta?
a. ⬜ Training
b. ⬜ IGA
c. ⬜ SILC
Current Employment
11. Remind them by referring to the timeline of Primary/ secondary livelihoods from the project and since
Current Skills & Support
12. Employment and/or self-employment since the end of project?
13. Quality of employment and income (periodic, occasional) & Income % of total from that livelihood and how 
it is changed.
60
14. Is your livelihood a result of another project? Which one?
15. Can estimate ranges by month now versus at project end how much earned:
a. ⬜ <10,000 FCFA
b. ⬜ 10-30,000
c. ⬜ 30,000-50,000
d. ⬜ 50,000+
Emerging
16. Have you used your skills in diverse ways (e.g., literacy/ numeracy)/ Have their partners (e.g., mentors helped 
them outside the project in diverse ways from how the project intended?
17. Has there been any discrimination against women due to their livelihood or discrimination by women in 
livelihoods (e.g., banned from pursuing them)?
61
ANNEX G: KEY INFORMANTS INTERVIEWED 
Name Name12 Institution Position / Title Location
1. Aude Vescovo EDC M&E Specialist Bamako, Mali
2. Adwoa Atta-Krah EDC Former DCOP PAJE-NIETA, now EDC 
USA Project Director
USA
3. Susan Ross EDC Senior Technical Advisor, PAJE-NIETA USA
4. Aminata Maiga Swisscontact Head of Technical Training, PAJE-NIETA Bamako, Mali
5. CRS Mali
6. AJA Mali
7. Amadou Ba EDC Volunteer Koulikoro, Mali
8. Fatoumata Traore Volunteer, literacy trainer Koulikoro, Mali
9. Mrs Diawara Zélé Traoré Independent Food processing trainer Kayes, Mali
10. Djénaba Cissé Independent Food processing trainer Kayes, Mali
11. Mme Kanté Ramata 
Bagayoko
Independent Food processing tainers Kayes, Mali
12. Brehima Sékou Keita Mamadou Séga Coulibaly 
Assistant, Form- Assistant trainer 
Trainer in market gardening and grain 
farming 
Kayes, Mali
12 The names of the respondents will be redacted before the report is uploaded on DEC.
62
Name Name12 Institution Position / Title Location
13. M. Makan Traoré ONG SOS KBK Trainer in market gardening and grain 
farming 
Kayes, Mali
14. Seydou Cissé KBK NGO, Agronome de 
formation
Trainer in market gardening and grain 
farming
Kayes, Mali
15. Baba Fofana Independent Soap Making trainer Kayes, Mali
16. N'faly Kanté/ Authorized veterinarian Livestock fattening and poultry training Kayes, Mali
17. Baga Nomogo Office of Tambaga Mayor 
 
Third deputy mayor, Councilor at the 
town hall in charge of education
Kayes, Mali
18. Famakan Dembélé Office of Tambaga Mayor Second Deputy Mayor Kayes, Mali
19. Denise Traoré ASAFFED NGO Trainer in literacy.
Former Volunteer with Paje Nièta 
Kayes, Mali
20. Boubacar Goita Trainer in farming Bougouni, Mali
21.
Seydou Dibate
Program Agent (Facilitator in all fields: 
L&N, IGA, SILC)
Bougouni, Mali
22. Aminata Diarra Secretary General of Local Agency Office Bougouni, Mali
23. Fousseyni Bagayoko Volunteer/ Trainer L&N Bougouni, Mali
24. Alassane Maiga Volunteer (Animator) Bougouni, Mali
25. Lamine Konate Service Chief of sport Bougouni, Mali
63
Name Name12 Institution Position / Title Location
26. Abdoulaye Coulibaly Programme Agent Bougouni, Mali
27. Moussa Camara Trainer in poultry raising Bougouni, Mali
28. Zoumana Mariko Trainer in animal fattening Bougouni, Mali
29.
Moribou Coulibaly
Trainer in market gardening and cereal 
culture 
Bougouni, Mali
30. Mamadou Doumbia Secretary General of Youth Bougouni, Mali
31. Broulaye Samake Trainer in poultry raising Bougouni, Mali
32. Mery Doumbia Volunteer Bougouni, Mali
33. Bagnan_Bamana Trainer in market gardening and farming Bougouni, Mali
34. Amadou Diakite Trainer in farming and market gardening Sikasso, Mali
35. Korotoumou Traore Volunteer Sikasso, Mali
36. Amadou Sidibe Volunteer Sikasso, Mali
37. Moussa Diamoutene Volunteer Sikasso, Mali
38. Mariam Dissa Volunteer Sikasso, Mali
39. Amadou Tapily Volunteer Sikasso, Mali
40. Abou Diabate Volunteer in Yorosso Sikasso, Mali
41. Assitan Dante Trainer in Restauration Sikasso, Mali
64
Name Name12 Institution Position / Title Location
42. Youssouf Keita Volunteer Sikasso, Mali
43. N'Guiouinsi Sissouma Trainer in farming and cereal culture Sikasso, Mali
65
ANNEX H: PROFILE OF EVALUATION TEAM
Jindra Cekan/ova, PhD is the Founder and Catalyst of Valuing Voices at Cekan Consulting LLC. An 
international political economist, she has 35 years of experience in listening to those she serves. Dr. 
Cekan/ova is an evaluator, speaker, and writer about Sustained and Emerging Impacts, including funding, 
strategic planning, designing, implementing, monitoring & evaluating to foster sustainability. Collaborating 
with many stakeholders, Dr. Cekan/ova aims to raise national voices for country-led sustainable 
development and long-term outcomes impacts long after projects close. Lessons from (post-project) 
sustained and emerging impacts and final evaluations are invaluable to shape funding, design, 
implementation, M&E, and learning and inform SDGs and Impact investment strategies. She has consulted 
to multilateral such as the Adaptation Fund (at IBRD), non-profits such as CRS, CARE, Mercy Corps, the 
Red Cross/ Red Crescent), Foundations (the Bill and Melinda Gates Foundation, Aga Khan Foundation), 
and for-profits (Price Waterhouse/ Coopers & Lybrand, Salomon Brothers). Dr. Cekan/ova has also co￾founded ImpactGuild.org, fostering (I)NGO collaborations with impact investors. She is a dual US/Czech 
citizen and has taught at The Fletcher School of Law & Diplomacy/Tufts, at Harvard and was a research 
fellow at the University of Cambridge (UK).
Sandra Basgall, PhD is an international development professional with over 20 years of experience in 
research, monitoring and evaluation, and gender and has lead MEL projects and departments in Africa and 
Asia. She has a master’s degree from New Mexico State University in agricultural extension and an ABD 
(PhD) in mass media from the University of Iowa, USA. She was Chief of Party in Iraq and Bangladesh; 
M&E Director in Nigeria, South Sudan, and Central Africa; and consulted for numerous agencies on 
programs and projects around the world. As a mixed methods researcher, she is as comfortable with 
qualitative as well as quantitative methodology and knows when to use which where to obtain good 
evaluation results. 
Mohamed Cherif Diarra, PhD is a Mali citizen and a retired education specialist who has worked in 
the education sector over 40 years in Mali, Africa, and the United States of America (USA). He is also an 
international development professional with over 20 years of experience in international development 
across the entire Africa continent. His areas of interest and expertise includes research and its 
components namely the design (qualitative and quantitative), the methodology the development of 
research tools and instruments, data collection and data analysis), monitoring and evaluation, educational 
finance, economics, policy, and leadership. He received a master’s degree in educational administration 
and a PhD degree in the Economics of Education from Louisiana State University (LSU) in Baton Rouge, 
Louisiana, USA. 
Mouhamadou Guèye, PhD has an exceptional experience in the field of population and development. 
He worked as head of research for thirteen years at CERPOD, a sub-regional research institution on 
population and development. In 2002, Dr Guèye co-founded CAREF and for about ten years, he was its 
Scientific Director. In May 2012, Dr Guèye founded CERIPS, a consultancy firm in population research. 
Since its creation, CERIPS has conducted over thirty studies for several institutions including JHU, 
Population Council, Red Cross Canada, EDC, UNICEF, UNFPA. Among these studies, there are baseline, 
midline, and final evaluations, including PAJE-NIETA midline evaluation conducted for EDC. Dr Guèye 
holds a Ph.D. in Demography from the University of Pennsylvania, USA.