1 MID-TERM EVALUATION OF FEED THE FUTURE – INDIA TRIANGULAR TRAINING (FTF-ITT) PROJECT July 2019 This publication was produced at the request of the United States Agency for International Development for USAID/India. Alter￾Modus International Corp independently prepared it. Alter-Modus is a certified Small Disadvantaged Business 8(a) management￾consulting firm that creates executives and provided solutions to business, associations, humanitarian aid organizations, and governments globally. 2 MID-TERM EVALUATION OF FEED THE FUTURE – INDIA TRIANGULAR TRAINING (FTF-ITT) PROJECT July 2019 Contracted under AID-REQM-386-19-000107 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 3 Contents EXECUTIVE SUMMARY ........................................................................................................................1 1. INTRODUCTION...............................................................................................................................6 Activity description.....................................................................................................................................6 Background .............................................................................................................................................6 Objective ..................................................................................................................................................6 Project approach.....................................................................................................................................7 Project targets and outputs....................................................................................................................7 2. EVALUATION PURPOSE AND QUESTIONS..............................................................................7 Evaluation purpose and audiences........................................................................................................7 Evaluation questions...............................................................................................................................8 3. METHODOLOGY .................................................................................................................................8 Data collection methods..........................................................................................................................9 Evaluation questions.............................................................................................................................10 Evaluation limitations...............................................................................................................................10 4. FINDINGS AND CONCLUSIONS ....................................................................................................12 Evaluation question 1: To what extent have the interventions been effective in achieving the overall objectives?.................................................................................................................................12 Evaluation question 2: To what extent has the new knowledge acquired during trainings been adopted in the target countries? ..........................................................................................................16 Evaluation question 3. To what extent has the project interventions addressed gender issues?...22 Evaluation question 4. Specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project. .................................................................24 Annex A: Evaluation Scope of Work ......................................................................................................29 Annex B: Getting to answers matrix.......................................................................................................37 Annex C: List of trainees who participated in Focus Group Discussions (Cambodia) ......................40 Annex D: List of trainees who participated in Focus Group Discussions (Kenya).............................41 Annex E: List of trainees who participated in Focus Group Discussions (Uganda)...........................42 4 LIST OF ACRONYMS BAWP Back at Work Plan DQA Data Quality Assessment EQ Evaluation Questions ICAR Indian Council of Agricultural Research FSO Food Security Office FTF Feed the Future FTF-ITT Feed the Future—India Triangular Training KSHS Kenyan Shillings MEA Ministry of External Affairs MANAGE National Institute of Agricultural Extension Management NIAM National Institute of Agricultural Marketing Management PMP Performance Management Plan UGX Ugandan Shillings USAID United States Agency for International Development 1 EXECUTIVE SUMMARY This report presents the findings, conclusions, and recommendation of the mid-term evaluation of the United States Agency for International Development/ India’s (USAID/India’s) Feed the Future-India Triangular Training Project (FTF-ITT) that is being implemented in two phases in selected countries in Asia and Africa. Evaluation purpose and questions The purpose of this mid-term evaluation is to review the project’s output, impacts, and the lessons learned against the intended objective of increasing human and institutional capacity in food and nutritional security in select African and Asian countries through training professionals from the partner countries. The expected outcome is that trained professionals will contribute to improving food security and poverty reduction in beneficiary countries through improved food and agricultural production and productivity. USAID/India will use the findings of this evaluation to inform future potential new designs that focus on the transfer of Indian innovation and successes in agricultural and dairy value chains to partner developing countries including institutional capacity building and partnerships with the private sector. The findings will also inform the programming through 2020. The findings could also benefit MANAGE and its’ local partner institutes/organizations and other USAID Missions worldwide. This evaluation responds to the following evaluation questions approved by USAID/India: 1. To what extent have the interventions been effective in achieving the overall objectives? a) How effective are the training programs in fulfilling the requirement of the participants? Are these trainings demand driven? b) How cost effective, i.e., cost per unit of output, is the project’s implementation of training programs and what factors have most affected costs? c) What is the composition of the trainings’ participants undergoing the training in terms of the selection of countries and the subject areas? d) How effective is the quality of the training programs in terms of the modules, syllabus, and capacity of the trainers, both at MANAGE and partner institutions? 2. To what extent has the new knowledge acquired during trainings been adopted in the target countries? Has there been a measurable development outcome in the agricultural practices/target communities? a) To what extent and how have the trainings contributed to improved capacity of the participants and how have the participants utilized the knowledge and skills gained in these training programs? b) Did the training by MANAGE address the human and institutional capacity gaps in food and nutritional security in the target counties? 2 c) What is the level of success achieved in adopting the new knowledge gained during trainings in the target countries and the sustainability of these initiatives? d) What are the specific enablers and barriers (both contextual and project interventions) that influenced outcomes and would also affect the sustainability of the interventions? 3. To what extent has the project interventions addressed gender issues, such as bias and underrepresentation? 4. Recommend specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project. Activity background In 2010, the United States and India announced a new agriculture partnership, the “Evergreen Revolution,” to address global food security. This effort includes trilateral cooperation in adapting technological advances and innovative solutions to address food security challenges in Africa and Asia. This cooperation envisages leveraging agriculture sector capacities of both the U.S. and India to address broader global development challenges in selected developing countries in Africa and Asia. The FTF-ITT activity envisages partnership between the United States and India to work with partner with partner countries in Asia and Africa to increase agricultural productivity and improve food security. As part of this commitment, USAID is representing the U.S. Government and the Ministry of External Affairs (MEA) of the Government of India partnered launch of the FTF-ITT program in July 2016. The National Institute of Agricultural Extension Management (MANAGE) representing the Government of India and 22 partner institutions in India which include the Indian Council of Agricultural Research (ICAR) and other institutions are implementing the project by training professionals from the partner countries and the trained professionals will contribute to poverty reduction and food and nutritional security through adoption and implementation of new technologies, ideas and knowledge gained. The project has the following objective:  To address human and institutional capacity gaps in food and nutritional security in select African and Asian countries Key Findings, Conclusions, and Recommendations Evaluation Question 1: To what extent have the interventions been effective in achieving the overall objectives? To what extent have the interventions been effective in achieving the overall objectives?  The interventions have been effective in improving the knowledge levels of training participants. Data collected by MANAGE showed that the average knowledge levels of participants in the 34 trainings improved by 50 percent as a result of the trainings. The average rating on the effectiveness of the trainings, based on data collected by the evaluation team from a sample of training participants, was 7.3 out of 10 with 10 as the best score. 3  The design of the training program is based on a desk assessment of the needs of partner countries. The assessments seem to have identified the generic constraints in the agricultural sector but not the knowledge gaps in the context of possible available opportunities for increasing agricultural production and food and nutritional security.  Available data and evidence suggest that the program has been cost effective with an average cost per participant per day at $48 for the trainings in India and $195 for the trainings in partner countries. These costs are reported to be less than the average costs for the other trainings offered by MANAGE under similar international training programs. The total cost per training in India is estimated to be the same for all the 34 trainings in India irrespective of the number of trainees and/or training location at $124,275 of which USAID’s share is about $97,200. It is not clear why the total costs are the same for all trainings.  Analysis of the composition of training participants showed that they represented a wide variety of partner countries as envisaged in the program. It should be noted that Bangladesh and Rwanda are not yet represented. The training programs also covered a variety of themes and subject areas as identified in the needs analysis.  On the quality of training programs in fulfilling their needs, the average rating by the sample of training participants was 7.6 out of 10. The average rating on the syllabus and coverage of the course modules was 7.8 out of 10 and the average rating on the capacity of trainers as perceived by the trainees was 8.4. Similarly, scores on the average rating of the training programs collected by MANAGE at the end of the training was also 9.1 out of 10. The above results seem to indicate the superior quality of the training. Evaluation question 2: Adoption of the new knowledge acquired during the trainings and measurable development outcome in the agricultural practices/target communities  All training participants prepared Back at Work Plans (BAWP) towards the end of the training. About 40.1 percent of the participants surveyed initiated implementation of the BAWPs using the knowledge and skills gained from the trainings: the highest was in Uganda (68%) followed by participants from Kenya (58%) and Cambodia (4.7%)  Approximately 83.3 percent of the respondents used the knowledge and skills gained from the training in their routine work and 84 percent of the training participants transferred the new ideas and knowledge they gained at the FTF-ITT training to their colleagues, students and other interested individuals  The training programs improved the knowledge of participants in the topics covered on the average by 50.1 percent.  The specific barriers and enablers that influenced the outcomes and sustainability include: o Finance and budget constraints; o Lack of incentives for training participants to effectively implement; o Organizational and enabling environment for adoption and implementation; and, o The transfers and movements of training participants. 4 Evaluation question 3: Project interventions on gender issues and impacts  Gender analysis showed that the program design and implementation is not biased against a particular gender and is thus gender neutral.  Gender disaggregated data on participants in the 34 trainings show that 61 percent of the trainees were female and 39 percent of females, reflecting in part the proportion of genders represented in the roles of government service in partner countries.  96 percent of the respondents indicated that the technologies covered in the training were gender neutral and were not biased towards any gender.  Among the 34 trainings completed three were specifically on entrepreneurship development among the rural women under the Kudumbasree model. The focus group discussions and interviews with the training participants in Kenya and Uganda showed that these trainings had a significant impact on the empowerment of women, entrepreneurship development, and improvement in their livelihoods.  The case studies presented in the fourth section, as reported by the participants in the focus group discussions, document the significant impacts of the training program on the beneficiaries and on improving the productivity of the agricultural sector and food security. Evaluation question 4: Opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project Below are recommendations and opportunities to enhance project effectiveness and improve future programming.  A Monitoring and Evaluation plan with a results framework and corresponding indicators and a mechanism to track the indicators and progress in implementation is needed to improve program efficiency  In addition to developing a PMP with a results framework and indicators to track performance, a mechanism needs to be in place to track the indicators, data quality assessment, and reporting progress in implementation and impacts.  Link the FTF-ITT training program with other agriculture and Food Security projects implemented by USAID and other donors so that the human and institutional capacity gained from the training and the FTP projects implemented by USAID and other donors could complement each other.  Make a provision in the budget to support the implementation of technologies and ideas gained from the training program  Increase participation of private sector and Non-Governmental Organizations (NGO) in the training program  Streamline/improve the selection of appropriate candidates by the point of contacts in the respective countries could increase the efficiency of the program.  Provide incentives for training participants, who have other regular job responsibilities and duties, to implement activities and/or programs using the knowledge they gained. 5  Consider a more targeted training needs assessment. It would be useful to get input from researchers and from subject matter specialists in the partner countries to identify possible gaps and opportunities to gain from India’s experience in the green revolution at the sectoral and sub-sectoral levels.  Organize trainings in partner countries, which is more cost effective as per our analysis, and more overall effective. The subject matter specialists from MANAGE could also get a better assessment of the constraints and opportunities in the country and orient the training accordingly.  Explore opportunities to link the training with the USAID’s regional and bilateral Missions 6 1. INTRODUCTION This report presents the findings, conclusions, and recommendations from the mid-term evaluation of the United States Agency for International Development/India’s (USAID/India’s) Feed the Future-India Triangular Training Project (FTF-ITT), that is being implemented in two phases in selected countries in Asia and Africa. USAID/India commissioned this evaluation and the Alter Modus International Corporation1 implemented this evaluation. The first section of this report provided background information about the FTF-ITT, including the results that the activity has achieved so far. The second section describes the purpose of the evaluation and presents the mid-term evaluation questions. The third section explains the methodology of this evaluation and its limitations. The fourth section presents the evaluation team’s findings and conclusions for each evaluation question. The last section presents the evaluation team’s recommendations, which is also the last Evaluation Question. Activity description Background In 2010, the United States and India announced a new agriculture partnership, the “Evergreen Revolution,” to address global food security. This effort includes trilateral cooperation in adapting technological advances and innovative solutions to address food security challenges in Africa and Asia. This cooperation envisages leveraging the agriculture sector capacities of both the U.S. and India to address broader global development challenges in selected developing countries in Africa and Asia. The rationale for this partnership is on India’s success in re￾orienting its extension system and approaches and that India’s successes s are expected to have relevance for other developing countries. Hence, India can work together with the United States and partner with countries in Asia and Africa to increase agricultural productivity and improve food security. As part of this commitment, USAID is representing the U.S. Government and the Ministry of External Affairs (MEA) of the Government of India partnered launch of the FTF-ITT program in July 2016. The National Institute of Agricultural Extension Management (MANAGE) representing the Government of India and 22 partner institutions in India which include the Indian Council of Agricultural Research (ICAR) and other institutions are implementing the project. Objective The project has the following objective:  To address human and institutional capacity gaps in food and nutritional security in select African and Asian countries The expected outcome is that the trained professionals will contribute to poverty reduction in beneficiary countries through improved food and nutritional security. 1 Alter Modus International is a certified Small Disadvantaged Business 8(a) management-consulting firm that creates executives and provided solutions to business, associations, humanitarian aid organizations and governments globally. 7 Project approach The FTF-ITT aims at building the capacity of public and private functionaries in agriculture and allied sectors in partner countries in Asia and Africa in emerging areas of agriculture and allied sectors by providing training to government executives, nongovernmental organizations, and advocacy groups in the agricultural sector. The project is implemented in two phases. In the first phase, MANAGE partnered with the Governments of Kenya, Liberia, and Malawi to implement trainings for the 219 public sector, private sector, and nongovernmental professionals. The trainings in the first phase were organized at the National Institute of Agricultural Extension Management (MANAGE) in Hyderabad and the National Institute of Agricultural Marketing Management (NIAM) in Jaipur. Following the initial series of trainings in the first phase, both participating and non-participating countries in Africa and Asia requested for the USAID and MEA to extend the duration and expand the geographical scope of the triangular training project. As a result, in the second phase the program was extended to 20 partner countries, 11 from Africa, and 9 from Asia. The project approach is to implement trainings in India and in target countries. Each course module will focus on themes or subsectors in which Indian institutions are reputed or have demonstrated comparative advantage to offer such trainings and ensure that it effectively responds to the target countries’ capacity gaps. Specific sub-sectors and themes are to be chosen based on an initial need assessment in target countries. The partner countries include Afghanistan, Cambodia, Lao-PDR, Myanmar, Mongolia and Vietnam in Asia and Botswana, Democratic Republic of Congo, Ghana, Kenya, Lao-PDR, Liberia, Malawi, Myanmar, Mongolia, Mozambique, Rwanda, Sudan, Tanzania, Uganda in Africa. Project targets and outputs The project target is to train 1400 executives from the partner countries over the period of four years from October 2016 to March 2020 through 44 training programs, As of March 2019, a total 887 executives from 21 countries (including India) were trained in 34 different training programs. Of the 34 trainings, 32 trainings were done in India at MANAGE and its partner institutions and one each in Afghanistan and Uganda. The project achieved 63 percent of the target in terms of number of executives trained and 77 percent of the target for the number of training programs. 2. EVALUATION PURPOSE AND QUESTIONS Evaluation purpose and audiences The purpose of this mid-term evaluation is to review the project’s output, impacts, and lessons learned against the intended objective of increasing human and institutional capacity in food and nutritional security in select African and Asian countries through training professionals from partner countries. The expected outcome is that the newly trained professionals will contribute to improving food security and poverty reduction in beneficiary countries through improved food and agricultural production and productivity. The evaluation is aimed at several audiences. The primary audience is USAID/India, particularly the Food Security Office (FSO) and Mission management. The findings are expected 8 to inform future potential new designs that focus on transfer of Indian innovations and successes agricultural and dairy value chains to partner developing countries including institutional capacity building and partnerships with the private sector. The evaluation findings will also inform the programming through 2020. The findings could also benefit MANAGE and its’ local partner institutes/organizations and other USAID Missions worldwide. Evaluation questions The FTF-ITT mid-term evaluation answers following evaluation questions (EQs), which are identical to those provided in the USAID/India’s evaluation statement of work (see Annex A). 5. To what extent have the interventions been effective in achieving the overall objectives? e) How effective are the training programs in fulfilling the requirement of the participants? Are these trainings demand driven? f) How cost effectively, i.e., cost per unit of output, is the project’s implementation of the training programs and what factors have most affected costs? g) What is the composition of the trainings’ participants undergoing the training in terms of the selection of countries and the subject areas? h) How effective is the quality of the training programs in terms of the modules, syllabus, and capacity of the trainers, both at MANAGE and partner institutions? 2. To what extent has the new knowledge acquired during the trainings been adopted in the target countries? Has there been a measurable development outcome in the agricultural practices/target communities? a) To what extent and how have the trainings contributed to improved capacity of the participants and how have the participants utilized the knowledge and skills gained in these training programs? b) Did the training by MANAGE address the human and institutional capacity gaps in food and nutritional security in the target counties? c) What is the level of success achieved in adopting the new knowledge gained during the trainings in the target countries and the sustainability of these initiatives? d) What are the specific enablers and barriers (both contextual and project interventions) that influenced outcomes and would also affect the sustainability of the interventions? 3. To what extent has the project interventions addressed gender issues, such as bias and underrepresentation? 4. Recommend specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project. 3. METHODOLOGY The evaluation team from the Alter-Modus International Corp designed and implemented the mid-term evaluation during May-June 2019. 9 Data collection methods The evaluation team followed a mixed-methods approach to answer evaluation questions, including qualitative and quantitative data collection from desk reviews, individual and focus group interviews, focus group discussions, and structured discussions and observations. Annex C provides profiles of the evaluation team members. At the start of the evaluation, the team conducted a targeted desk review of the key FTF-ITT documentation including documents and online material from MANAGE and its partner institutions. The team then carried out field-based data collection in May 2019 in India, Cambodia, Kenya, and Uganda. During the field research, the team conducted individual and group interviews, focus group discussions, and structured observations of participants in the training program and a limited number of beneficiaries2 in Cambodia, Kenya, and Uganda. To answer the evaluation questions, the team also interviewed and collected data from trainers and subject matter specialists from MANAGE and its partner institutions in India. Table 1 summarizes the data collection events that the team completed. Table 1: Data collection activities completed Method/meeting and respondent type Number of participants Kick-off meeting, data collection and discussion with USAID/India 6 Meeting and discussion with DPA, MEA 3 Visits to MANAGE and discussion with the project management unit Structured discussion and interview with project management unit 4 Structured discussion with the core MANAGE staff and Subject Matter Specialists (SMS) 16 Visit to the partner training institutes in Hyderabad, India Structured discussion with the staff and SMS at National Institute of Plant Health Management 8 Structured discussion with the staff and SMS at Indian Institute of Rice Research 7 Structured discussion with the staff and SMS at Indian Institute of Millet Research 5 Structured discussion with the staff and SMS Central Research Institute for dryland Agriculture 6 Visit to the partner training institutes in Thiruvanamthampuram, India Structured discussion with the staff and SMS at Kudumbasree 5 Structured discussion with the Directorate of Kudumbasree 4 Visit to Ministry of Agriculture, Government of Cambodia Focus group discussion and structured interviews with the training participants from Cambodia 26 Visit to the Ministry of Agriculture, Government of Kenya Focus group discussion and structured interviews with the training participants from Kenya 18 2 Since the team was in the field for one day in each beneficiary country, detailed observations and data collections from the beneficiaries were not possible. 10 Visit to the Ministry of Agriculture, Government of Uganda Focus group discussion and structured interviews with training participants from Uganda 19 Evaluation questions Evaluation question 1 (Effectiveness of the program in achieving overall objectives) In order to answer this question, the team collected data on the effectiveness of the program from the sample of trainees in the three countries. The team also collected data generated by MANAGE on knowledge levels of trainees before and after the trainings. To test whether the trainings are demand driven the team reviewed the analyses on the training needs of partner countries done by the MANAGE in relation to the trainings organized. In order to analyze the cost-effectiveness, the average cost per trainee was computed and compared with costs for similar training programs to assess the efficiency. The sample of trainees was also asked to rate the quality of the training based on their perceptions on the syllabus and materials covered in relation to their needs and the capacity of the trainers. Evaluation question 2 (Adoption of knowledge from the trainings and development outcomes in target countries) In order to assess the utilization of knowledge and skills from the training, data were collected from the sample of trainees in Cambodia, Kenya, and Uganda on the implementation of back-at￾work plans. In addition to back at work plans, the sample of trainees were also asked to report on other activities initiated or being implemented based on knowledge and skills gained from the training program. The sample of trainees were also asked to report on the transfer of knowledge gained from the trainings to their colleagues and other interested individuals and groups. The team engaged the sample of trainees in focus group discussions on enablers and barriers, factors affecting the sustainability of the results from the program and are presented under the findings. Evaluation question 3 (Project interventions and gender issues) The gender analysis of the project interventions includes the gender composition of the trainees, analysis of gender neutrality of the knowledge, and skills presented at the trainings and analysis of gender bias, if any, in implementation of knowledge gained from the training. Data collected from the desk reviews and from the trainees were used for the identify gender issues, if any, and gender neutrality. Evaluation question 4 (Recommendations to enhance effectiveness and sustainability) Data and observations by the evaluation team and inputs from focus group discussions and key information interviews of different project participants were used to present recommendations to improve the design of the similar programs in the future and to increase the efficiency and sustainability of the program. Evaluation limitations Several factors constrained the evaluation team’s ability to collect data or produce findings to answer the evaluation questions. We outline below the main challenges and consequent limitations. 11 Lack of a monitoring and evaluation plan. The FTF-ITT activity does not have a monitoring and evaluation plan, a results framework or an associated indicator to track the outcomes and impacts. The only systematic data available are the details/number of trainees and the data on the knowledge levels of trainees before and after the training collected by MANAGE. With limited availability of performance data including baseline data, the team could not clearly answer the evaluation questions on impacts. Challenges in locating beneficiaries and impacts. The knowledge and skills gained from a training and extension program get dispersed to a wide variety of beneficiaries. The beneficiaries are dispersed across geographical locations. The beneficiaries include both direct and indirect beneficiaries involved in agricultural and related activities across different communication channels. In addition, the trainees also transmitted the knowledge gained to their colleagues and subordinates. Further, an improvement in knowledge levels without adoption may not result in real-world impacts. With such wide groups of beneficiaries dispersed across geographical locations, it was difficult for the evaluation team to locate the beneficiaries and quantify the impacts. The evaluation team having spent only one day in each country could not adequately reach the final beneficiaries and assess impacts of the program on incomes, food security, and livelihoods. Challenges in assessing impacts on human and institutional capacity development. As documented in the evaluation literature, this evaluation also suffers from the difficulties in the evaluation of impacts of human and institutional capacity development. While the trainees improved their knowledge levels which increased human and institutional capacity in the agricultural extension system, there are no direct measures of impacts of improvement in institutional and human capacity. Further, there are no baseline measures and indicators of institutional and human resource capacity and gaps in capacity in the participating countries. With no baseline data and capacity gaps the evaluation team could not assess the impacts, if any, of improvements in human and institutional capacity at the sectoral level. Cognitive bias of the respondents. Interview data are well known to be prone to cognitive biases on the part of the respondent and/or the interviewer. These include social desirability or acceptability bias – the tendency of individuals to provide responses that they believe will be “socially desirable” in the context or desirable from the researcher’s/sponsor’s point of view. To ensure the validity and reliability of its findings, the evaluation team worked to mitigate potential cognitive biases of the research by using systematic triangulation of interview sources and interview questions. Selection biases. The evaluation’s sampling approach depended on the availability of trainees and possible beneficiaries from whom the team collected qualitative data based on their perceptions. Those respondents who were willing to share their views, or who were identified by the contact persons in the countries/implementing partner, may not be representative of FTF-ITT training participants as they were not drawn at random. Data collected from this sample is used primarily in this report to situate the context from which the team collected and reported on the qualitative data. Logistical and timing challenges. The evaluation team spent only one day in each country to collect data from the trainees, contact persons, and beneficiaries. Within the limited time, the team was constrained in making logistical arrangements to collect data and observe the in￾country beneficiaries from the activity and thus document the impact of the program. 12 Country sample. The evaluation used data collected from trainees and beneficiaries in only three out of 20 countries. The country selection for data collection was not random and may not be a good representation, for Asia in particular. Lag between training and observable impacts: A number of the trainings were conducted in the year 2018 and it may be too early for the trainees to implement activities using the knowledge they gained from the trainings in their home countries. 4. FINDINGS AND CONCLUSIONS Evaluation question 1: To what extent have the interventions been effective in achieving the overall objectives? The overall objective of the FTF-ITT activity is to address human and institutional capacity gaps in food and nutritional security in select African and Asian countries. In order to address the above overall objective, the FTF-ITT program implemented the following activities as of March 2019:  A total of 887 executives from 21 countries (including India) were trained in modern techniques in agricultural, food technology, marketing, and entrepreneurship development  34 different training programs were conducted in India (32 in India) and in partner countries (2 in partner countries)  The training programs resulted in significant improvement in knowledge levels of trainees after the training compared to the knowledge levels before the training This section discusses, using available data and evidence, the extent to which the above activities have been effective in improving human and institutional capacity gaps in the partner countries. a. How effective are the training programs in fulfilling the requirements of the participants? Are these trainings demand driven? Effectiveness of training programs: In order to assess the effectiveness of training programs, the evaluation team collected data from the sample participants in the three countries. The participants were asked to rate the effectiveness of the training programs in fulfilling their requirements on a scale from 1 to 10-a score of 10 implies that the participants rated the training as the most effective. The average score for the sample of participants from the three countries surveyed was 7.3 out of 10.0. Among the training participants from the three countries, the average was highest for participants from Uganda at 8.3 out of 10 followed by the average for Kenya (7.7) and then the average for Cambodia (7.4). Another measure of the efficiency of the training program, available to the team, is an indicator of the knowledge levels of participants before and after the trainings collected and maintained by MANAGE and its partner institutions. A review of the test scores showed that average test scores for all participants from all the 34 training increased by 50.1 percent after the training (see Table 4 ) for the average pre and post test scores for different training programs). The above results seem to indicate that the training has generally been effective in improving the knowledge of the trainees. 13 Are the trainings demand driven? In order to design the training programs, MANAGE conducted training needs assessments for the executives in agricultural extension/development officers in partner countries. The reviews of those assessments by the evaluation team showed that the assessments were based on the desk analysis of the agricultural sector in the partner countries. The analyses covered the constraints and opportunities for development of the agricultural and allied sectors. The needs assessment identified problems of slow growth, lack of capital formation, the scope for transfer of technology for crop improvement and diversification, water and soil conservation measures, information communication technology initiatives to strengthen agricultural extension services. It may be noted that the above constraints were generic constraints to the development of the agricultural sector in developing countries. Identification of specific needs may need more detailed analyses and an expanded Scope of Work. Further, it is not clear whether the assessment covered knowledge gaps and available opportunities for increasing knowledge and capacity of executives in partner countries. Identification and assessment of knowledge gaps and opportunities in the agricultural sector may need more detailed consultations with the research and extension agents in the partner countries. Such specific inputs could be used to further improve the design and contents of the training programs. These should, however, be considered in the context of the budget and logistical constraints faced by MANAGE and its partner organizations. b. How cost effectively, i.e., cost per unit of output, has the project implemented the training programs and what factors have most affected costs? Cost effectiveness: Out of the 34 training programs under FTF-ITT, 32 trainings were conducted in India and one each in Afghanistan and Uganda. The 32 training programs in India trained 760 executives and the duration of each training was for 15 days. The average cost per participant for these trainings was $5232.60 for 15 days and the average cost per day per participant was $348.80. The above cost is comparable to costs per trainee per day for similar international training programs developed and implemented by MANAGE at around $500 per day per trainee. The two training programs organized in Afghanistan and Uganda were for ten days each and 127 executives were trained. The average cost per participant for these trainings was $1957 for 10 days and the average cost per day per participant was $195.70. The per unit costs for the two trainings outside India seem to be significantly less than the corresponding per unit costs for the trainings in India. The lower per unit costs for the trainings outside India is due to the higher number of participants in trainings, lower transportation, and boarding costs. The above results suggest the cost-effectiveness of the training program. Program budgeting and spending: The total cost per trainings in India is estimated to be $124,275 of which USAID’s share is about $97,200. The total cost seems to be the same for all the 34 trainings irrespective of the number of trainees and/or training location. The same total costs irrespective of the number of trainees and training location seem to suggest inefficiencies in budgeting and spending3 . The evaluation team did not have access to cost components for the 3 This suggests (but not verified) that MANAGE and its partners utilized and spent the full allocated budget without regard to the number of trainees, the training location, transportation costs, the boarding and lodging costs, and other logistical costs. 14 different trainings to further analyze the cost components and compare them across the different trainings. Improving efficiency in spending on the different cost components could improve overall efficiency. c. What is the composition of the trainings’ participants undergoing the training in terms of the selection of countries and the subject areas? Training participants by countries: The country-wide composition of training participants presented in Table 2 shows that partner countries were well represented in the training programs. Of the 887 training participants, 64 percent were from Africa and 36 percent were from Asia. Among the partner countries, Afghanistan accounted for 17 percent of the total participants followed by Uganda (15.8%), Kenya (12.2%), and Malawi (10.6%). d. Training participants by subject areas: Among the subject matter areas covered ICT applications in agricultural extension and entrepreneurship development among rural women accounted for 11.8 percent of the total participants. See Table 3 for the complete distribution of participants by subject matter areas. Table 3: FTF-ITT International training programs completed; October 2016-March 2019 Program title No. of trainees 1 ICT applications in agricultural extension 105 (11.8) 2 Public-private partnerships in extension management 29 (3.3) Table 2: Country-wide participation in the training program as of March 2019 Asia Africa Afghanistan 155 (17.5) Botswana 22 (2.5) Bangladesh 0 DR Congo 11 (1.2) Cambodia 35 (3.9) Ghana 56 (6.3) India 10 (1.1) Kenya 108 (12.2) Lao PDR 0 Liberia 73 (8.2) Mongolia 37 (4.2) Malawi 94 (10.6) Myanmar 64 (7.2) Mozambique 22 (2.5) Nepal 15 (1.7) Rwanda 0 Sri Lanka 5 (0.6) Sudan 24 (2.7) Vietnam 1 (0.1) Tanzania 15 (1.7) Uganda 140 (15.8) Sub-total Asia 322 (36.3) Sub-total Africa 565 (63.7) Total trained 887 (Figure in parentheses are percentages of the total trained from each country) 15 3 Emerging trends in marketing of fruits and vegetables 27 (3.0) 4 Agripreneurship development among farm women 25 (2.8) 5 Production and post-harvest technology for tuber crops 23 (2.6) 6 Farm mechanization for small farmers 23 (2.6) 7 Modern dairy technology and management 24 (2.7) 8 Convention for points of contacts of African and Asian countries 22 (2.5) 9 Modern storage technologies in agriculture 20 (2.3) 10 Plant health management technologies, biosecurity, and approaches 44 (5.0) 11 Harvest and post-harvest technologies in fisheries 22 (2.5) 12 Post-harvest technology in horticulture crops 26 (2.9) 13 Entrepreneurship development in rural women 105 (11.8) 14 Linking farmers to markets: opportunities and challenges 20 (2.3) 15 Strategies for enhancement of farmers' income dryland agriculture 22 (2.5) 16 Value addition and market linkage mechanisms in millets 20 (2.3) 17 Management of technology for soil testing based advisory services 21 (2.4) 18 Production, processing, and marketing of organic vegetables 21 (2.4) 19 Agricultural credit for sustainable livelihoods 20 (2.3) 20 Income generating enterprises in the plantation sector 21 (2.4) 21 Production and processing technologies for rice 20 (2.3) 22 Plant biosecurity and food safety 23 (2.6) 23 Management of dairy cooperatives 22 (2.5) 24 Entrepreneurship development in food processing 20 (2.3) 25 Modern poultry management 29 (3.3) 26 Value addition in spices 22 (2.5) 27 Modern storage technologies in agriculture 21 (2.4) 28 Value chain innovations in agricultural marketing 30 (3.4) 29 Modern dairy technology and management 29 (3.3) 30 Seed production, processing, and commercialization 31 (3.5) Total trainees 887 (100) (Figures in parentheses are percentages of the executives trained in each subject￾percentage of total) The above results showed that the training covered a broad mix of subject matter areas and are consistent with the training needs identified in the needs assessment. d. How effective is the quality of the training programs in terms of the modules, syllabus, and capacity of the trainers, both at MANAGE and partner institutions? 16 In order to assess the quality of training programs in terms of modules and syllabus, the evaluation team collected data on the participants’ assessment of the quality of the training programs. The sample of participants from the three countries were asked to rate the overall quality of the training programs on a scale from 1 to 10, with 10 as the highest rating. The overall average rating on the quality of the training program was 7.6 out 10.0. Among the participants from the three countries, the average was highest for participants from Uganda and Kenya both at 8.5 followed by Cambodia at 6.6. Similarly, in order to assess the coverage of modules and syllabus the sample of participants were asked to rank their perception on coverage on a scale from 1 to 10. The results showed an overall average rating of 7.8 out of 10. The participants from Uganda had the highest average rating at 9.1 followed by the average for Kenya (8.1) and then Cambodia (6.8). The sample of training participants were also asked to rate the capacity and knowledge of the trainers at MANAGE and its partner institutions. Based on ratings on a scale of 1 to 10, the average rating on the capacity and knowledge of the trainers was 8.4. The average score was the highest for participants from Uganda at 9.1 followed by the average for participants from Kenya at 8.9 and Cambodia at 7.8. The above responses responded to the quality of their syllabus and the capacities of their trainers. MANAGE and its partner institutes also asked the training participants to provide an overall rating of their training programs at the end of each training program on a scale of 1 to 10, with 10 being the best score. The data provided by MANAGE showed that the average overall rating by all participants in all the 34 training program is 9.1 out of 10. This result is consistent with the rating on the quality and coverage of the training program collected by the evaluation team. The above results indicate that the training programs were effective in terms of the quality of the modules, syllabus, and capacity of trainers at MANAGE and its partner institutions. Evaluation question 2: To what extent has the new knowledge acquired during trainings been adopted in the target countries? Has there been a measurable development outcome in the agricultural practices/target communities In order to answer this question, the team collected data from the sample of training participants on the adoption of knowledge gained from the training in their respective countries and the results are presented below. a. To what extent and how have the trainings contributed to the improved capacity of the participants and how have the participants utilized the knowledge and skills gained in these training programs? All training participants prepared Back at Work Plans (BAWP) towards the end of the training. The BAWP present a concept for a specific project/activity that each participant plans to implement as they get back to their respective countries. The BAWPs were developed based on prospective opportunities that may be available in the participant countries where the knowledge and skills gained at the training could be utilized. The evaluation team collected data on the implementation of BAWP by the sample of training participants. The results showed that 17 40.1 percent of the participants surveyed initiated implementation of the BAWPs using the knowledge and skills gained from the trainings. BAWP implementation was the highest in Uganda (68%) followed by participants from Kenya (58%). Among the three countries surveyed Cambodia had the lowest implementation rate at 4.7 percent. In addition to implementing and working on the BAWP, the participants surveyed were asked about their utilization of knowledge and skills gained from the training. Nearly 83.3 percent of the respondents reported that they used the knowledge and skills gained from the training in their routine extension work. This could include the transmission of knowledge and ideas gained from the training to their colleagues, farmers, and also students (when the trainee was affiliated with an agricultural training institute or college). It may be noted that transmission of knowledge and ideas and their impacts may not be easy to measure and document without extensive field surveys and data collection. b. Did the training by MANAGE address the human and institutional capacity gaps in food and nutritional security in the target countries? The training program by MANAGE and its partner institutions trained 887 executives from 20 countries in Africa and Asia. As of March 2019, 34 training programs were organized. The training programs covered a range of topics in agriculture and allied activities and entrepreneurship development providing new ideas and technologies for agricultural development, food security, and poverty reduction. The training programs improved the knowledge of participants in the specific topics covered (see Table 4 for a complete list of training programs and the number participants in the trained and the knowledge levels before and after the training). Table 4: FTF-ITT International training programs: improvement in knowledge levels of participants Test scores before and after the trainings Program Title No. of trainees Pre￾training Post trainin g % change ICT applications in agricultural extension 71 14.0 24.6 75.9 Public-private partnerships in extension management 29 13.3 15.6 17.4 Emerging trends in marketing of fruits and vegetables 27 Agri-preneurship development among farm women 25 15.0 19.0 26.7 Production and post-harvest technology for tuber crops 23 13.4 201.2 1396.9 Farm mechanization for small farmers 23 10.8 18.4 70.2 Modern dairy technology and management 24 15.0 17.7 17.7 Convention for Points of contacts from partner countries 22 18 Modern storage technologies in agriculture 20 9.9 15.8 59.6 Plant health management technologies and approaches 24 11.3 24.5 117.0 Harvest and post-harvest technologies in fisheries 22 7.4 18.9 156.5 Post-harvest technology in horticulture crops 26 12.9 18.7 45.2 Entrepreneurship development in rural women 56 18.9 23.0 21.7 Linking farmers to markets: opportunities and challenges 20 Strategies for enhancement of farmers' income dryland agriculture 22 8.8 18.3 108.6 Value addition and market linkage mechanisms in millets 20 15.6 19.3 23.8 Management of technology and for soil testing based advisory services 21 20.6 24.2 17.8 Production, processing, and marketing of organic vegetables 21 10.1 17.7 75.3 Agri-cultural credit for sustainable livelihoods 20 8.4 18.1 116.8 Income generating enterprises in plantation sector 21 4.8 13.2 176.5 Production and processing technologies for rice 20 5.7 15.0 163.2 Plant biosecurity and food safety 23 11.3 17.7 56.4 Management of dairy cooperatives 22 16.0 21.9 36.8 Entrepreneurship development in food processing 20 10.9 18.0 65.3 Modern poultry management 29 11.8 18.5 57.1 Entrepreneurship development among rural women 23 15.8 21.7 37.3 Value addition in spices 22 12.0 19.1 59.1 Modern storage technologies in agriculture 21 10.1 17.0 68.3 Value chain innovations in agricultural marketing 30 13.9 15.2 9.3 Entrepreneurship development among rural women 26 15.1 17.9 19.0 Plant health management, biosecurity, and quarantine 20 10.8 15.5 43.1 Modern dairy technology and management 29 13.6 16.7 23.2 Seed production, processing, and commercialization 31 13.2 16.0 20.7 ICT applications in agricultural extension 34 12.7 17.1 34.3 Overall average 887 12.34 18.52 50.1 The above results indicate that the training programs improved the knowledge of the participants on average by 50.1 percent. The training participants were exposed to possible technologies and opportunities that could be further developed for implementation in their home countries. The trainees also conveyed their knowledge and skills gained to the farmers/beneficiaries and their colleagues. In addition to the implementation of BAWP, transfer of new ideas and knowledge to colleagues, students and others interested could also increase human capacity. Data collected from the 19 sample of respondent showed that about 84 percent of the training participants transferred the new ideas and knowledge they gained at the FTF-ITT training to their colleagues, students, and other interested individuals. Improvements in the knowledge levels of the training participants, their colleagues, students, and beneficiaries imply improvements in the human capacity to address food security and agricultural development in partner countries. c. What is the level of success achieved in adopting the new knowledge gained during the trainings in the target countries and the sustainability of these initiatives? The knowledge gained from the trainings were adopted by the trainees through the implementation of BAWPs, and the use of knowledge and skills gained from the trainings in routine extension work and through the transmission of knowledge to colleagues and students. Data collected from the sample of trainees showed that 40.1 percent of the participants surveyed initiated implementation of the BAWPs. The evaluation team, having spent only one day in the partner countries, could not verify the actual stage of implementation and the impacts on the beneficiaries due to logistical and time constraints. The rate of the BAWP implementation was the highest in Uganda (68%) followed by participants from Kenya (58%). Among the three countries surveyed Cambodia had the lowest implementation rate at 4.7 percent. In addition to implementing the BAWP, the participants surveyed were asked about their utilization of knowledge and skills gained from the training. About 83.3 percent of the respondents reported that they used the knowledge and skills gained from the training in their routine extension work that could have led to the adoption of improved practices by the beneficiary farmers and entrepreneurs. Due to time and logistical constraints, 4 the evaluation team did not conduct detailed surveys of beneficiary farmers who adopted new technologies and hence could not estimate the adoption rate of new technologies and the impacts of the training program at the sectoral level. However, the evaluation team came across cases of successful activities that are being implemented by the trainees. We present below five case studies of such activities as reported by the trainees at the interviews and focus group discussions. 1. Organization of farm groups into viable income generation activities in Kenya following the Kudumbasree model. This is one of the activities implemented in Kenya following the Kudumbashree model. As part of this activity, about 1600 women are now registered as self-help groups under the Women Farmers Association of Kenya. The groups were then trained in micro-enterprise and entrepreneurship development, record keeping, marketing and finance. These self-help groups are now engaged in activities like the production of fermented porridge, climate-smart agricultural production activities, and canteen units following the café Kudumbashree model in India. The groups are now selling fermented porridge in various canteen units. These activities have increased their incomes, created jobs and improved their livelihoods thus improving women 4 The evaluation team spent only one day in each of the three countries to collect data and information. 20 empowerment. Most of the self-help groups now generate their reserve (thrift) funds from their savings which are being used to support members of the group through loans. 2. Assessment of physical parameters of soil health with farmer-friendly rapid techniques using soil testing mini lab kit Agricultural productivity in Kenya has been declining due to the degradation of soil and poor soil and land management practices. This activity is to promote soil testing and soil mapping to promote soil fertility management using Indian technology, which is cheaper and more efficient compared to the technology currently in Kenya. This activity was developed after the FTF-ITT training program to develop Mini Soil Test laboratories and the development of Leaf Color Charts to improve soil fertility management and soil fertilization practices among farmers. The advantages of the proposed technology are: lower cost of analyzing the soil characteristics, portability of the soil testing mini lab kits, opportunity to create additional employment for the youth and improvements in agricultural production and productivity through better soil health and nutrient management. In addition to the Mini Soil testing labs, this activity was introduced by the Indian technology of Leaf Color to assess the nitrogen content of the plants and thus enhance the effective use of nitrogen fertilizer management. The proposed Mini Soil testing Lab is expected to generate significant cost savings and generate results in a shorter time. The current soil testing practices in Kenya costs about Kenyan Shillings (Kshs) 1000 per sample for analysis and to generate fertilizer recommendations based on the results. It also takes about a month on the average to get the results. The proposed Indian technology, conceived and is being developed after the FTF-ITT training, is estimated to cost Kshs. 350 per sample and estimates 10 soil parameters to better characterize soil conditions and soil management. The BAWP developed is being proposed to the Ministry of Agriculture, Livestock, and Fisheries in Kenya for approval and collaboration with the Indian Institute of Soil Science in collaboration with a Kenyan Agricultural and Livestock Research Organization. Implementation of the proposed activity will improve institutional and human capacity development for soil testing based advisory and soil mapping services to the farmers in Kenya. These services could improve agricultural production and productivity that could lead to improvements in food security and resilience. 3. Linking small dairy and poultry farmers to markets to improve livelihood and entrepreneurship among women farmers in Kenya The objective of this activity is to improve the livelihood and incomes of women farmers engaged in dairy and poultry production activities by improving their entrepreneurship skills following the Kudumbasree model in India. Under this activity, a group of twenty youth was formed and were trained in procuring milk, storing, and linking them with the market. In addition to raising cows and poultry the group also procured milk from other farmers. The group also raised money to acquire a cooler to store the milk and eggs. The group was then 21 linked to a milk processor to market the milk and poultry products. This activity is expected to generate an average monthly income of Kshs., 20,000 for each member of the group, which is significantly higher than the average incomes they were earning before this activity. 4. Improve the coffee processing and marketing by small holders to gain premium pricing and higher incomes This activity was to organize small coffee growers in Uganda to improve processing and marketing so as to gain higher prices. The activity targeted 1296 coffee farmers and organized them into sixty farmer groups at the primary level and six cooperatives at the secondary level. The coffee farmers were mobilized and trained to bulk process to improve quality to the level of Fair Average Quality Coffee standards. The farmers then marketed the product as a group. This activity resulted in improvement of the quality of the product and as an added bonus. The farmers received higher premium prices for their product. The average prices received by the farmers increased by 6.7 percent from Ugandan Shillings (Ugx) 4500 to 4800 per kilogram as a result of this activity. 5. On-farm production and use of bio-fertilizers The production and use of bio-fertilizers is another activity that is being initiated in Kenya and Uganda as a result of the knowledge gained from the FTF-ITT training. At the focus group interviews the trainees reported that they are working with the Indian National Institute for Plant Health Management on on-farm production of bio-fertilizers such as Mycorrhiza, Rhizobium, Azotobacter, and Azospirilium. These products are in use by farmers at the farm level. This activity is being initiated and is expected to be adopted by farmers and result in about 25 percent savings in the use of chemical fertilizers. The cost of producing the bio-fertilizer for one acre is about Indian Rupees 200. The saving in the use of fertilizer and the resultant increase in yield could be significant compared to the cost of producing bio-fertilizer. The case studies5 presented above demonstrate examples of impacts of the knowledge and skills gained by training participants on beneficiaries in participating countries. These and other similar practices could further expand to more beneficiaries and could lead to larger adoption in the long-run which will, in turn, improve agricultural production, food security, and livelihoods of the poor. d. What are the specific enablers and barriers (both contextual and project interventions) that influenced outcomes and would also affect the sustainability of the interventions? The evaluation team identified a number of barriers and/or constraints that affected the adoption and implementation of the knowledge and skills acquired at the trainings and are outlined below: 5 These are activities reported by participants at focus group interviews in Kenya and Uganda. 22 1. Finance and budget constraint: Adoption and implementation of new ideas and knowledge require financing. The respondents indicated that project-level financing and budget provisions are not available for implementation of the BAWP or other ideas conceived from the FTF-ITT training. The absence of such financing and budget provisions constrained adoption and implementation in the target countries. 2. Lack of incentives for implementation: Most of the participants who attended the training programs were executives from the partner country governments. These executives have other regular roles and responsibilities which constrains them from designing programs and activities using the knowledge they gained from the training program. These executives do not have sufficient incentives in investing additional effort to design and implement activities using the knowledge gained from the training program. With the lack of incentives, in the form of financial, professional advancement and/or other incentives, the chances implementation and use of knowledge and skills in home countries are very low. 3. Organizational and enabling environment for adoption and implementation: Adoption of new practices and or knowledge may require an enabling environment or improvement in policies and practices at the community and or district/regional level. For instance, most of the proposed activities are to be implemented in local communities and the enabling environment in local districts/communities. This would positively affect the implementation and improve coordination with the local governments, which is needed for the success in program implementation. Such organizational and enabling environment in the form of business environments, markets, and infrastructure are constraints to the adoption of improved knowledge and skills from the training. 4. Transfers and movements of training participants: The government officials tend to change their duties and positions within the Department of Agriculture. Some of the executives, including the point of contact in the country, who attended the training tend to get transferred or have assumed other duties. As they assume new responsibilities implementation and utilization of new knowledge and skills acquired from the training will be affected or they fail to follow-up on activities they initiated or plan to initiate. Such movements and transfers from one position or responsibility to another affect the adoption and implementation and thus the sustainability of outcome. Evaluation question 3. To what extent has the project interventions addressed gender issues? In order to do a gender analysis of project interventions the evaluation team used data from MANAGE, data collected from desk reviews and also from the focus group interviews of training participants. Gender disaggregated data on participants in the 34 trainings show that out of the 887 training participants, 543 were male (61 percent) and 345 were female (39 percent). A closer analysis of the data in Table 5 shows that training in ICT applications in agricultural extension with a 23 significantly higher proportion of males6 skewed the results. Further, it may also be noted that the proportion of females in the roles of government executives from which the training participants were selected may be less than males which in turn resulted in a slightly less proportion of female executives among the training participants. These results suggest that there has not been a gender bias in selection and attendance of training participants. Table 5: Gender disaggregated data on the training participants Male Female Total 1 ICT applications in agricultural extension 63 8 71 2 Public-private partnerships in extension management 13 16 29 3 Emerging trends in marketing of fruits and vegetables 11 16 27 4 Agripreneurship development among farm women 4 21 25 5 Production and post-harvest technology for tuber crops 14 9 23 6 Farm mechanization for small farmers 18 5 23 7 Modern dairy technology and management 16 8 24 8 Convention for points of contacts of African and Asian countries 19 3 22 9 Modern storage technologies in agriculture 11 9 20 10 Plant health management technologies and approaches 12 12 24 11 Harvest and post-harvest technologies in fisheries 17 5 22 12 Post-harvest technology in horticulture crops 14 12 26 13 Entrepreneurship development in rural women 18 38 56 14 Linking farmers to markets: opportunities and challenges 10 10 20 15 Strategies for enhancement of farmers' income dryland agriculture 16 6 22 16 Value addition and market linkage mechanisms in millets 8 12 20 17 Management of technology and for soil testing based advisory services 17 4 21 18 Production, processing, and marketing of organic vegetables 11 10 21 19 Agricultural credit for sustainable livelihoods 14 6 20 20 Income generating enterprises in plantation sector 15 6 21 21 Production and processing technologies for rice 12 8 20 22 Plant biosecurity and food safety 17 6 23 23 Management of dairy cooperatives 16 6 22 6 This training was done in Afghanistan. A significant number of participants in this training were from Afghanistan. It may be noted that agricultural department and extension system in the country has a higher proportion of males than females. 24 24 Entrepreneurship development in food processing 12 8 20 25 Modern poultry management 16 13 29 26 Entrepreneurship development among rural women 2 21 23 27 Value addition in spices 9 13 22 28 Modern storage technologies in agriculture 17 4 21 29 Value chain innovations in agricultural marketing 26 4 30 30 Entrepreneurship development among rural women 6 20 26 31 Plant health management, biosecurity, and quarantine 13 7 20 32 Modern dairy technology and management 23 6 29 33 Seed production, processing, and commercialization 22 9 31 34 ICT applications in agricultural extension 30 4 34 All trainings 542 345 887 The evaluation team further collected data from the sample of training participants on gender neutrality of knowledge and technologies covered in the trainings. 96 percent of the respondents indicated that the technologies and subject matter covered in the trainings were gender neutral and not biased towards any gender. Among the 34 trainings completed, three were specifically targeted on entrepreneurship development among the rural women under the Kudumbasree model. The focus group discussions and interviews with the training participants in Kenya and Uganda showed that these trainings had significant impacts on empowerment of women, entrepreneurship development, and improvement in their livelihoods. The above results suggest that the FTF-ITT training program was gender neutral and there has not been any bias towards a particular gender. At the focus group discussions, the sample of training participants further reported that the organizers were paying careful attention to accommodate the needs of the each represented gender. Evaluation question 4. Specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project. Below are recommendations and opportunities to enhance project effectiveness and improve future programming. 1. A Monitoring and Evaluation plan with a results framework and corresponding indicators and a mechanism to track the indicators and progress in implementation is needed to improve program efficiency: The FTF-ITT program does not have a monitoring and evaluation plan to monitor and track progress in implementation: A performance management plan and a results framework with indicators to track activities, outputs and impacts will improve effectiveness in implementation and evaluation of the project impacts. The indicators currently tracked by MANAGE are not sufficient to 25 monitor and evaluate the impacts-thorough improvements in human and institutional capacity and impacts on food security- in partner countries. Although MANAGE is maintaining IT based applications like WhatsApp and Facebook groups to report progress in implementation, such voluntary reporting may not capture all the project impacts in partner countries. USAID/India, in future program development, should engage a contractor to develop the monitoring and evaluation plan with a Performance Management Plan (PMP) and indicators to be tracked. The monitoring and evaluation plan should also involve Data Quality Assessment (DQA) to ensure the consistency and quality of the data reported. 2. Link FTF-ITT training program with other agriculture and Food Security projects implemented by USAID and other donors. Adoption and implementation of new technologies and skills gained from the training in partner countries need additional resources and facilitation. Linking the training program with other FTF programs or other agricultural sector programs by other donors could facilitate better adoption of new technologies and skills received from the FTF-ITT training programs. Linking the training program and use of knowledge and institutional capacity with other FTF programs could be achieved through better coordination with the USAID field missions in the partner countries. The evaluation team, based on informal discussions with the field missions, could not find evidence of coordination with the field missions in linking training program with the FTF and/ or other agricultural and entrepreneurship development programs implemented by the bilateral and regional missions. The coordination could include, among other things, involvement in the selection of trainees from the countries for the trainings relevant to projects in their FTF and entrepreneurship development project portfolio, coordination with MANAGE and its partner institutions in the design of appropriate trainings that are relevant to their project portfolio. USAID Missions, both bilateral and regional, could also engage with other donors like the World Bank and development agencies in coordinating with MANAGE on the development of training programs that suit their portfolio, selection of trainees, and implementation and use of knowledge and skills gained from the training by linking the trained professionals with their projects and activities in the country. , 3. Provision of budget support to implement the technologies and ideas gained from the training program. Adoption and implementation of BAWP and the ideas acquired from the training require a provision of budget and finances, which usually is not available for the government executives and is a constraint. A provision of budget support could facilitate initiation of the project ideas and larger implementation. The budget could be facilitated through host government programs in the sector or by linking them with other FTF programs implemented in the country. The budget support could be in the form of initial seed funds in the case of entrepreneurship development program like the ones following the Kudumbasree model. 26 4. Increase participation of private sector, research, and training institutes and Non￾Governmental Organizations (NGO) in the training program. Most trainees who participated in the program are government executives from the partner countries. There were also a few trainees from NGOs and advocacy groups the evaluation team found that adoption rates and implementation of new technologies and ideas by trainees representing NGOs and advocacy groups, in general, are better than by government executives. Hence increasing participation of NGOs, advocacy groups and private sector could improve outcomes and impacts of the training program. The evaluation team also found that there were a few participants from universities and agricultural training institutes that attended past training programs. Interviews with those participants showed that they made additional efforts in implementing or developing proposals or new ideas for improving agricultural production, productivity, and food security in their home countries. They were instrumental in developing new initiatives and proposals based on the knowledge and ideas they were exposed to. These observations and results suggest that participation of research and training staff from research and training institutes from partner countries could facilitate their interaction with subject matter specialists in MANAGE and its partner institutes. Such close interaction could facilitate a synthesis of new ideas based on existing knowledge in the partner countries and technological advances in India that may be suited for the partner countries. It could further lead to improvements in collaboration and institutional and human resource capacity development in partner countries that will advance food security objectives. 5. Selection of government executives for the training. The selection of trainees is now coordinated by the point of contacts in the respective countries. The short list of proposed candidates is then sent to MANAGE and MANAGE approves the list. MANAGE has a very limited role in the selection of appropriate candidates for the training. The review of the qualifications and background of the candidates selected by the points of contacts shows there are opportunities for improving and streamlining the selection process and thus improve the project outcomes. Selection of more appropriate candidates by the point of contacts in the respective countries could increase the efficiency of the program. The selection of candidates for a particular training program requires careful attention to their qualifications. Attention needs to focus on their current job responsibilities, the suitability of the training theme in relation to their position, their interests and their commitment to the program’s objectives. In order to ensure the sustainability of the project output, the selection process should also consider the probability that the selected trainees will continue to work in their respective jobs in the medium to long-term to ensure the transfer of knowledge. 6. Incentives for training participants to implement/adopt knowledge gained. All government executives who attended the training program have other duties and responsibilities in their jobs they are currently holding. Hence implementation of the BAWP and ideas generated from the training are in most cases additional duties and/or 27 activities that require additional effort. The economic theory of incentives suggests that incentives motivate economic agents to work hard or to produce quality output and to invest7 . The evaluation team found that the government executives who attended the training have limited or no economic incentives to implement these additional tasks. Provisions of incentives could increase implementation of the ideas and technologies received from the training and thus improve project outcomes. The incentives could be devised in accordance with the organizational regulations and policies in partner countries. Economic incentives, though difficult to implement under the regulations and policies in partner countries, could include on the spot awards, annual performance incentives, discretionary bonuses, etc. Other possible incentives include awards and recognition, retention in jobs and positions held, time awards to implement the knowledge, and skills gained from the trainings. . 7. Training needs assessment and design of the trainings. MANAGE conducted needs assessment to find the training needs based on desk reviews and review of available literature. The findings of these reviews seem to be generic constraints to agricultural sector development in African and Asian countries. A more targeted analysis and review at the sub-sectoral level could be useful in identifying the constraints, opportunities and knowledge gaps. It may also be useful to get input from researchers and from subject matter specialists in the partner countries to identify possible gaps and opportunities to gain from India’s experience in the green revolution at the sectoral and sub-sectoral levels. As suggested earlier, it may also be useful to include the researchers and subject matter specialists into the training programs to facilitate confluence of ideas, knowledge, and opportunities. The needs analysis could also include feedback from government executives in the agricultural extension system to assess their needs and gaps. 8. Design and implementation of the training. The review of the background and interests of the training participants are very diverse. With such diverse interests and backgrounds including participants coming from different geographical regions, it may be difficult to design the trainings to address the specific needs of all participants. Selection of participants with similar backgrounds and interests and targeting course materials to address the specific needs of the participants could improve the design and delivery of the trainings. 9. Reduce theory and improve emphasis on practical applications. The evaluation team surveyed the training participants on the quality and effectiveness of the training and for suggestions for improvement. The average rating on the quality and effectiveness were 7.6 and 7.4 out of a maximum of 10. Although the ratings were high, the respondents provided the following suggestions for improvements: a. Revise training modules with more practical applications and more exposure to demonstrations in the field including field visits; 7 Laffont, Jean Jacques and David Matrimort, The Theory of Incentives: The Principle-Agent Model, Princeton University Press, 2002 28 b. Some of the course modules have an excessive emphasis on theory, which may not always be useful; and, c. Two weeks may be too short for the extent of material covered in the training as it makes the training program tight. 10. Mechanism to implement program monitoring and evaluation. In addition to developing a PMP with a results framework and indicators to track performance, a mechanism need to be in place to track the indicators, conduct data quality assessments and reporting progress in implementation and impacts. Currently, MANAGE is tracking a couple of indicators on the quality of the training and impacts of the training on knowledge levels of the participants, which are the output level indicators. The project does not have outcome level indicators. Although MANAGE maintains internet groups (WhatsApp and Facebook groups) to maintain links to trainees and to report progress in implementations, these reporting and feedback mechanisms are voluntary. Such reporting may not be complete, nor the quality of the data reported are assessed and are not sufficient to monitor the FTF-ITT impacts on food security in partner countries. Hence USAID/India should develop a mechanism to monitor and evaluate the outcomes and impacts of the program in partner countries. Future project development and implementation should encompass a monitoring and evaluation system and a mechanism to implement it, including reporting while maintaining the data quality. 29 Annex A: Evaluation Scope of Work I. Introduction This statement of work (SOW) is for a mid-term performance evaluation commissioned by the United States Agency for International Development’s India mission. II. Project background The United States and India announced a new agriculture partnership, the “Evergreen Revolution,” to address global food security. This effort includes trilateral cooperation in adapting technological advances and innovative solutions to address food security challenges in Africa and Asia. As part of this commitment, USAID and the Ministry of External Affairs (MEA) of the Government of India partnered with the Governments of Kenya, Liberia, and Malawi to launch the first phase of the triangular training project. During the first phase of these trainings, 219 public sector, private sector, and nongovernmental professionals were trained at the National Institute of Agricultural Extension Management (MANAGE) in Hyderabad and the National Institute of Agricultural Marketing Management (NIAM) in Jaipur. Following the initial series of trainings, both participating and non-participating countries in Africa and Asia requested for USAID and MEA to extend the duration and expand the geographical scope of the triangular training project. This cooperation envisages leveraging agriculture sector capacities of both the U.S. and India to address broader global development challenges in selected developing countries in Africa and Asia. The project has the following objective:  To address human and institutional capacity gaps in food and nutritional security in select African and Asian countries The expected outcome is that the trained professionals will contribute to poverty reduction in beneficiary countries through improved food and nutritional security. Project approach: The project approach is to implement trainings in India and in target countries. In India, a total of thirty-two 15-day training courses at select Indian institutions will be conducted. Each course module will focus on themes or subsectors in which Indian institutions are reputed or have demonstrated a comparative advantage to offer such trainings and ensure that it effectively responds to the target countries’ capacity gaps. Specific sub-sectors and themes are to be chosen based on an initial need assessment in target countries. In addition to the trainings in India, the project will conduct a total of twelve, 10- day trainings in select target countries in Africa and Asia, for up to 50 participants per session. Project Geographic Scope: The evaluation will cover the program implementation in 17 African and Asian countries. The countries include Afghanistan, Cambodia, Lao-PDR, Myanmar, Mongolia, and Vietnam in Asia and Botswana, Democratic Republic of Congo, Ghana, Kenya, Lao-PDR, Liberia, Malawi, Myanmar, Mongolia, Mozambique, Rwanda, Sudan, Tanzania, Uganda in Africa. 30 III. Evaluation Purpose and Use The objectives of the mid-term performance evaluation are: to measure, document, and determine whether the Manage FTF-ITT project has achieved its objective of addressing human and institutional capacity gaps in food and nutritional security, in select African and Asian countries. The mid-term evaluation will also provide lessons learned, recommendations and suggestions for mid-course corrections and help guide the Mission on future project design. IV. Evaluation Questions The FTF-ITT mid-term evaluation seeks to answer the following evaluation questions:  To what extent have the interventions been effective in achieving the overall objectives? i) How effective are the training programs in fulfilling the requirement of the participants? Are these trainings demand driven? j) How cost effective, i.e., cost per unit of output, is the project’s implementation of the training programs and what factors have most affected costs? k) What is the composition of the trainings’ participants undergoing the training in terms of the selection of countries and the subject areas? l) How effective is the quality of the training programs in terms of the modules, syllabus, and capacity of the trainers, both at MANAGE and partner institutions? 2. To what extent has the new knowledge acquired during trainings been adopted in the target countries? Has there been a measurable development outcome in the agricultural practices/target communities? a) To what extent and how have the trainings contributed to improved capacity of the participants and how have the participants utilized the knowledge and skills gained in these training programs? b)Did the training by MANAGE address the human and institutional capacity gaps in food and nutritional security in the target counties? c) What is the level of success achieved in adopting the new knowledge gained during the trainings in the target countries and the sustainability of these initiatives? d) What are the specific enablers and barriers (both contextual and project interventions) that influenced outcomes and would also affect the sustainability of the interventions? 3. To what extent has the project interventions addressed gender issues? 4. Recommend specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project. V. Existing Information Below is a list of preliminary documents the evaluation team will review. USAID/India will assist the evaluation team in obtaining as needed additional documentation that may be available. The 31 Evaluation Team is encouraged to look for and use other documents that may be useful for the evaluation.  FTF-ITT Project Appraisal document  FTF-ITT PMP and M&E data including the results framework  FTF-ITT Annual reports  FTF-ITT Performance reports  Any other project document VI. Evaluation Design and Methodology This evaluation will be conducted by the Alder-Modus International Corporation. We propose to apply a mixed-method approach to answer the evaluation questions, including qualitative and quantitative data collection to evaluate the performance of the FTF-ITT. The evaluation will include data collection methods such as quantitative and qualitative data collection, individual or focus groups and structured interviews, checklists and other tools. Quantitative and qualitative data collection methods include:  Review of documents, strategy documents, project reports  Surveys using questionnaires and/or data collection instruments  Key informant interviews with implementing partners and key stakeholders  Focus group discussions  Case studies if possible  Performance reports and data The evaluation will primarily focus on training programs on various agriculture and food security related aspects provided to participants from 17 countries (11 African and 6 Asian countries). The evaluation team will visit three countries (two African and one Asian country) to do the field survey of the participants. In addition, the evaluation team will visit MANAGE and selected associate institutions to interact with the stakeholders. In addition to the methods and data sources described above the evaluation team will use any other data that may be available or could be collected from stakeholders and or beneficiaries through cost-effective approaches. Any other relevant third-party data sources will also be used to answer some of the evaluation questions. The evaluation team will ensure data analysis methods are in line with best practices. For both quantitative and qualitative data, the evaluation team will articulate methodologies for analyzing collected information, including any statistical software programs to be used. For qualitative data, the evaluation team will ensure key informant interviews and/or focus groups are recorded and transcribed. The evaluation team is highly encouraged to propose an evaluation approach and set of methods that are cost-effective and robust enough, despite the limited time and budget, in order to draw a strong picture of the sustainability of the USAID/India FTF-ITT outcomes. The evaluation team responding to this SOW will propose specific data analysis methods on a question-by-question 32 basis, including the appropriate mix of methods necessary to respond to the evaluation questions. Gender and other relevant beneficiary characteristics should be part of data analysis. Strengths and limitations of the proposed evaluation design and methodology should be disclosed in the final evaluation design and report. VII. Evaluation Team Composition USAID anticipates that the evaluation team will include two core members: a Team Leader supported by a Senior Agricultural Extension/ Training Specialist (agriculture policy /extension /market or capacity building expert). Senior Evaluation Specialist, (Team Leader): The Senior Evaluation Specialist will be responsible for the overall implementation of the evaluation, ensuring that all expected tasks and deliverables are achieved on time and of high quality. S/he will oversee the overall design of the evaluation framework, including methodology determinations, organize of calendar/travel/meetings, oversee interviews and other data collection events, and analyze the data with input from team members to draft the evaluation report and presentation. The team leader will be responsible for aggregating the findings and to produce a quality report Key qualifications expected for the Team Leader include:  Master’s degree with at least 10 years professional experience coordinating similarly complex evaluations in the agriculture sector, and leading evaluation teams.  Exceptional organizational, analytical, writing, and presentation skills.  Knowledge of evaluation methodologies and their practical applications  Previous experience in innovation transfer, scaling up and sustainability of these efforts in various geographies are highly desirable.  Prior work experience in Kenya is highly preferred. Senior Agricultural Extension/ Training Specialist The Senior Agricultural Extension/ Training Specialist will work in close coordination with the Team Leader and will be actively engaged in efforts to oversee and ensure the quality of data collection and documentation. The Senior Agricultural Extension Specialist should have at least 10 years of international agricultural experience and a Master’s degree in a related field. S/he must have significant experience in agricultural extension, capacity building/ training, agri-business/agriculture commercialization, and should have prior experience working in African and Asian countries. Home Office Support Home Office support will be provided by the firm that will be implementing this evaluation, as required, including quality assurance, research and analysis support, financial management, administrative oversight, and logistics. 33 USAID Participation To enhance the quality of the evaluation, the USAID staff may participate in the evaluation except for certain data collection, analysis, and reporting tasks. VIII. Evaluation deliverables The following are key evaluation deliverables and their estimated due date: Deliverable Estimated Due Date 1. Draft Evaluation Design Proposal 05/03/2019 2. Final Evaluation Design Proposal 05/07/2019 3. Debriefing of Preliminary Findings 06/06/2019 4. Draft Evaluation Report 06/07/2019 5. Final Evaluation Report 06/25/2019 6. Debriefing of final evaluation 06/25/201 All documents and reports will be provided electronically to USAID. All qualitative and quantitative data will be provided in electronically to USAID in a format consistent with ADS 579 requirements. Prior to the submission of the final evaluation design proposal, the evaluation team will discuss with USAID whether its preliminary dissemination plan for this evaluation indicates other deliverables that should be prepared. Such additions as agreed with USAID will then be included in the final evaluation design proposal. Evaluation Design Proposal Prior to the implementation of data collection activities for this evaluation, the evaluation team will deliver an evaluation design proposal that describes the conceptual framework for the evaluation and the justification for selecting this approach. USAID/India must provide its approval of the design proposal before the evaluation team begins in-country data collection. The design proposal must at least contain the following:  Discussion of the overall approach of the evaluation, highlighting the conceptual model(s) adopted by evaluation question and demonstrating a clear understanding of the FTF-ITT intervention logic. Discussion of the data collection and data analysis methods that will be used to answer each evaluation question, and the limitations for each method. To ensure the quality of the evaluation, the proposed evaluation design must use a mixed￾methods and rigorous social science research methods.  Detail key data sources that will be selected to inform the answer to each evaluation question.  Detail of analysis methods to be used for qualitative and quantitative data  Discussion of risks and limitations that may undermine the reliability and validity of the evaluation results, and the proposed mitigation strategies for each. 34  Summarized evaluation methodology in a matrix that contains for each evaluation question: measure(s) or indicator (s), data collection method(s), data source, and data analysis method(s).  Timeline showing the key evaluation phases (e.g., data collection, data analysis, and reporting) and specific deliverables and milestones.  Responsibilities and qualifications of each evaluation team member  Discussion of the USAID staff’s participation in each evaluation phase and their anticipated roles, responsibilities, and reporting requirements.  Discussion of logistical considerations for carrying out the evaluation, including specific assistance that will be required from USAID, such as providing arrangements for key contacts within the mission or government.  Detailed estimated budget. Draft Evaluation Report The evaluation team will prepare a thirty-page maximum draft evaluation report (excluding Annexes) for USAID review. The draft evaluation report must contain at least the following:  Executive Summary: This section should be up to five pages in length and describe the purpose, project background, evaluation design and methodology including the evaluation questions, and key findings, conclusions, recommendations, and lessons learned from the evaluation.  Background: This section will provide a brief description of the FTF-ITT that highlights its scope, development hypothesis, and activities undertaken.  Evaluation Design and Methodology: This section will detail the overall evaluation design and methodology and related research protocols undertaken in conducting the evaluation, including the relevant data collection and analysis methods, sampling approach, and related challenges or limitations encountered during the evaluation and mitigation approaches employed.  Findings: This section will present findings collected from the evaluation relevant to each evaluation question. The findings must be specific, concise, and supported by the quantitative and/or qualitative evidence analyzed through scientifically plausible methodologies.  Conclusions: The evaluation report will present evaluation conclusions that are interpretations and judgments based on the findings described, and must logically follow from the gathered data and findings and be explicitly justified. If necessary, the evaluation team will state its assumptions, judgments, and value premises in presenting a conclusion so that readers can better understand and assess them.  Recommendations: This section will concisely and clearly present recommendations that are drawn from specific findings and conclusions provided in the report. The recommendations must be stated in an action-oriented fashion and be practical, specific, and with defined target audience(s). Final Evaluation Report 35 Following receipt of all USAID comments on the draft evaluation report, the evaluation team will prepare a final version that incorporates and responds to this feedback. The final evaluation report should contain the same sections as noted above for the draft evaluation report and should also include:  References: This section should include a list of all documents reviewed, including background documentation.  Annexes: These may include, but are not limited to, the evaluation statement of work, instruments used in conducting the evaluation, any statements of differences received, as well as other relevant sources of information. The final report must meet the evaluation report quality criteria described in Annex A of the USAID Evaluation Policy. VIII. Scheduling and Logistics The following chart provides an illustrative overview of the preliminary estimated timeframe for the evaluation and key deliverables. The evaluation design proposal will include a detailed schedule and proposed delivery dates. The following chart provides an illustrative overview of the preliminary estimated timeframe. The evaluation design proposal will include a detailed schedule and proposed delivery dates. IX. Scheduling and logistics The following chart provides an illustrative overview of the preliminary estimated timeframe for the evaluation and key deliverables. The evaluation design proposal will include a detailed schedule and proposed delivery dates. The following chart provides an illustrative overview of the preliminary estimated timeframe. The evaluation work plan proposal will include a detailed schedule and proposed delivery dates. Task Estimated completion date Evaluation SOW finalized 05/07/2019 Evaluation design finalized 05/07/2019 In-country work in India 05/07 to 05/13/2019 In-briefing with the mission 05/07/2019 Team planning meeting and piloting instruments 05/07/2019 Team meeting and data collection in Hyderabad 05/10/2019 Team meeting and data collection in Trivandrum 05/12/2019 In-Country work in Cambodia Team planning meeting and piloting instruments 05/16/2019 Data collection and preliminary analysis 05/16/2019 In-Country work in Kenya Team planning meeting and piloting instruments 05/20/2019 Data collection and preliminary analysis 05/20/2019 36 In-Country work in Uganda Team planning meeting and piloting instruments 05/22/2019 Data collection and preliminary analysis 05/22/2019 Data analysis 06/09/2019 Draft report for comments 06/12/2019 Comments from USAID 06/17/2019 Final report 06/25/2019 Report debriefing with USAID-India 06/25/2019 The evaluation team will be responsible for all logistics for its team members, including coordinating all travel throughout the region, lodging, printing, office space, equipment, car rentals, etc. The USAID staff participating in data collection activities will be responsible for their own lodging, car rentals, printing, etc. and the evaluation team will coordinate closely with the participating USAID staff on field work logistics. USAID or its host government partners will provide support to set up initial meetings with key stakeholders. X. Evaluation Budget AMI Corporation will propose a budget as part of the evaluation design proposal. 37 Annex B: Getting to answers matrix Evaluation question Evidence needed Data source(s) Data collection methods Data collection instruments Sampling approach Data analysis methods 1. To what extent have the interventions been effective in achieving the overall objectives? Yes/No Project documents and relevant secondary sources Progress reports -Desk review - KIIs - Group interviews - Structured observation Data collection template KII guides for Implementing agencies KII guides for Trainers KII guides for Trainees Convenience sampling depending on the ability to identify and contact informants -Planned/actual comparisons -Descriptive statistics and analysis Yes Description Yes Comparison Explanation 1a. How effective are the training programs in fulfilling the requirement of the participants? Are these trainings demand driven? Yes/No Project documents and relevant secondary sources Progress reports -Desk review - KIIs - Group interviews - Structured observation Data collection template KII guides for Implementing agencies KII guides for Trainers KII guides for Trainees Convenience sampling depending on the ability to identify and contact informants -Planned/actual comparisons -Descriptive statistics and analysis Yes Description Yes Comparison Explanation 1b. How cost effective, i.e., cost per unit of output, is the project’s implementation of the training programs and what factors have most affected costs? Yes/No Project documents Project accounts Project cost data Project log frame and indicator data on activities, outputs, outcomes, and impacts -Desk review - KIIs - Group interviews - Structured observation Data collection template Microsoft excel applications Convenience sampling depending on the ability to identify and contact informants from the implementing agency with knowledge of project components and costs associated with the project components Descriptive statistics Comparisons with data on similar programs implemented elsewhere and may be available in the literature Yes Description Yes Comparison Explanation 1c. What is the composition of the trainings; participants undergoing the training in terms of the selection of countries and the subject areas? Yes/No Project documents Project reports Evaluation reports Assessments -Desk review - KIIs Data collection template KII guides for Implementing agencies Convenience sampling depending on the ability to identify and contact informants from the implementing agency with knowledge of country selection and subject areas Descriptive statistics on participant countries, subject areas, Review of relevance of subject areas in the context of agricultural sector in the countries Yes Description Explanation 1d. How effective is the quality of the training programs in terms of the modules, syllabus, and capacity of the trainers, both at MANAGE and partner institutions? Yes/No Project documents Project reports Training materials Reputation of training institution and trainers Desk review KII of trainers KII of trainees Focus group meetings of beneficiaries in beneficiary countries Case studies, if possible Site visits of potential impact sites Data collection template KII guides for trainers KII guides for trainees Convenience sampling of trainers Convenience sample of trainees Descriptive statistics Planned/actual comparisons Outputs and impacts with training compared to baseline Yes Description Yes Comparison Explanation 2. To what extent has the new knowledge acquired during trainings been adopted Yes/No Project documents Project progress reports Desk review of documents and reports Data collection template KII guides for trainers KII guides for trainees Convenience sample of trainees Descriptive statistics Planned/actual comparisons Yes Description With and without project comparisons 38 in the target countries? Has there been a measurable development outcome in the agricultural practices/target communities? Yes Comparison Data from secondary sources KII of trainers KII of trainees FGD of target communities Site visits Convenience sample of beneficiaries in FTF countries Convenience sample of beneficiary communities in FTF countries Explanation 2a. To what extent and how have the trainings contributed to improved capacity of the participants and how have the participants utilized the knowledge and skills gained in these training programs Yes/No Project documents Project progress reports Data from secondary sources Data collected from participants Desk review of documents and reports KII of trainers KII of trainees Case studies of project participants Data collection template KII guides for trainers KII guides for trainees Convenience sample of trainees Convenience sample of trainees Descriptive statistics Planned/actual comparisons Yes Description With and without project comparisons Yes Comparison Explanation 2b. Did the training by MANAGE address the human and institutional capacity gaps in food and nutritional security in the target counties? Yes/No Project documents Annual reports Evaluation reports Desk review of documents and reports KII of trainers KII of trainees Data collection template KII guides for trainers KII guides for trainees Convenience sample of trainees Convenience sample of beneficiaries in FTF countries Descriptive statistics With and without project comparisons Indicators of sustainability Yes Description Yes Comparison Explanation 2c. What is the level of success achieved in adopting the new knowledge gained during trainings in the target countries and the sustainability of these initiatives? Yes/No Project documents Annual reports Quarterly reports KII with trainees KII with beneficiaries KII with beneficiary communities Desk review of project documents, annual reports PMP and indicator data Data collection template KII guides for trainers KII guides for trainees Convenience sample of trainees Convenience sample of beneficiaries in FTF countries Convenience sample of beneficiary communities in FTF countries Descriptive statistics Planned and actual comparisons Yes Description With and without project comparisons 2d. What are the specific enablers and barriers (both contextual and project interventions) that influenced outcomes and would also affect the sustainability of the interventions? Yes/No Project documents KII with trainers KII with trainees KII and/or focus group interviews with beneficiaries Project documents KII with trainers KII with trainees KII and/or focus group interviews with beneficiaries Data collection template KII questionnaires Convenience sampling Planned versus actual comparisons Case studies Yes Description 3. To what extent has the project interventions addressed gender issues? Yes/No Project documents Project progress reports Data from secondary sources Gender disaggregated data on trainees Gender disaggregated data on beneficiaries and beneficiary communities Desk review of documents and reports Project reports Project progress reports Gender disaggregate d data from project beneficiaries and beneficiary communities KII of trainees Case studies of project participants Data collection template KII questionnaires Convenience sampling Planned versus actual comparison Descriptive statistics Yes Description Comparison 39 4. Recommend specific opportunities to enhance project effectiveness and sustainability of the initiatives for the remaining period of the project Recommendations based on results for the above questions. 40 Annex C: List of trainees who participated in Focus Group Discussions (Cambodia) Name Designation Office 1 Koh Chaino Deputy Director Personal of MAPF 2 lay Lini Officer Department of Agricultural Extension 3 Cheymontha Chief Department of Agroindustry 4 Lay Pisey Officer DAPAIC/GOA 5 Yem Moeurn Vice Chief DAPAIC/GOA 6 Phon Chansophal Head of Lab PNCA 7 Pauv Samrit Officer PNCA 8 Ouk Sokhet Vice Chief DACP 9 Kim Sothea Official DIC 10 Khien Sokny Cice Chief PNCA 11 En Net Official DIC 12 Seng Seyuk Vice Chief PDAFF.TBK 13 Hina Pirun Vice Chief POAFF-TBK 14 Sun Sreayleah Officer DPARD 15 PhathBugphum Officer PDAFF.TBK 16 Choup Rith Vice Chief PDAFF.TBK 17 Long Puthodono Vice Chief Aidoc 18 Bou Socheata Officer DPS/MAFF 19 Vaug Vanthy Vice Chief DACP/GDR/MAPF 20 Meas Sothany Deputy Director DPS/MAFF 21 Ouk Savy Officer DPHRD 22 Sim Thy Deputy Director Thoung Karum 23 Tith Sarah Chief of HRD office MAFF 24 Bunny Peask Vice Chief DPHRD/MAPFF 41 Annex D: List of trainees who participated in Focus Group Discussions (Kenya) 1 Wycliffe Amariati Ministry of Agriculture, Livestock, Fisheries & Irrigation 2 Stanley K. Kirui Ministry of Agriculture, Livestock, Fisheries & Irrigation 3 Maurice Onyango Ministry of Agriculture, Livestock, Fisheries & Irrigation 4 Albert Mwangi Ministry of Agriculture, Livestock, Fisheries & Irrigation 5 Caleb Imusineya Ministry of Agriculture, Livestock, Fisheries & Irrigation 6 Mary W. Kanyi Ministry of Agriculture, Livestock, Fisheries & Irrigation 7 Elizabeth M. Kilonzo Ministry of Agriculture, Livestock, Fisheries & Irrigation 8 Mary Morara Ministry of Agriculture, Livestock, Fisheries & Irrigation 9 Roseline Ambani Bukura Agricultural College 10 Benson Mbun Mganja Ministry of Agriculture, Livestock, Fisheries & Irrigation 11 Stephen K Kamani Kenya Agricultural and Livestock Research Organization 12 Stanley Humaiya Ministry of Agriculture, Livestock, Fisheries & Irrigation 13 Wafula M.Mathias Ministry of Agriculture, Livestock, Fisheries & Irrigation 14 Janne Nyesya Ministry of Agriculture, Livestock, Fisheries & Irrigation 15 Leonard Kipkisgeri Kenya Seed Company 16 Faith Gitahi Ministry of Agriculture, Livestock, Fisheries & Irrigation 17 Atika Ombacki Ministry of Agriculture, Livestock, Fisheries & Irrigation 18 Allan Njorge Agricultural Development Commission 19 Jamon Kiptoo Training and extension Services 20 Rinth Mwangi Private sector 21 Lucy N. Njenga Ministry of Agriculture, Livestock, Fisheries & Irrigation 22 Leah Warfla Ministry of Agriculture, Livestock, Fisheries & Irrigation 42 Annex E: List of trainees who participated in Focus Group Discussions (Uganda) 1 Nankay Betty Kyto Mushroom Growing Group 2 Badaru Gertrude Ministry of Agriculture 3 Kasule Timothy Ministry of Agriculture 4 Msamba David Ministry of Agriculture 5 Nabachwa Mary Jacquiline Ministry of Agriculture 6 Tugume Besteo Ministry of Agriculture 7 Abaasa Innocent Stephen Ministry of Agriculture 8 Nalweyuso Amina Agricultural Engineer 9 Rosemary Naggujja Plugowa New Cycle Company 10 Mulumba Mutema Mathias Ministry of Agriculture 11 Tubwangye Samuel Ministry of Agriculture 12 Owami Simian Olk Ministry of Agriculture 13 Mila Douglas Wabirongo Womens coffee Production Group 14 Dominic Foram Ministry of Agriculture 15 Magala Jingh APO Belura DLG 16 Bugemba Richard Ministry of Agriculture