PERFORMANCE EVALUATION OF THE FEED THE FUTURE APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE (AREA) ACTIVITY FINAL EVALUATION REPORT Haiti Evaluation and Survey Services (ESS) Project February 2022 This publication was prepared independently by Social Impact, Inc., at the request of the United States Agency for International Development. AREA’s plant pathology team conducting plant diagnostic training at the American University of the Caribbean in Les Cayes. (AREA photo) April 2020. PERFORMANCE EVALUATION OF THE FEED THE FUTURE APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE (AREA) ACTIVITY DRAFT EVALUATION REPORT Date: February 2022 Submitted to: USAID/Haiti Contract Number: AID-521-C-17-00002 Contact: Salima Mutima, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV TABLE OF CONTENTS Abbreviations and Acronyms ..................................................................................iii Executive Summary ..................................................................................................1 Introduction and Evaluation Purpose....................................................................10 Introduction ............................................................................................................................................................................10 Situational Analysis of the Haitian Agriculture Sector .................................................................................................10 Evaluation Purpose and Evaluation Questions...............................................................................................................13 Activity Theory of Change ..................................................................................................................................................13 Interventions and Intended Results...................................................................................................................................14 Evaluation Methodology .........................................................................................16 Planning and Preparation .....................................................................................................................................................16 Data Collection ......................................................................................................................................................................16 Sampling Strategy ...................................................................................................................................................................16 Data Analysis...........................................................................................................................................................................18 Dissemination and Utilization.............................................................................................................................................18 Methodological Limitations.................................................................................................................................................18 Findings .....................................................................................................................20 Lessons Learned.......................................................................................................42 Recommendations...................................................................................................43 Annexes ....................................................................................................................44 Annex A. Statement of Work ............................................................................................................................................44 Annex B. Data Collection Protocols................................................................................................................................53 Annex C. Documents Reviewed........................................................................................................................................61 Annex D. Quantitative Surveys..........................................................................................................................................64 Annex E. Preliminary List of Key Informants..................................................................................................................68 Annex F. List of AREA Publications in Peer-Reviewed Journals................................................................................70 Annex G. List of Articles AREA Translated into Creole ............................................................................................73 Annex H. List of Articles Published with the MARNDR Collaboration..................................................................74 USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | ii TABLES AND FIGURES Table 1. Distribution of the Survey Sample According to the Length of the Training........................................................18 Table 2. Plant Diseases Diagnosed at the MARNDR Plant Pathology Laboratory ..............................................................23 Table 3. Research Programs Implemented by Area.....................................................................................................................24 Table 4. Technologies Tested by Area with the Extension Models Experiment..................................................................31 Table 5. Distribution of the Experiment Features........................................................................................................................32 Table 6. Training Sessions Per Intervention (Research Programs) ..........................................................................................35 Table 7. Lessons Learned....................................................................................................................................................................42 Figure 1. Relationship Between The Overarching Ftf Goal And Area’s Intermediate Results, Sub-Intermediate Results, And Indicators.......................................................................................................................................................15 Figure 2. Number And Percentage Of Individuals Interviewed By Stakeholder Group .....................................................17 Figure 3. Mycotoxin Analysis Unit In The Microbiology Laboratory, December 2021 (Et)..............................................22 Figure 4. Dark Field Microscopes The Laboratory Needs Needed For Certain Protocols In Which Area Held Trainings On Identifying Fungi (Et, 2021).......................................................................................................................25 Figure 5. Samples Of Bean And Green Pepper Seeds At Famv Laboratory (Et, 2021) ......................................................27 Figure 6. Types Of Training Received By Respondents..............................................................................................................36 Figure 7. Relevance Of The Training Received By Respondents..............................................................................................39 Figure 8. Training Usefulness .............................................................................................................................................................40 Figure 9. Effect Of The Training On Behavior Change ...............................................................................................................41 iii | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV ABBREVIATIONS AND ACRONYMS AREA Appui à la Recherche et au Développement Agricole CHIBAS Centre Haïtien d'Innovation sur les Biotechnologies et l'Agriculture Soutenable CIMMYT International Maize and Wheat Improvement Center CITI Collaborative Institutional Training Initiative CRDD Rural Center for Sustainable Development EQ Evaluation Question ESS Evaluation and Survey Services ET Evaluation Team FAMV Faculté d’Agronomie et de Médecine Vétérinaire FCR Findings, Conclusions, and Recommendations FFS Farmer Field School FGD Focus Group Discussion FONHDAD Haitian Foundation for Sustainable Agricultural Development FtF Feed the Future GI Group Interviews GoH Government of Haiti HQ Headquarters IP Implementing Partner IRB Institutional Review Board ISD Input Supply-Driven KII Key Informant Interview LSU Louisiana State University M&E Monitoring and Evaluation MARNDR Ministère de l’Agriculture, des Ressources Naturelles et du Développement MF Master Farmer NGO Non-Governmental Organization Ph.D. Doctor of Philosophy PICSA Participatory Integrated Climate Services for Agriculture PTRA Programme Triennal de Relance Agricole QA Quality Assurance QU Quisqueya University SI Social Impact, Inc. SoW Scope of Work ToC Theory of Change UF University of Florida UI University of Illinois UNEPH Université Episcopale d'Haïti UNIQ Université Quisqueya UPC Université Publique du Centre U.S. United States USAID United States Agency for International Development WFP World Food Program FY Fiscal Year 1 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV EXECUTIVE SUMMARY OVERVIEW The activity Appui à la Recherche et au Développement Agricole (AREA) is a five-year Feed the Future Haiti multifaceted initiative beginning in 2015 with a $13.7 million award from the United States (U.S.) Agency for International Development (USAID). Its implementing partners are the University of Florida (UF) as the prime implementer and Louisiana State University (LSU) and the University of Illinois (UI) as the sub￾partners. The activity covers Port-au-Prince, Kenscoff, Cul-de- Sac, and Matheux corridors, but its capacity-building activities had national coverage. The objective of the AREA activity was to increase the availability of improved production technologies to farmers and the private sector through effective extension and development of an agricultural innovation system. Its theory of change hypothesizes that improved access to agricultural innovations would likely improve food security for Haitian farmers. Specifically, if Haitian farmers used improved technologies more widely, combined with more effective agricultural extension services and nutrition information, their livelihoods would improve. EVALUATION PURPOSE AND METHODS USAID/Haiti requested that Social Impact’s (SI) Evaluation and Survey Services (ESS) design and conduct the final performance evaluation of the Feed the Future AREA Activity implemented by UF, LSU, and UI. The purpose of this final performance evaluation is to inform USAID/Haiti as it considers future similar agriculture programming in Haiti. Specifically, USAID/Haiti seeks information on how to (1) better engage higher education institutions in agriculture, (2) support a local system to conduct applied research and support an extension system to contribute to a more productive agricultural sector, (3) identify any lessons learned and recommend possible follow-up actions, and (4) guide similar USAID programs in Haiti and elsewhere. This final evaluation addresses four evaluation questions (EQs): 1. To what extent and in what ways did the AREA activity improve applied agricultural research at local universities? In answering this question, the evaluation team (ET) should examine the immediate effectiveness and long-term sustainability of the approaches the project used to build capacity in applied research. 2. AREA conducted and facilitated a variety of research. How was this information disseminated, and to what extent and how did various stakeholders take it up? 3. On the basis of AREA’s applied research and experimentation, what agricultural extension models (or aspects thereof) appear to be the most promising and why? 4. To what extent and in what ways did AREA’s interventions and training (both long and short term) increase or improve trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities? EVALUATION LIMITATIONS During the evaluation kick-off meeting, USAID alerted the ET to potential limitations in the project’s a monitoring and evaluation (M&E) data, nonetheless, the ET was able to validate its findings using other sources. The ET reviewed and commented on the methodology or methodologies AREA used to collect data on key indicators in its findings to mitigate this potential limitation. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 2 Moreover, AREA did not have a database of beneficiaries for the ET to use for purposive sampling. The ET was given hand-written lists of beneficiaries for various AREA activities. Those lists included many duplicate names of people who had participated in different activities. The initial lists provided to the ET included 1,666 names who turned out to be 750 individuals, requiring adjustments to the initial sample for the surveys. Some AREA stakeholders opted not to participate in the evaluation when contacted by the data collection team. Self-selection introduces the potential for bias because those who opt to participate may tend to have more extreme negative or positive views. The ET relied on several sources to triangulate its findings and mitigate this potential bias. To respond to EQ3, the ET needed the results of the experiments conducted by UF on the different extension models. At the time of this evaluation, UF had not published those results and could not provide the ET with the results. Therefore, the ET relied on group discussions, stakeholder interviews, and document review to address EQ3. This evaluation relied mainly on qualitative data collection methods to respond to EQ3. Therefore, the ET cannot generalize the findings and conclusions regarding the experiment beyond the participants’ groups and communities. Finally, the ET was limited by Haiti's safety and security situation during the evaluation. Political and social unrest and a protracted fuel shortage prevented the ET from visiting all the sites they planned to visit. The ET observed all of ESS’s safety and security protocols and stayed abreast of unrest or other harmful activities. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EQ 1: To what extent and in what ways did the AREA activity improve applied agricultural research at local universities? In answering this question, the ET should examine the immediate effectiveness and long-term sustainability of the approaches the activity used to build capacity in applied research. FINDINGS AREA conducted research directly to address the problems identified in the Haitian agricultural sector, undertaking several research programs, sometimes involving local entities. AREA sought local institutional collaboration with universities and Rural Agricultural Research Centers (CRDDs) on some research projects (KIIs with universities, the Government of Haiti [GoH], CRDDs, and IPs). For example, AREA partnered with Université Quisqueya (UNIQ) to implement the legume breeding program. It also worked with the Bas boen CRDD on the plant pathology program (KIIs with CRDDs, GoH, and IPs) and FAMV on the soil program (KIIs with IPs and universities). The partnership with local universities aimed to strengthen their capacity to develop and implement adequate applied agricultural research. In trying to improve the capacity of local universities and partners to conduct research, all KII stakeholders’ groups agreed that AREA leveraged existing resources at universities and CRDDs to improve applied agricultural research. AREA supported local institutions in the agricultural research sector at various levels: community-based organizations, research institutions, universities, and public and private institutions. 3 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV AREA offered direct financial support for some of their research activities by providing mini-grants to local researchers. This allowed AREA to identify solutions to several problems related to plant diseases fertilization and discover locally adapted and high-yield seed varieties. AREA increased the number of scientists involved in research in Haiti while collaborating with local institutions on some research programs. Although local institutions were involved, most of the latter felt that their level of involvement was not significant enough and should have started during the Activity planning phase. For example, while trying to increase the number of researchers, AREA hired local and international experts to lead some research projects at local universities, but key informants told the ET that this was neither efficient nor sustainable. AREA also provided long-term master’s degree training to 25 Haitians in U.S. universities to increase research capacity, yielding unexpected results. The students were well trained and two of the twelve interviewees, were even hired by AREA after graduation. However, since some of the students found the salaries in local institutions too low, they sought employment outside of the network of local institutions. Others decided to return to the United States to pursue their Ph.D. AREA provided short-term training to professionals involved in research in Haiti. AREA provided a list of 1,666 short-term trainees for the survey, which the ET reduced to a sampling frame of 750 unique trainees (319 female and 431 male) because many participants attended multiple trainings throughout the activity. Some interviewees (CRDDs, Universities, USAID) noted that, with more involvement of the institutions in decision-making processes, AREA could have had more sustainable results. Respondents told the ET that AREA’s choice to contract researchers locally and internationally and pay their salaries directly was not a sustainable way to operate in the Haitian context. In addition, the sustainability of some of AREA interventions has been heavily constrained by external factors, such as the sociopolitical situation that prevailed throughout the activity’s implementation. CONCLUSIONS AREA appropriately understood that human resources are vital to improving applied research at local universities. This is reflected in AREA’s focus on local institutions and training. AREA also realized that laboratories and research institutions in Haiti were no longer functional due to a lack of public and private investment in the sector. Therefore, AREA decided to equip these laboratories and research institutions and support access to funding to implement research programs capable of finding solutions to problems facing the agricultural sector. AREA has also invested in improving training curricula to help train better researchers in Haiti. AREA’s choices would have been more viable had local institutions been more involved in identifying implementation options. AREA invested heavily in training students at the graduate level by providing scholarships to 25 students to complete their master’s degrees in the United States. These students all completed their degrees, meaning that more Haitian nationals are now involved in agricultural research. According to AREA, all young professionals returned to Haiti initially to work as government officials, researchers, or in the private sector, but then some returned to the United States to complete a Ph.D. However, some KIIs noted that it could have been worth exploring cost-effective options such as strengthening the higher education system so that more young researchers could emerge. Finally, AREA’s investments in agricultural research produced results, but their viability is threatened by the environment, the lack of interest from the government and the private sector, and the activity’s lack of an exit strategy. EQ 2: AREA conducted and facilitated a variety of research. How was this information disseminated, and to what extent and how did various stakeholders take it up? USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 4 FINDINGS AREA conducted trainings for selected stakeholders to disseminate scientific knowledge and modern technologies. Local researchers were trained on AREA-promoted technologies to support their dissemination activities. They were also trained on using the newly donated equipment to their laboratories and how to conduct some basic lab testing. Finally, local researchers were trained to interpret scientific data from their experiments and publish their work in peer-reviewed journals. Farmers who took part in the experiments learned how to use AREA-promoted technologies and passed the knowledge on to other farmers. While there was a clear understanding for first-line beneficiaries, AREA did not plan to reach second-line beneficiaries, limiting the activity’s dissemination coverage. Also, AREA concentrated most of its energy on capacitating local researchers rather than farmers. According to some interviewees, this was justified since AREA’s primary objective was to support local institutions to perform their work more effectively. AREA also organized workshops for dissemination purposes. It supported the participation of local and international researchers in workshops to disseminate the results obtained through its various research programs in Haiti. However, some GoH respondents felt that AREA did not do enough to target local audiences such as local scientists and farmers, probably because of a lack of local scientific journals or other appropriate means of dissemination. Finally, AREA published articles in peer-reviewed journals reaching out to the international scientific community in climate change, plant pathology, seeds and crop management, soil fertility management, gender, and water resources, according to KIIs with most relevant stakeholder groups (USAID, IPs, and universities). AREA-supported researchers published a total of twenty-three articles in international journals. Local universities and other local institutions participated in the dissemination exercise by producing articles, scientific publications, and booklets. A consortium of local institutions (universities and CRDDs) is working on multiplying seeds of bean and corn varieties that AREA promoted through the legume breeding program and the corn research program for wider dissemination to farmers (CRDDs, IPs, and universities). CRDDs were involved in dissemination by supporting farmers in better understanding the newly introduced technologies. CONCLUSIONS AREA had many research programs that generated mixed results. AREA disseminated those results directly in its extension activities for farmers, during workshops, and in peer-reviewed journals. AREA extensively trained researchers, and local partner staff to increase their knowledge about the results of its research and strengthen their capacity to use them. Local partners, universities, and research centers that took part in the activity interventions engaged research results through seed multiplication, extending newly acquired knowledge to other parts of Haiti, and reaching out to other researchers in national and international networks. Farmers directly involved in the experiments noted an increase in their knowledge and the benefits of the promoted technologies. Although AREA and their partners engaged in extension activities, the coverage of local audiences such as farmers groups was still low since very few farmers apart from those directly involved in the experiment and training were reached via its interventions. 5 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV EQ 3: On the basis of AREA’s applied research and experimentation, what agricultural extension models (or aspects thereof) appear to be the most promising and why? FINDINGS According to the experiment’s results, the Master Farmer (MF) and the Farmers Field School (FFS) models both led to better utilization of technologies in the experimentation area (KIIs with AREA staff), MF where farmers belong to moderate-resource farmers associations and FFS where farmers belong to low-resource farmers associations. Besides the models, the strength of the farmers’ association played a fundamental role in technology adoption. In addition, factors related to location may influence the experiment’s outcome, including prior exposure to the Master Farmers model in the experiment area. Prior USAID projects in the experiment area, such as WINNER and CLP, have promoted the use of MF as an extension model. Many farmers’ association leaders in the experiment area are certified MFs. Hence, they successfully increased their knowledge and adequately used the promoted technologies, which could have been different if they had not had prior exposure. Because of the minimal involvement of researchers and extensionists in the Input-Supply Driven (ISD) model, farmers who took part in that experiment were, in most cases, not satisfied with their outcomes. The MF and FFS models that carry a minimum level of the organization were quicker to support technology dissemination, according to KIIs with IPs. Farmers in Focus Group Discussion (FGDs) praised their respective extension model as the best model, except for the farmers using the ISD model. In all 3 FGDs with farmers of the MF model, they indicated that MF is the best model. Likewise, farmers of the FFS model said that FFS was the best model. No farmers in FGDs claimed the ISD was the best model. In all three sets of FGDs, farmers said they would continue to use the technologies, but they could not find inputs/materials. Irrespective of the extension model, farmers shared that the most important factors determining their adoption or not of a given technology are their user-friendliness, the cost or time they require, the demand on local markets, and the return on investment. About the dissemination of new technologies, farmers will pick them up if they can perceive the potential benefits, including more production, more revenue, and a market for production. CONCLUSIONS Considering the experimental condition, ceteris paribus, the MF and FFS models triggered farmers’ utilization of technologies in the Kenscoff region. Farmers appreciated MF and FFS, saying they met their need for information to make informed decisions about adopting new technology. FFS and MF brought the information and support that the farmers needed to adopt the technology, and such support was reinforced by the activities of the farmers’ association. A strong association contributed significantly to triggering farmers’ utilization of technologies, this support being more efficient with the MF model. In other words, the performance of the models varied with association strength in terms of adoption rate. The association provided much-needed additional follow-up and support for farmers to adopt new technologies. The association leaders worked in synergy with the MFs to support utilization, although the ET could not determine the magnitude of this USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 6 support. Ultimately, as shared by FGD participants, farmers’ behaviors towards the new technologies were dictated by the perceived benefits they potentially yield. EQ 4: To what extent and in what ways did AREA’s interventions and training (both long and short term) increase/improve trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities? FINDINGS AREA provided training to a wide range of audiences on various topics in the agriculture sector. AREA provided a list of names with 750 unique short-term training participants. Researchers and lab technicians were trained to use the newly distributed equipment and perform specific laboratory tests related to AREA-promoted technologies. AREA provided long-term training to 25 students to pursue graduate studies at U.S. universities. Recruitment was fair and open, according to KIIs. However, KIIs criticized the lack of a plan for the professional integration of scholarship students. AREA hired some of them after completion of the master’s program, based on the Activity’s interest in their research topics. According to AREA staff, some of them returned to the United States, pursuing Ph.D. degrees. Most of the training provided by AREA was deemed relevant and useful. While some key informants thought the training was relevant and AREA provided trainees with adequate skills for their jobs, some thought the training was too short (local institutions). ● Of 462 survey respondents, 128, or 28 percent, did not hear about the theme before the training. ● However, 377, or 86 percent, claimed that the training was related to their work. ● From 462 survey respondents, 345, or 75 percent, agreed that the training was very relevant; and 323, or 70 percent, said they had hands-on training. ● Considering the weakness of extension services in Haiti (see EQ2 findings), AREA conducted specific training about extension and invited GoH representatives to participate. ● Of 462 respondents, 447, or 97 percent, thought they learned something useful, and 433, or 94 percent, claimed they have used what they learned from the training at work. ● Some 321, or 69 percent, indicated that supervisors asked them to use the newly acquired knowledge. ● However, only 205, or 44 percent, had new tasks that required going back to the training materials, and just 262, or 57 percent, could enumerate two new skills that they used after the training In addition, trainees noted that they were experiencing behavior change because of the training they had received from AREA ● Of 462 respondents, 426, or 92 percent, claimed to do things differently at work after the training. Some 453 respondents, or 98 percent, would recommend the training to peers. ● About 94 percent of the trainees agreed that they are better researchers due to the training. Among respondents, 89 percent thought that the new skills are required for their jobs, and only 24 percent claimed that they would hire someone without these skills for the job. 7 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV CONCLUSIONS AREA performed well in identifying relevant topics and identifying appropriate recipients to the short￾term training, which also appears to have increased trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities in Haiti. The training proved helpful in addressing the needs for better skills at research institutions and triggered trainees’ behavior change. The topics were thoughtfully chosen and deemed relevant according to trainees. In some cases where farmers were involved, they perceived the training to be too short. AREA trained graduate students to address the lack of human resources in local research institutions. While some of them were able to secure a contract with AREA (two out of twelve surveyed students), other returned to a university abroad to further pursue higher education. For several reasons, many long-term trainees did not end up joining the local institutions. AREA provided some equipment at local laboratories and trained technicians on their use. But laboratories at local institutions are still under-equipped: At the Activity’s end, assisted laboratories still did not have the equipment necessary to support research on critical issues, such as low crop yields for many crops, the impact of environmental deterioration on production systems, or the low adoption of recommended agricultural innovations in Haiti agriculture. LESSONS LEARNED The following table describes both what worked well and what did not work well in AREA’s interventions: WHAT WORKED WELL WHAT DID NOT WORK WELL EQ 1 – AREA support to applied agricultural research Mini-grants to support ad hoc research in agricultural value chains had some impacts. According to many stakeholders, neither the government nor the private sector invests enough. The support provided by AREA to local researchers through the provision of mini￾grants was a success, looking at all the ad hoc solutions that those researchers identified. Provision of equipment to laboratories and research institutes: Haiti's agricultural laboratories are under￾equipped. The provision of lab equipment is a foundation for successful research in the sector. AREA distributed essential lab equipment that included microscopes to detect the most common plant diseases. Support to laboratories through their equipment was a good choice. Local laboratories, universities, and research institutes still need equipment and additional support. Under EQ 1, the ET noted that the equipment provided to laboratories was essential but technology gaps to perform adequate lab testing still exist. Future support could focus on equipping one reference lab at least to run those tests locally if need be. Enlisting public and private support to AREA activities was not a success. AREA did not attract more government and private-sector funding in applied agricultural research and training. The lack of an exit strategy is not a good practice. Apart from the routine research undertaken by universities and some local partners, most of the interventions AREA supported have ceased after the funding stopped. AREA did not have an exit strategy. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 8 WHAT WORKED WELL WHAT DID NOT WORK WELL EQ 2 – Dissemination of AREA knowledge products Provision of support to publish scientific papers and articles: AREA succeeded in supporting the publication of scientific articles in peer-reviewed journals. Interviewees from the IP, universities, USAID, and CRDDs agreed that the body of knowledge generated through AREA is tremendous and believe it will contribute to further developing the sector. EQ 3 – Extension models in the agricultural sector The MF and FFS models fit well with farmers’ needs. Farmers who participated in the extension models experiments recognized that the MF and FFS models, as opposed to the ISD model, are adapted to their situation. The ISD model was not as effective as the other models. Research on the extension models showed that the ISD model, which relied only on exposure to technology, was not adapted to the needs of the farmers. The major outcome of the AREA experiment was that the strength of the associations influences the performance of the models. Strong association leaders influence farmers’ decisions to adopt sound technologies, regardless of the model. Higher synergy takes place with the MF model. The availability of inputs for farmers to continue to use the technologies is critical and was not a success. while AREA’s main objective was not technology dissemination, effective implementation of the AREA’s proposed approaches were grounded in the availability of certain promoted technologies that were unfortunately unavailable to farmers once the Activity ended, limiting their ability to continue implementing the promoted approaches. Irrespective of the technology, the availability of inputs in local markets is key to adoption on a broader scale. EQ4 – AREA support to training of agricultural stakeholders Short-term trainings did help improve trainees’ capacities. Trainees from all stakeholders’ groups agreed that short￾term training helped improve their capacity and provided them with the needed skills to perform their duties better. Support to the sector through short-term training of agricultural professionals appears to be an excellent option. Long-term training of graduate students overseas is not the best option, considering the risk of emigration and the possibility of strengthening local institutions with a local master’s program. Sending graduate students overseas is not the best option to immediately improve the sector within Haiti. An alternative option could have been to reinforce the local universities graduates’ programs to provide the same type of degrees locally. RECOMMENDATIONS To USAID, the evaluation recommends to: 1. Provide ad-hoc institutional support to local institutions and universities that agree to contract the trainees for at least a year following their studies. 2. Strengthen local universities sustainably by supporting their graduate program and research activities, developing adequate training curricula, and upgrading laboratories in local universities. 3. Further, support the equipment of laboratories at local research institutions to address critical issues in Haitian agriculture. 4. Ensure that a plan to disseminate research results to local audiences, not just the scientific community, is rolled out for future similar programs. 5. Support the implementation of an exit strategy for future research projects to ensure that research continues after the funding ends and that relevant input (technology) is available to 9 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV those who want to utilize and later adopt the technology, even if they are a second or third-line actor. 6. Support the utilization of the MF and FFS models in the Kenscoff region to disseminate agricultural technologies and work with Haitian extension services to determine the best extension model for the other regions. 7. Strengthen farmers' associations to enable them to play a more substantial role in extension activities USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 10 INTRODUCTION AND EVALUATION PURPOSE INTRODUCTION Appui à la Recherche et au Développement Agricole (AREA) is a U.S. Agency for International Development (USAID) Feed the Future (FtF) initiative that supported multiple approaches to address the root causes of hunger and poverty and forge long-term solutions to chronic food insecurity and under-nutrition in Haiti. In 2015, USAID signed the AREA activity cooperative agreement to increase the availability and implementation of improved agricultural production technologies and systems and strengthen agricultural technologies and nutrition information extension systems. SITUATIONAL ANALYSIS OF THE HAITIAN AGRICULTURE SECTOR The Republic of Haiti has an area of 27,750 square kilometers (km2) and a population estimated today at over 10 million people. It is a predominantly mountainous country with more than half the land with slopes greater than 40 percent and plains occupying only 5,500 km2, or 20 percent of its total area. The agriculturally exploitable potential is 7,700 km2 (29 percent), while the area cultivated is around 11,900 km2 (44%), which indicates that 4,200 km2 of marginal land is under cultivation.1 The agriculture sector is critical to the Haitian economy, being the primary source of income for approximately 60 percent of the population.2 The sector constituted 20.5 percent of the country’s gross domestic product (GDP) in 2017.3 Despite the sector's importance, its contribution to the GDP has steadily declined over the past years. PRIMARY CULTIVATED CROPS IN HAITIAN AGRICULTURE From before Haiti’s independence until the 1970s, agriculture in Haiti was oriented toward industrial and export crops, such as coffee and cocoa. Mountainous areas over 900 meters are well suited for coffee cultivation, making it possible to produce quality coffee while protecting the environment and allowing cocoa to be grown in the lower altitudes. Coffee and cocoa served the dual purpose of providing agricultural incomes and protecting the environment by acting as cover crops that reduced topsoil erosion by water or wind. However, food crops were also grown alongside these crops.4 Nowadays, the main cereal crops grown in Haiti are maize, rice, and sorghum (MARNDR, 2014), which also make up the staple food of the population. Currently, these crops are cultivated in approximately one-third of the total agricultural land. FOOD SECURITY CONCERNS Food insecurity and food shortages are common in Haiti. From a nutritional standpoint, cereal crops are essential, supplying around 37.5 percent of the population’s total caloric intake. Rice alone supplies nearly one-fifth of the total caloric intake. Average caloric intake is around 511 kilocalories lower than 1 Bigi, A. 2020. Guide de référence sur les pratiques de subvention et de lutte contre l’érosion dans le département du Sud d’Haïti. 2 According to different sources, including the World Bank and the Institut Haitien de Statistique et d’Informatique (IHSI). 3 http://www.ihsi.ht/produit_economie_comptes_na_pib.htm. 4 MARNDR. 2015. Diagnostic des systèmes de production en vue de la relance de la vulgarisation agricole dans les communes d’intervention des projet RESEPAG et PTTA. 11 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV the required level of 2500 kilocalories per day, and protein intake is nearly 7.3 grams lower than the daily recommended 56 grams.5 Food insecurity has been a longstanding challenge in Haiti, exacerbated by natural disasters, a lack of resources to increase farm productivity, and a myriad of other factors. Rapid population growth (1.2 percent in 20206) combined with inappropriate farming practices resulted in pressure on natural resources and acceleration of land degradation. Fifty-two percent of the population is chronically undernourished.7 As a result, Haiti ranks 170th of 189 countries on the 2020 Human Development Index, and, according to the World Food Program, “Haiti has one of the highest levels of food insecurity in the world. Around 4.4 million Haitians, or nearly half the population, need immediate food assistance, and among these, 1.2 million suffer from severe hunger.”8 LOW PRODUCTIVITY IN AGRICULTURE IN HAITI National agricultural production accounted for only 48 percent of food availability in 2010, complemented by commercial imports (44 percent) and food aid.9 In the early 1980s, these imports accounted for only 19 percent of the food supply. Current rice crop productivity is among the lowest in Latin America. Yields of maize, rice, and sorghum have declined since the 1990s. Yields of critical food security crops fall short of the country’s yield potential, partly because traditional varieties are not optimally adapted to Haiti’s diverse agroecological zones. Plant diseases and pests cause significant crop losses, yet Haiti has few laboratories to diagnose these problems. Post-harvest losses are substantial and range between 50 and 60 percent of production10 depending on local conditions, contributing to lower farmer incomes. Adopting new technologies, such as improved seeds and production methods, suffers from ineffective or limited formal agricultural extension and technology transfer programs. There are also a lack of quality seeds, poor irrigation infrastructure, low access to credit, weak governmental extension services, poor quality soil and water, and natural disasters. AGRICULTURAL POLICIES IN HAITI The Government of Haiti’s (GOH) agriculture sector policies, strategies, programs, and plans are presented in a series of documents, of which the Ministère de l'Agriculture des Ressources Naturelles et du Développement Rural—MARNDR/Politique de Développement Agricole, 2011, is the centerpiece.11 In line with the country’s growth and poverty reduction strategy, the GoH’s overall agricultural policy objective is to contribute sustainably to meet the Haitian population’s food needs and socio-economic development. Specific objectives are to (1) increase the domestic consumption coverage rate by domestic production from 45 to 70 percent, (2) ensure that the agricultural sector provides a decent 5 https://www.copenhagenconsensus.com/publication/haiti-priorise-agriculture-rd-bairagi. 6 https://data.worldbank.org/indicator/SP.POP.GROW?locations=HT. 7 FAO. 2014. The State of Food Insecurity in the World. Strengthening the enabling environment for food security and nutrition. 8 WFP, 2021. Haiti Country brief. https://www.wfp.org/countries/Haiti. 9 http://agriculture.gouv.ht/view/01/?-Securite-Alimentaire,58-. 10 https://www.copenhagenconsensus.com/publication/haiti-priorise-post-harvest-losses-theodat. 11 Other important policy documents include the Programme Triennal de Relance Agricole, 2013; Plan Directeur de Vulgarisation Agricole, 2011; Plan National d’Investissement Agricole (PNIA), 2010; Plans Stratégique et Programmatique 2014-2019 pour la Modernisation des Services de Protection Agro-phytosanitaire et d’Innocuité des Aliments, 2013; and Plan Directeur de la Formation Professionnelle Agricole, 2013. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 12 income to about 500,000 farm households, and (3) increase the coverage of imports by agricultural exports from 5 percent (2009 figure) to 50 percent in 2025.12 Haiti’s agricultural development policy13 favors (i) a territorial approach based on the potential and specifics of Haiti’s agro-ecological zones and seeking alignment and complementarity among the different actors, (ii) a watershed approach14 from summit to sea level, and (iii) a sectoral approach gathering the GoH, non-governmental organizations (NGO), the private sector, and producer organizations around the same table.15,16 NEEDS FOR RESEARCH AND EXTENSION SERVICES IN HAITI The success of Haiti’s agricultural policies requires a vital research and development system to solve the problems discussed above. The country still needs to develop national strategies to strengthen the Haitian agriculture research and extension system. Haiti has no formal, ongoing investments in agricultural research and development. The government has set up research entities in the past, but many were unsuccessful and ceased operations.17 While universities and local institutions continue to conduct some agricultural research, it does not address the most important needs such as low crop yields for many crops, the impact of environmental deterioration on production systems, the low adoption of recommended agricultural innovations. COMPLEX SOCIAL DISTINCTIONS IN RURAL HAITI In addition to the agricultural-related problems, social distinctions, including class and gender, are complex in rural Haiti. Formal and informal social institutions regulate much of households’ access to critical resources. Informal institutions like social networks reflect and reinforce class and gender inequalities.18 Failures to understand how these inequalities function and are reproduced across generations can cause interventions to create outcomes that exacerbate social divisions. In addition, gender plays a vital role in both access to and control of resources and women’s participation in agriculture. In Haiti, rural women are the most vulnerable. Despite their vulnerability, they play a crucial role in developing agri-food chains and sustaining income-generating activities.19 They contribute significantly to agricultural production, crop processing, food security, and nutrition; land and natural resource management; and building adaptive capacity to climate change. According to the traditional division of labor in agriculture, women are primarily involved in small-scale trade activities, small processing units (cassava, peanut butter, honey, etc.), accompanying people in the fields (picking and harvesting activities), sorting charcoal, preparing food, selling fish, etc. Women often have a significant 12 Politique de développement agricole 2010-2025. 13 Politique de développement agricole 2010-2025. 14 The watershed approach is a coordinating framework for environmental management that focuses public and private sector efforts to address the highest priority problems within hydrologically defined geographic areas, taking into consideration both ground and surface water flow. 15 Les axes prioritaires de l’administration Moise, 2018-2022, and Politique de développement agricole 2010-2025. 16 Feuille de route du ministère de l’Agriculture et Les axes prioritaires de l’administration Moise, 2018-2022. 17 “Etude de l’évaluabilité du programme triennal de relance agricole (PTRA) 2013-2016 du MARNDR (Haiti),” Marie Line Daphnée François. 2018 18 Haiti: A Country Social Analysis, 2006. https://publications.iadb.org/publications/english/document/Country￾Program-Evaluation-Haiti-(2001-2006).pdf. 19 http://ingenaes.illinois.edu/wp-content/uploads/ING-Landscape-Study-2016-Haiti-published-2016_03_09.pdf, 2016. 13 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV role in the local market but may be excluded from participation in global value chains. Successful entrepreneurial women often serve as role models for other women, sometimes providing less experienced women with resources. EVALUATION PURPOSE AND EVALUATION QUESTIONS USAID/Haiti requested Social Impact’s (SI) Evaluation and Survey Services (ESS) activity to conduct a final performance evaluation of the AREA activity implemented by the University of Florida (UF), Louisiana State University (LSU), and the University of Illinois (UI). The purpose of this final performance evaluation is to inform USAID/Haiti’s design and implementation of future agricultural programming. Specifically, USAID/Haiti seeks information on how to: (1) better engage higher education institutions in agriculture, (2) support a local system to conduct applied research and support an extension system that would contribute to a more productive agricultural sector, (3) identify any lessons learned and recommend possible follow-up actions, and (4) guide similar USAID programs in Haiti and elsewhere. USAID tasked the evaluation team (ET) with answering four evaluation questions (EQs): 1. To what extent and in what ways did the AREA Activity improve applied agricultural research at local universities? In answering this question, the ET should examine the immediate effectiveness and long-term sustainability of the approaches the project used to build capacity in applied research. 2. AREA conducted and facilitated a variety of research. How was this information disseminated, and to what extent and how did various stakeholders take it up? 3. On the basis of AREA’s applied research and experimentation, what agricultural extension models (or aspects thereof) appear to be the most promising and why? 4. To what extent and in what ways did AREA’s interventions and training (both long and short term) increase/improve trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities? ACTIVITY THEORY OF CHANGE To address the problems in the Haitian agriculture sector discussed above, AREA sought to build the capacity of public and private institutions and organizations to provide results-oriented agricultural research and extension programs that are inclusive of all farmers, promote gender equity, and are appropriate for Haiti. The Activity’s theory of change (ToC) is that effective and inclusive agriculture research programs would benefit all Haitian farmers by increasing the availability of improved production technologies and systems by strengthening the extension of agricultural technologies and nutrition information.20 AREA’s strategy recognized that it cannot increase food production and security using a merely technical approach. According to the AREA strategy, successful approaches in Haiti tend to share key characteristics. They are market-driven, oriented toward the private sector, incorporate value chain relationships, provide economic benefits that outweigh the risks, and respond to the nation’s social, environmental, and geographic diversity.21 The activity was designed to conduct field research to analyze the complex issues affecting Haitian farmers’ food security and classify farm types based on shared characteristics. AREA would then develop and test appropriate interventions to increase food 20 Cooperative Agreement No. AID-OAA-A-15-00039. University of Florida. USAID, 2015. 21 AREA proposal. AREA, 2015. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 14 production and security for these different groups. AREA planned to continue, expand, and extend successful interventions to the types of commercially oriented farmers who are most likely to adopt or adapt and benefit from these technologies. Smallholder farmers who serve the domestic market could also be interested in these technologies. Furthermore, subsistence farmers have little or no access to resources other than their labor. Indeed, their resources are often so limited that social networks and the social capital they exchange provide essential factors of production. The Activity’s ToC stressed that innovations that address seasonal hunger gaps and improve household food production would likely strengthen their food security. Developing a typology of farmers in the target regions would help identify the most appropriate technologies for each category and inform the best ways to deliver these technologies to increase adoption.22 INTERVENTIONS AND INTENDED RESULTS AREA’s overall goal was to build the capacity of Haitian institutions to increase the availability of improved production technologies to farmers and the private sector through effective extension services and the development of an agricultural innovation system. Developing functional and sustainable strategies to generate knowledge, create process and product innovation, and extend information to farmers requires building on successful existing models and forging linkages to the national and international efforts already in place. The activity worked to develop and grow the capacity of government institutions and private organizations to serve the agricultural sector with research and extension programs that increase productivity in the target regions. To implement its ToC, AREA focused on three major components: ● Research. AREA worked to increase the Haitian agricultural research community’s effectiveness by improving access to and availability of scientific knowledge and technological innovations that lead to increased farm productivity. The activity’s three-stage approach to building capacity in applied research was designed to leverage existing resources, increase the number of agricultural research scientists, and provide producers with research-based diagnostic services. ● Extension. Effective extension services play a key role in farmers’ adoption of improved technologies, resulting in higher yields and incomes. Thus, AREA investigated models that have been used to transfer technologies and information to Haitian farmers. It implemented a research-based approach to identifying effective extension models for different farmers, including female farmers, essential to designing an equitable, sustainable, and pluralistic system to increase technology adoption rates. ● Capacity building through higher education. A third component was to increase the capacity of Haiti’s agricultural universities and improve their curricula so that graduates could better contribute to meeting Haiti’s agricultural sector needs. AREA partnered with seven agricultural colleges to develop an innovative higher education program to better prepare the next generation of university students. The activity also worked with U.S. land grant universities to provide Master of Science (MS) degree scholarships to future agricultural leaders. 22 Cooperative Agreement Number: AID-OAA-A-15-00039. DRAFT Monitoring and Evaluation plan. AREA, 2015. 15 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV Under the FtF goal to “sustainably reduce poverty and hunger,” AREA was designed to contribute to the objective of “Improved Agricultural Productivity” through Intermediate Result 1, “Improved Agricultural Sector Productivity” (see Figure 1 below for the indicators and interventions in three major sections): ● Sub-Result 1.1: Enhanced human and institutional capacity development for increased sustainable agriculture sector productivity ● Sub-Result 1.2: Enhanced technology development, dissemination, management, and innovation ● Sub-Result 1.3: Reduce gender disparities in access to, control over, and benefit from resources, wealth, opportunities, and services – economic, social, political, and cultural. Figure 1 presents a visual overview of the relationship between the overarching FTF goal and AREA’s intermediate results, sub-intermediate results, and indicators. This framework describes the process by which AREA planned to achieve these sub-intermediate results in its intervention locations by providing illustrative activities and relevant indicators. FIGURE 1. RELATIONSHIP BETWEEN THE OVERARCHING FTF GOAL AND AREA’S INTERMEDIATE RESULTS, SUB￾INTERMEDIATE RESULTS, AND INDICATORS USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 16 EVALUATION METHODOLOGY PLANNING AND PREPARATION This evaluation took place from August 2021 to February 2022. The period included approximately five weeks for a desk review of AREA’s documents; drafting the inception report and evaluation design report, including the development of data collection protocols and other data collection planning; five weeks for data collection and analysis; and four weeks for finalizing data analysis and writing the draft and final reports, including USAID’s review of the draft report. The evaluation began with a kick-off call with ESS staff, USAID/Haiti personnel, the UF representative, and the ET. The purpose of the kick-off call was to discuss USAID’s expectations for the evaluation and its results and to make sure all parties had a shared understanding and agreement on the overall evaluation schedule. Following an initial review of AREA materials (e.g., the project proposal, documents related to agricultural development and research in Haiti, program monitoring reports, research reports, etc.), the ET drafted the evaluation inception and design reports. Together, the reports identified the main findings and information gaps related to each EQ, the draft questionnaires and other data collection protocols, a timeline, a listing of data sources and respondent selection procedures, and a discussion of evaluation design limitations. DATA COLLECTION The ET used a mixed-methods evaluation design. This includes qualitative and quantitative methods and comprised document review, key informant interviews (KIIs), focus group discussions (FGDs), direct observation of AREA-supported activities, and trainees’ surveys. The interview guides and surveys consisted of questions derived from the EQs, document review, initial discussions with USAID, and the ET’s evaluation design knowledge. SAMPLING STRATEGY The study participants included staff from the IP, USAID/Haiti personnel, and other direct activity stakeholders. In addition, the ET included farmers and trainees who had indirect contact with AREA personnel or service providers. KIIs. The ET used a purposive sampling approach and conducted individual and small-group KIIs with 58 individuals (46 males, 12 females) from the following key stakeholder groups: USAID staff, Rural Agricultural Research Centers (CRDD) agents, GoH staff, staff from Haitian universities, implementing partners (IPs) including prime implementers and subcontractors, local partner organizations, and representatives of farmers’ groups. The ET chose purposive sampling to ensure that representatives from all AREA stakeholder groups participated. KIIs and GIs lasted approximately 60–90 minutes each. Figure 2 below gives a breakdown of the different interviewees. 17 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV USAID, 3, 5% CRDD, 3, 5% GoH, 4, 7% Universities, 4, 7% IPs, 17, 29% Other local institutions, 27, 47% USAID CRDD GoH Universities IPs Other local institutions FIGURE 2. NUMBER AND PERCENTAGE OF INDIVIDUALS INTERVIEWED BY STAKEHOLDER GROUP FGDs. The ET used convenience sampling to select farmers to participate in FGDs for each extension model. The ET conducted 12 mixed-gender FGDs with AREA trainees and AREA-affiliated farmers involved in the extension model experiments. The ET mobilized 6 to 12 participants for each two-hour discussion that covered key EQ issues, especially for EQ3. The FGDs helped gather data to assess the extent to which AREA-promoted models were adapted to the needs of smallholder farmers. ● 3 FGDs with farmers who participated in the Master Farmer model experiment (25 males, 3 females) ● 3 FGDs with farmers who participated in the Farmers Field School model (11 males, 7 females) ● 3 FGDs with farmers who participated in the exposition to technology model (10 males, 10 females) ● 3 FGDs with trainees (14 males, 4 females) Direct observation. The ET visited AREA-supported laboratories at local universities and CRDDs to observe ongoing activities, including follow-up AREA-supported research. Because of the security situation that prevailed during data collection, the ET used convenience sampling to select three laboratories supported by AREA for direct observations. The selected laboratories were at the following three sites: ● Plant pathology laboratory at the Ministère de l’Agriculture, des Ressources Naturelles et du Développement (MARNDR) ● Aflatoxin laboratory unit at the Faculty of Agronomy and Veterinary Medicine (FAMV) ● The weather station of the school of agriculture at FAMV Survey. The ET administered online surveys consisting primarily of close-ended questions. The ET surveyed a total of 462 long- and short-term trainees (288 male, 174 female). The survey collected information on trainees’ participation in AREA interventions, curricula they received, the knowledge they acquired, their level of satisfaction with the training, and how they currently use knowledge gained through the training. The ET used the list of beneficiaries provided by UF and reached out to the entire group of trainees (750) in the list. For short-term trainees, 450 out of 750 responded to the survey. For USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 18 long-term trainees, only 12 out of 25 individuals responded to the survey despite the ET and USAID reaching out directly to all of them and urging them to participate in the evaluation. Table 1 below breaks down the survey sample according to the length of the training received by respondents. TABLE 1. DISTRIBUTION OF THE SURVEY SAMPLE ACCORDING TO THE LENGTH OF THE TRAINING METHOD STAKEHOLDER TOTAL Surveys Long-term trainees 12 Short-term trainees 450 Total surveys 462 DATA ANALYSIS The ET first analyzed data from each interview and field visit separately. The breadth of data sources used and the diversity of stakeholder perspectives sought were essential elements of this performance evaluation. This provided a broad outreach to stakeholders involved, allowing for triangulation of the findings during the data analysis phase. Then, an overarching analysis compared, triangulated, and synthesized perspectives and experiences across the different data sources and stakeholder groups. To ensure the methodologies were as thorough and reliable as possible, the ET employed various analytical tools such as qualitative content analysis, thematic analysis, and a Findings, Conclusions, and Recommendations matrix for qualitative data, as well as descriptive statistics, using SPSS, for the quantitative data. The ET then analyzed trends to identify possible quality improvements to inform the design of similar activities. DISSEMINATION AND UTILIZATION Following fieldwork, the ET conducted an out-brief with USAID/Haiti. Once USAID approves the final report, ESS will upload it to the Development Experience Clearinghouse. METHODOLOGICAL LIMITATIONS Data availability and quality. During the evaluation kick-off meeting, USAID alerted the ET to potential limitations in the project’s Monitoring and Evaluation (M&E) data, nonetheless, the ET was able to validate its findings using other sources. The ESS notified USAID that UF was not sharing the information they needed to respond to all the EQs. While the ET did have access to the publications produced by AREA, the field data from experiments and reports from some important activities such as workshops, equipment distribution, and field trip reports from AREA were not available. The ET reviewed and commented on the methodologies AREA used to gather data on key indicators in its findings to mitigate this potential evaluation limitation. Moreover, AREA did not have a database of its beneficiaries for the ET to use for sampling. The ET received hand-written beneficiary lists for various AREA interventions. Those lists included many people who had participated in multiple activities, resulting in duplicate inclusions of the same people. The initial lists the ET received included 1,666 names who turned out to be 750 individuals. Because the 19 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV dataset received from UF had 750 unique participants (as opposed to the 2,000 unique participants that the ET had estimated when calculating the initial sample), a new sample was calculated comprising 451 short-term trainees and 25 long-term ones. Therefore, the sample size was supposed to be 476 trainees, provided they were willing to participate, and 450 short-term and 12 long-term trainees ended up responding to the surveys. To respond to EQ3, the ET needed the results of the experiments UF conducted on the different extension models. At the time of this evaluation, UF had not published those results and could not provide them to the ET as those data are still being processed and will be published in peer-reviewed journals once they are done with the analysis. The ET accepted using the available data and triangulating between various sources such as FGDs, KIIs, and its document review to address this EQ. AREA performance reports were scrutinized, and the information they contained was verified with interviewees where possible. Participants’ self-selection. Some AREA stakeholders opted not to participate in the evaluation when contacted by the data collection team. Self-selection comes with potential bias because those who opt to participate may decide to do so because they have extremely positive or negative views. Again, the ET relied on several sources to triangulate its information and mitigate this potential bias. Generalizability. This evaluation relied mainly on qualitative data collection methods to respond to EQ3. Therefore, the ET cannot generalize the findings and conclusions regarding the experiment beyond the participants’ groups and communities. Security preparedness. During the evaluation, the ET was limited by Haiti's prevailing safety and security situation. Political and social unrest and a protracted fuel shortage prevented the ET from visiting all the planned sites. The three laboratories visited for direct observations were accessible during data collection; the other supported laboratories and institutions were not. The ET had wanted to do direct observations outside Port-au-Prince, but that was not possible because of a fuel shortage and security problems in Haiti during this evaluation. The ET observed all of ESS’s safety and security protocols and stayed informed of unrest and other harmful activities. Accessibility of some activity locations and participants. At the time of this evaluation, Haiti had just been hit by an earthquake in the Les Cayes region, which suffered widespread devastation and was not accessible to data collection teams. In addition, activity participants were not always available to speak with the team because they were dealing with the earthquake’s aftermath. The ET reached out to the staff of local partner organizations and the representatives of the farmers’ associations to request their help in finding other farmers who participated in the interventions. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 20 FINDINGS EQ 1: TO WHAT EXTENT AND IN WHAT WAYS DID THE AREA ACTIVITY IMPROVE APPLIED AGRICULTURAL RESEARCH AT LOCAL UNIVERSITIES? IN ANSWERING THIS QUESTION, THE ET SHOULD EXAMINE THE IMMEDIATE EFFECTIVENESS AND LONG￾TERM SUSTAINABILITY OF THE APPROACHES AREA USED TO BUILD CAPACITY IN APPLIED RESEARCH. AREA developed the technical capacity of local actors to conduct research. According to KII participants from universities and the implementing partner (IP), AREA strengthened university professors, students, and other local actors’ capacity to conduct research. For example, because access to relevant literature to inform research is critical to designing sound research projects, AREA reported in Q3 of 2019 that it completed training on access to free peer-reviewed scientific journals, books, and databases, and articles to improve university professors’ writing skills. Following the training, many Haitian scientists then participated in publishing the research that AREA supported. A list of this research is in Annex F. AREA also indicated in its 2019 quarterly reports that it trained and supported local researchers from NGOs, ministries, farmers’ organizations, and universities to conduct ethical research. Over 30 participants (24 percent of which were female) were trained by AREA on this topic, as reported in the AREA 2019 Q2 report. AREA also translated several documents, including a presentation titled Ethics & the Researcher, to Haitian Creole. A list of the other articles translated is in Annex G. The presentation was used as an alternative training to the Collaborative Institutional Training Initiative (CITI) Social and Behavioral Research Investigators training required by the UF Institutional Review Board (IRB) for all individuals working on research projects. The training was later used by AREA staff to train Haitian researchers. KII participants from universities and CRDDs mentioned that AREA held two workshops to help researchers and agricultural professionals to develop the skills needed to organize research trials, implement standard operating procedures in a breeding program, and integrate farmers into the plant breeding process. Those workshops followed a training held at UI in 2018. Also, in 2018, twelve researchers and agricultural professionals attended a workshop on maize seed systems. During the workshop, participants mapped out the maize research activities, planned their institutions, and discussed collecting maize phenotyping data. AREA later held a subsequent maize seed systems workshop focused on conducting participatory field trials. Ten researchers and agricultural professionals from MARNDR, FAMV, Bas Boën, and Montrouis CRDDs; CHIBAS, the Episcopal University of Haiti (UNEPH); and the American University of the Caribbean (AUC) attended. Participants in those workshops used their newly acquired knowledge to engage in AREA research programs, as indicated by university stakeholders and CRDDs groups. Additionally, as documented under EQ4, participants in these trainings generally agreed about the relevance of the themes and that they used the knowledge they acquired to do their jobs better. In addition, according to IP, university, and USAID KII participants, AREA supported students at specific local universities to conduct research for their theses. For example, the postharvest loss management and food safety program included a large-scale study investigating storage conditions and reducing aflatoxin contamination in grain crops and peanuts. AREA stored cereal grains (corn, rice, and sorghum) and peanuts in four public communal markets under traditional and improved conditions: Arcahaie, 21 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV Croix-des-Bouquets, Kenscoff, and Vialet. In preliminary work before conducting this research, AREA performed a short-term baseline study to determine the aflatoxin content of the commodities in these markets. A student selected from the Department of Food Science at FAMV conducted this preliminary study for his undergraduate degree. AREA supported long-term training for young Haitian researchers. Twenty-five college￾educated applicants were given scholarships to UF and LSU for graduate studies in agricultural sciences, including horticulture, biological engineering, and entomology. The students were well trained and two of the twelve interviewees, were even hired by AREA after graduation. However, KIIs from the universities, USAID, and the IP noted that some of the students who graduated from U.S. master’s programs were not currently engaged in research at local Haitian institutions. One explanation offered by a key informant in the universities group was that some of those students wanted to pursue their PhDs. Another reason was the low salaries paid at local institutions, which university representative KIIs found too low to attract young graduates. AREA provided funding for researchers in the agricultural sector. According to AREA performance reports and KII participants from universities, AREA developed a manual to guide grant recipients in managing their projects. The manual gave details of USAID and UF requirements related to the conduct of researchers, cost principles, auditing, and other award provisions. The prospective grantees later used the manual to submit their proposals and manage their grants. As per the third quarterly report of FY2018, 61 Haitian agricultural students, faculty, and professionals applied for competitive grants of up to $20,000 as part of AREA’s mini-grant program to support researchers, encourage collaboration, and address agricultural research priorities. Grant recipients included faculty members from local universities such as FAMV, Université Episcopale d'Haïti (UNEPH), and Université Publique du Centre (UPC). At the beginning of FY2020, the AREA annual report for the previous year noted that the activity had awarded 11 mini-grants up to $4,000 to support the research of local agricultural scientists. According to representatives from the universities and the IP, the mini-grants provided to local researchers improved their ability to identify solutions to several problems related to plant diseases, fertilization, and discovering new seed varieties farmers liked. Furthermore, according to USAID, some IPs, and universities, AREA modernized curricula to train scientists at FAMV. The AREA Higher Education program objective was to strengthen the capacity of Haiti’s agricultural higher education institutions and help improve curriculum to better fulfill the country’s agricultural sector needs. AREA consulted university and local institution representatives to identify curricula gaps in agriculture higher education. According to the IP and USAID group representatives, AREA improved 25 curricula in the universities and other higher-level educational institutions in Haiti. As a result of the curricular improvement, some KIIs from the universities group indicated they can now teach better, which was also confirmed by some trainees. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 22 FIGURE 3. MYCOTOXIN ANALYSIS UNIT IN THE MICROBIOLOGY LABORATORY, DECEMBER 2021 (ET) AREA conducted research with local universities and partners on several agricultural topics. KIIs in almost all stakeholders’ groups (IP and sub-partners, universities, CRDD, local partners, and farmers) indicated that AREA conducted research to directly address problems identified in the Haitian agriculture sector in collaboration with local partners. Examples include: ● In its 2015 Q4 report, AREA reported meeting with the University of Quisqueya to discuss ways to support institutional research capacity and address its research goals with faculty members. After the meeting, AREA, and the University of Quisqueya signed an agreement that later involved the university in several thematic research conducted with the university and AREA staff. In 2016, AREA brought together the UF Tropical Research and Education Center, the Division of Plant Defense of MARNDR, and FAMV to collaborate on research on nematodes on horticultural crops. Finally, AREA was able to bring together UF and the University of Quisqueya to work together on the soil salinity study undertaken with a local soil scientist, Dr. Wesly Lejeune. ● KIIs with representatives of local institutions and the IP revealed that, in 2019, AREA’s plant pathology laboratory technician collaborated with an AREA master’s degree student to train farmers in Montrouis and students and faculty at the American University of the Caribbean in Les Cayes. Together, they prepared the training modules and collected pepper roots infected by Meloidogyne sp. at the Bas Boën CRDD. The infected roots were maintained on live plant hosts in the CRDD’s greenhouse to study the evolution of the nematodes on pepper. ● According to KIIs with universities, IPs, and CRDDs, AREA worked with the Bas-Boen CRDD and the “Centre Haïtien d'Innovation sur les Biotechnologie et l'Agriculture Soutenable” (CHIBAS) through the AREA legume program. The CHIBAS research center of the University of Quisqueya partnered with AREA to initiate a legume-breeding program that focused on increasing the availability of improved legume cultivars such as peanut and pigeon pea common in Haiti. According to representatives of the local partners, the activity provided them with financial support, expertise from scientists directly hired by AREA, and various ad hoc trainings. The partnership with the Bas Boen CRDD and CHIBAS allowed AREA to plant two field trials to test the agronomic performance of four improved black bean lines AREA selected to increase 23 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV seeds. The results of the bean research have been published in two scientific articles. At the time of this evaluation, AREA was still waiting to receive the results from CHIBAS before making a final determination regarding the tested varieties’ performance. Under this collaboration, the activity conducted several other research programs, including seed production, adaptation trials, response to nodulation,23 and fertilization with several legumes. KIIs from local institutions and universities who were knowledgeable about this research all agreed the partnership strengthened legume research and improved Haiti’s research capacity to improve agricultural productivity and food security. ● KIIs with CRDDs, GoH, and IPs revealed that the plant pathology program with the CRDD allowed AREA to provide farmers with diagnostic services for various plant-related problems. AREA staff and local researchers regularly visited bean fields at the Bas Böen CRDD and on farms around the Plaine du Cul-de-Sac to help farmers better manage their plots and control pests and diseases through such interventions as recommending preventative measures and providing pest and disease identification and management guidelines. Though the program was discontinued in March 2020 because of coronavirus restrictions, by the end of the interventions, the diseases listed in Table 2 below had been diagnosed at the MARNDR plant pathology laboratory. TABLE 2. PLANT DISEASES DIAGNOSED AT THE MARNDR PLANT PATHOLOGY LABORATORY PLANT LOCATION DISEASE/SYMPTOMS CAUSAL/ASSOCIATED AGENT Common bean Plaine du Cul-de-Sac Insects Aphids, mites, thrips Common bean Plaine du Cul-de-Sac Physiological damage Not found Tomato (hydroponic) Kenscoff Fusarium wilt Fusarium oxysporum Peanut Mirbalais Peanut rust Puccinia arachidis Peanut Mirbalais Early and late leaf spots Cercospora arachidicola Rice St. Marc Sheat rot Fusarium sp./Sarocladium oryzae Rice St. Marc Blast Pyricularia grisea Common bean Plaine du Cul-de-Sac Rhizochtonia root rot Rhizochtonia solani KIIs from several stakeholders’ groups such as universities, IPs/sub-IPs, and local partners noted that AREA directly implemented several research programs addressing issues in various agricultural value chains, collaborating with local universities and research centers. AREA’s second annual performance report also documented several of these research programs and collaborations with local university partners, listed in Table 3. 23 Nodulation is the process of forming root nodules containing symbiotic bacteria. Bacteria provide nitrogen, and the plants provide carbohydrates in the symbiotic association. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 24 TABLE 3. RESEARCH PROGRAMS IMPLEMENTED BY AREA PROGRAM IMPLEMENTER LOCAL PARTNERS CROPS SOURCES Climate Smart Solutions AREA: UF, University of Reading Haiti’s Unité Hydrométéorologique, MARNDR Crosscutting Q2Y3, Pg. 9 Maize Seed System CIMMYT, UI CRDD Bas Boen Maize Q1Y3, Pg.14 Legume Research AREA, UF Quisqueya University (QU), University of Notre Dame* Beans, peanut, pigeon pea Q1Y3, Pg. 20 Plant Pathology AREA, UF CRDD Bas Boen, FAMV Crosscutting, banana, eggplant Q3Y3, Pg. 16 Soil Science Research AREA FAMV Crosscutting Q2Y3, Pg. 29 Postharvest and Food Safety AREA, UF FAMV Crosscutting, peanut Q2Y3, Pg. 49 MS and graduate certificate UF, LSU Q3Y3, Pg. 30 Small grant program AREA FAMV, UNEPH, UPC Crosscutting Q4Y3, Pg. 29 In addition, under the plant pathology program, AREA worked with FAMV in developing Djon-djon research led by a FAMV professor in his plant pathology laboratory, producing spawn for mushrooms using agricultural waste such as peanut shells, rice shells, coconut husks, and banana leaves. AREA conducted a soil attributes and mapping study on about 2,600 hectares in the Cul-de-Sac plain using the same approach. AREA built on this work to generate continuous, quantitative information on the distribution of soil attributes and properties that markedly influenced the availability of soil nutrients (i.e., pH, texture, electrical conductivity, and organic matter). The FAMV soil laboratory collaborated with AREA, running tests to measure soil organic matter, texture, nitrogen, phosphorus, and potassium content for this area. AREA worked with MARNDR to identify the causal agents of two diseases in Haiti. Both FAMV faculty members and professionals in plant protection within the ministry identified soft root disease on banana. Many farmers had at that time reported a loss of their banana crops due to an unknown disease. AREA worked with MANDR to conduct a preliminary study in July 2018 at four plantain farms in Arcahaie. The UF Plant Diagnostic Center ran tests on infected and control plants. They used high-tech technologies to identify Klebsiella variicola as the pathogen that was the source of the problem, which was never formally identified and documented in Haiti before that study. In trying to improve local universities and partners’ capacity to conduct research, all KII stakeholders’ groups agreed that AREA leveraged existing resources at universities and CRDDs to improve applied agriculture research. For example, AREA involved Dr. Wesly Jeune at the University of Quisquéya and the staff from the Bas-Boën CRDD in the soil salinity study. Dr. Wesly served as the lead researcher and developed the sampling strategy for the research. The research was conducted with the Bas Boen CRDD and the farmers’ organizations around the CRDD. In another example, AREA used a greenhouse at the Kenscoff CRDD to establish a crossing block for its trials to evaluate improved bean lines. Still at the Bas Boen CRDD, AREA worked with personnel to establish a designated laboratory area for the plant pathology program. Microscopes (see Figure 4) and other equipment purchased under 25 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV the WINNER project were tested and repaired to ensure they functioned properly. AREA ordered missing parts and chemical reagents from Florida to help upgrade the laboratory. The collaboration between AREA and local research institutions showed that the latter did not have the full range of technologies they needed to conduct fully-fledged applied research in agriculture. For example, when it came to identifying the cause of the soft root disease with bananas, local and international researchers teamed up to collect the field samples. Still, according to AREA performance reports and KIIs with representatives from the IP, the diagnosis and tests required to name the microorganism responsible for the disease were made in the United States at the UF plant pathology laboratory. Therefore, AREA launched an upgrading program for local universities and research centers to support their applied agricultural research activities. According to AREA performance reports, KIIs with IPs and sub-IPs, universities, CRDDs, and GoH, along with findings from direct observations, AREA supported laboratories at local institutions by providing new materials and equipment. AREA equipped partner laboratories, including FAMV, Bas boen CRDD, American University of Les Cayes, and the Ministry of Agriculture Vegetal Protection Lab, with high-powered microscopes and other essential scientific equipment to conduct research. The ET visited two laboratories and noted a few types of equipment that AREA provided (the mycology laboratory at FAMV and the MARNDR plant pathology laboratory [see Figure 3]). According to KIIs from GoH and the universities, the microscopes and other essential equipment AREA provided have increased their capacity to conduct basic research but were not enough to increase that capacity to perform many of the tests that labs at that level would be expected to run. Although AREA worked with the CRDDs and local universities to provide farmers with diagnostic services to solve their problems, representatives from the CRDDs, the universities, and USAID did not think that those local partners were fully involved in the decision-making processes in that FIGURE 4. DARK FIELD MICROSCOPES THE LABORATORY NEEDS NEEDED FOR CERTAIN PROTOCOLS IN WHICH AREA HELD TRAININGS ON IDENTIFYING FUNGI (ET, 2021) USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 26 collaboration. Instead, AREA simply requested that local partners implement interventions decided without sufficient consultation with them. According to respondents from CRDDs and universities, the lack of consultation with local partners led to some non-viable choices, such as AREA’s contracting of local and international staff to conduct research directly. This research dynamic came to a halt as soon as AREA ended because the CRDDs could not pay researcher salaries. According to CRDD and university representatives, the wages and benefits that AREA could pay differed from those provided by local universities. Therefore, as soon as the activity ended, local universities could not renew the contracts for these researchers. They reported cases of several researchers who left their institutions and got hired abroad or elsewhere after their contract with AREA. AREA supported women’s participation in applied agricultural research, according to representatives from the IP and university groups. AREA reported in the fourth quarterly report of FY2020 that in July 2020, the activity held a four-day virtual conference via Zoom, explicitly aimed at female agronomy students from partner universities in Haiti. The goal was to strengthen women’s capacities in the agricultural sector and inspire them to get involved in Haitian agriculture as entrepreneurs upon graduating. The workshop covered various topics, including the importance of failure in leadership, presented by the founder of PH4 Global, resources for Haitian startups, financial stability in micro-enterprises, and a case study of a real woman entrepreneur, the owner of Chokokap, which specialized in producing natural chocolate. The ET did not find any evidence that could allow them to conclude that trainees in this group went on and became better entrepreneurs following the training. FACTORS THAT ADVERSELY AFFECTED AREA PERFORMANCE IN SUPPORTING AGRICULTURAL RESEARCH The lack of conducive environments limited AREA’s performance in supporting applied agricultural research in Haiti and limited local institutions’ capacity to be involved in research. According to KIIs from GoH and the IPs/sub-IPs, there is no legal framework for research in Haiti. The lack of a legal framework limits agricultural research because it prevents the government and private partners from prioritizing it. Therefore, Haiti has no national strategic direction that defines the priorities of agricultural research, the operational mechanisms for conducting this research. According to KIIs from universities and CRDDs, because it is not a priority for the government, applied agricultural research is not a priority for public and private sector expenditures, which is a severe threat to AREA’s investments in the sector. For example, according to KIIs from universities, it is difficult for laboratories and research institutions to find all the chemical reagents they need when performing specific tests in local markets. Imports for such reagents are not attractive to the private sector since their demand and profits are generally low. Therefore, there is no structured supply chain for laboratories engaged in agricultural research. When needed, they must rely on the black market, which is not reliable in terms of either quantity or quality. The few providers of chemical reagents are independent providers who source their products, mainly from the Dominican Republic. AREA supported local laboratories by importing some of the reagents or sending samples overseas for testing when needed. Moreover, at the time of this evaluation, all key informants, except some from the universities, noted that there is still a lack of qualified human resources and infrastructure despite AREA’s investment. They do not have enough people, equipment, or budget to do the research they want to 27 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV do. Laboratories are still not functional because most are too old and lack the equipment needed to conduct proper applied research. FIGURE 5. SAMPLES OF BEAN AND GREEN PEPPER SEEDS AT FAMV LABORATORY (ET, 2021) The lack of an exit strategy has prevented some stakeholders’ groups from fully continuing the work undertaken by AREA. According to representatives from the universities, CRDD, and local partner groups, most of the work they had started with AREA has now come to a halt because of a lack of funding. At the time of this evaluation, farmers were still looking for some improved seeds of beans, maize, and groundnuts without success, and no one seemed to have a plan to produce and sell them in local markets. CONCLUSIONS AREA appropriately understood that human and material resources are vital to improving applied research at local universities. This is reflected in AREA’s focus on local institutions and training. AREA also realized that laboratories and research institutions in Haiti were no longer functional due to a lack of public and private investment in the sector. Therefore, AREA decided to equip these laboratories and research institutions and support access to funding for the implementation of research programs capable of finding solutions to the problems of the agricultural sector. AREA has also invested in improving training curricula to help better train researchers in Haiti. AREA’s choices could still be more viable if local institutions were more involved in identifying options for implementation. AREA has invested heavily in training students at the graduate level. These students have all completed their degrees, which means more nationals in agricultural research. Some of these students were able to enter the national research system. According to AREA, all young professionals returned to Haiti initially to work as government officials, researchers, or in the private sector, but then some of them returned to the United States to complete a Ph.D. However, some KIIs noted that it USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 28 could have been worth exploring cost-effective options such as strengthening the higher education system so that more young researchers could emerge. Finally, AREA’s investments in agricultural research have produced results, but their viability is threatened by the environment, the lack of interest from the government and the private sector, and a lack of exit strategy from the activity. EQ 2: AREA CONDUCTED AND FACILITATED A VARIETY OF RESEARCH. HOW WAS THIS INFORMATION DISSEMINATED, AND TO WHAT EXTENT AND HOW DID VARIOUS STAKEHOLDERS TAKE UP THE RESEARCH? AREA used extension activities with CRDDs, local partners, and farmers’ associations to disseminate the results of its research. For example, in collaboration with the University of Reading, AREA introduced the Participatory Integrated Climate Services for Agriculture (PICSA) method. This agricultural extension approach integrates location-specific climate information with participatory decision-making tools for farmers in the Kenscoff region. This activity centered on training extension agents to assist farmers in reducing climate-related losses. AREA then organized a workshop to train nine local researchers on evaluating the impact of the PICSA training. The AREA team compiled and translated the evaluation interviews and produced a fact sheet to provide partners with an overview of the results. AREA then shared the fact sheet through three workshops to disseminate the study results to local researchers and farmers (IP, universities, and CRDDs). Representatives from local partner groups indicated that the PICSA trainings were successfully extended to other parts of Haiti and had contributed to noticeably reducing crop losses. AREA conducted workshops to disseminate the results of its research and supported local researchers’ participation in national and international workshops, according to KIIs with USAID, IPs, universities, and CRDDs. KII participants from the IP, universities, and local partners similarly indicated that they participated in several workshops organized by AREA to discuss the results of research performed in Haiti. CRDD staff who participated in the workshop used the acquired knowledge to train farmers involved in the extension models experiments. CRS staff who participated in the workshops disseminated the PICSA training beyond AREA target areas. University and research institution staff trained in many different subjects matter recognized that the trainings were valuable and allowed them to do their jobs better (see the discussion under EQ4 for details). AREA completed trainings for various audiences to disseminate scientific knowledge and modern technologies. All relevant KII stakeholder groups (IP, universities, and local partners) indicated that AREA used training extensively to communicate research results, knowledge, and technology. For example, according to representatives from the universities and IPs, AREA trained researchers and students under the maize seed system to compile and analyze maize variety data and use phenotyping technology and standard operating procedures to conduct maize experiments. AREA trained 17 agricultural public and private sector professionals and researchers on phenotyping technology. Following that training, AREA organized a workshop for local researchers to teach them how to develop a list of codes and units of measurement for the traits measured. AREA used data compiled and obtained from its trial sites for that workshop. One outcome of the corn breeding training was the establishment of a network of local institutions, including: CHIBAS, FONDADH, CRDD de Montrouis, the Episcopal University of Haiti, ORE, and UAC. The objective of the network was to breed and disseminate suitable corn varieties throughout the country. To test the agronomic performance of 29 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV the MP1 variety, a very productive and very tolerant corn variety that CIMMYT introduced in 2016, the network set up multi-site trials in the different agricultural regions of the country. This research work is still ongoing at CHIBAS, Bas Boen and ORE, according to university representatives. AREA supported the publication of research results in several peer-reviewed journals. For example, the Climate Smart Solutions team’s manuscript “Coping with Climatic Shocks: Local Perspectives from Haiti's Rural Mountain Regions” was published in the peer-reviewed journal Population and Environment. The team of scientists from the United States and Haiti also completed a second article, “Farmers Trained in PICSA Report Improved Farming Practices in Haiti,” published in the peer-reviewed Journal of Rural Studies.24 The results of the collaboration with the MARNDR have also been published in two peer-reviewed journals. See Annex F for a list of publications of international and local AREA￾supported researchers. AREA’s strategies to reach national actors were less successful than its strategies for the international scientific community. According to representatives from the IP and USAID, by supporting publication in peer-reviewed journals and having researchers participate in national and international conferences, AREA has disseminated its results to the local and international scientific community. However, according to some representatives of the IP and universities, farmers, who are the primary users of the research results, have not been reached in the same way. In addition, while AREA worked with CRDDs and other local partners to disseminate some of the results, its coverage in the field was limited. Only a few farmers benefited from these extension activities because, according to some stakeholder groups such as the IP and USAID, AREA’s primary objective was not to conduct agricultural extension in the field. The results achieved with AREA-supported research were intended to reach producers through Haiti’s extension services. Since AREA did not have plans to reach all producers in the intervention areas, the activity did not emphasize this aspect. Similarly, according to KIIs with CRDDs and GoH, extension activities are neglected in the agricultural sector, which may have influenced AREA’s performance in its dissemination effort. For instance, GoH structures providing extension services are not functioning well, according to many GoH respondents, and KII participants from universities reported that universities in Haiti are not generally involved in extension activities. Some GoH respondents felt that AREA did not do enough to target local audiences such as local scientists and farmers, probably because of a lack of local scientific journals or other appropriate means of dissemination. The dissemination activities undertaken by AREA, CRDDs, and other local partners with farmers who were directly involved in the research activities did have an impact. Indeed, local partners and farmers who took part in the PICSA experiment reported having changed their behaviors. The article “Farmer￾led participatory extension leads Haitian farmers to anticipate climate-related risks and adjust livelihood strategies” (Staub et al. 2020) reported that “seventy percent of survey respondents [63 percent of whom were women] said that training in PICSA led them to make positive changes in their farming and livelihood enterprises. While most of the changes pertained to the management of existing farming enterprises, farmers also made changes in scale and started new enterprises.” Participants in this evaluation were directly involved in the research itself. Farmers who were expected to receive the training from those directly engaged in the research were not included in the sample. It is worth noting this because the reader can expect to see an improvement in the food security status of farmers in the 24 The following link can be used to access the publication: https://www.sciencedirect.com/science/article/pii/S0743016720301431. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 30 same (or neighboring) communities as those who participated in the experiment. That did not happen because AREA did not plan to reach those. The results can be tangible, but they are still experimental. Following the dissemination activities, representatives from the IP, the CRDDs, and universities indicated that Bas Boen CRDD and UNIQ are multiplying promoted-bean variety seeds for wider dissemination. In addition, according to representatives from the university groups, American University of Les Cayes, UNIQ, Episcopale University, ORE, and Bas Boen CRDD were multiplying seeds and conducting more research on the promoted corn variety. KII participants from universities also mentioned that the students at FAMV were continuing with promoting research for their undergraduate theses. Although AREA engaged significant efforts in extension activities, at the time of this evaluation, the Haiti extension system is still not functional, according to representatives of the GoH and the university groups. It was nonetheless noted by a representative from USAID and a representative from the IP that AREA’s objective was not to fill all the gaps that existed in the extension system. Therefore, the dissemination services provided had limited coverage. CONCLUSIONS AREA had many research programs that generated mixed results. AREA disseminated those results directly in its extension activities for farmers, during workshops, and in peer-reviewed journals. AREA extensively trained researchers, and local partner staff to increase their knowledge about the results of its research and strengthen their capacity to use them. Local partners, universities, and research centers that took part in the activity interventions engaged research results through seed multiplication, extending newly acquired knowledge to other parts of Haiti, and reaching out to other researchers in national and international networks. Farmers directly involved in the experiments noted an increase in their knowledge and the benefits of the promoted technologies. Although AREA and their partners engaged in extension activities, the coverage of local audiences such as farmers groups was still low since very few farmers apart from those directly involved in the experiment and training were reached via its interventions. EQ 3: ON THE BASIS OF AREA’S APPLIED RESEARCH AND EXPERIMENTATION, WHAT AGRICULTURAL EXTENSION MODELS (OR ASPECTS THEREOF) APPEAR TO BE THE MOST PROMISING AND WHY? AREA conducted an experiment from 2017 to 2019 with Haitian farmers to assess what agricultural extension models lead to better technologies’ adoption and identify additional factors explaining why a model would be the most promising in the Haitian context. However, at the time of this evaluation AREA researchers were still analyzing the experiment’s data and preparing to publish the results in several journal articles, according to the IPs. Therefore, to answer the evaluation question, the ET relied primarily on KIIs with the researchers and FGDs with farmers involved in the experiment. The sections below describe the experiment and indicate key results about the extension models’ performance. AN EXPERIMENTAL DESIGN TO CAPTURE THE EFFECT OF SEVERAL FACTORS ON ADOPTION AREA implemented a research-based approach to identify effective extension models or approaches to technical assistance for different groups of farmers, including women farmers, which are essential to designing an equitable, sustainable, and pluralistic system that can increase technology adoption rates. 31 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV The research involved thousands of farmers in the Kenscoff region in a large-scale research project to evaluate three extension models and determine the best ways to transfer new technologies, tools, and high-yield varieties of crops to Haitian farmers. Three extension models were tested using seven promising technologies. The three extension models were: 1. Master farmer (MF), which centers on an expert who teaches farmers in a lecture-style setting. 2. Farmer field school (FFS), which primarily relies on the community of farmers to teach each other; and 3. Simple distribution, or input supply-driven (ISD), in which farmers come to a central location to receive the technologies and basic information on their use. The technologies tested in the different extension models are described in Table 4. TABLE 4. TECHNOLOGIES TESTED BY AREA WITH THE EXTENSION MODELS EXPERIMENT TECHNOLOGIES BRIEF DESCRIPTION 1 Pre-germination test A technique used to evaluate seed quality 2 Germination test With these tests, farmers can monitor the quality and the germination rate of seeds to determine whether they are viable 3 Biodegradable pots These pots allow germination to occur before transplanting the seedlings to the plot 4 Light hoe Lighter than the traditional hoe used by Haitian farmers to remove weeds 5 Kale seeds Fast-growing leaves high in nutritional value 6 Mustard seeds Fast-growing leaves high in nutritional value 7 Purdue Improved Storage Bags Three-layered bags designed to store and protect grains and seeds without chemicals, reducing post-harvest losses from common pests and insects The experiment’s primary objectives were to: 1. Evaluate the differences between three commonly used extension models in Haiti. 2. Assess the interactions between extension models and farmer associations as predictors of farmers’ willingness to test innovations on their farms. 3. Determine whether there is a relationship between the gender of the farmer, the traits of farmer associations, and the effectiveness of the extension model. The experiment featured a factorial quasi-experimental design with multiple post-tests and two factor: ● Factor 1: Three extension models (MF, FFS, and ISD) ● Factor 2: Three farmers’ group types (women, moderate resources, and low resources). All seven technologies were applied to all experimental units. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 32 TABLE 5. DISTRIBUTION OF THE EXPERIMENT FEATURES FACTOR 2: FARMER GROUP TYPE FACTOR 1: TREATMENT – EXTENSION MODEL MASTER FARMER (WINNER) FARMER FIELD SCHOOL (FAO) INPUT SUPPLY DRIVEN (MARNDR, NGO) Women’s Group 7 Technologies (7T) 7T 7T Moderate resources 7T 7T 7T Low Resources 7T 7T 7T Interaction between extension models and farmers group type. Farmer group type, which is the factor expressing the strength of the farmers’ association, is driving technology adoption rates, regardless of the extension model (KII with USAID and IPs, FGD with master farmers). According to AREA researchers, experimental data regarding the extension models’ performance in terms of rate of farmer adoption of innovations are still confidential. However, they indicated that while the MF model provided the highest score in farmers’ adoption of the promoted technologies, the data do not allow direct comparison because of interaction between extension models and farmers’ group types. Farmers’ group type or strength of farmers association plays a significant role in affecting the models’ performance. In other words, the ranking of the models varies according to the strength of the associations. With low-resource associations, performance ranked in decreasing order is (1) FFS, (2) MF, and (3) ISD, while the ranking becomes (1) MF, (2) FFS, and (3) ISD when working with farmers belonging to moderate resources associations. According to FGDs and KIIs with AREA staff, both MF and FFS models appeared to lead to farmers’ adoption of technologies in the experimental conditions, outperforming the ISD model. Farmers who participated in the FGDs for both MF and FFS models agreed that the extension model they embarked on was adapted to their needs. In all three FGDs with farmers of the MF model, they indicated that MF is the best model. Likewise, farmers of the FFS model said that FFS is the best model. However, none of the farmers of the ISD model claimed in FGDs that ISD is the best model. In all three sets of FGDs, farmers said they would continue to use the technologies only if the inputs/materials were still available. As of the time of the evaluation, they could not find the inputs/materials in local markets to continue using the technologies that AREA promoted during the experiment. According to KIIs with IPs, the MF and FFS models carry a minimal level of the structural support needed to more quickly facilitate technology dissemination. IPs suggested that associations with moderate resources seem to work in synergy with MFs to provide better support to farmers. In settings where low-resources associations prevail, the FFS model seems to work better, demanding extra effort and time from farmers but providing better support to those farmers to make informed decisions. According to IPs, FFS gives outstanding results if and only if the farmers live next to each other. According to farmers, the proximity between farmers facilitates inter-farms visits, and FFS can be time￾consuming. This extra time seems required since FFS outperformed the other models with farmers belonging to associations with low resources. The MF model generally gives the best results in locations where farmers’ associations with moderate resources exist, and the farmers have confidence in the associations’ leadership. Leaders of moderate-resources farmers’ associations seem to work 33 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV synergistically with the MFs to support farmers, helping them make informed decisions, thus increasing adoption rates. Other factors affecting farmers’ behavior change. AREA indicated in its 2016 annual report that the extension models can affect the enabling, reinforcing, and predisposing factors affecting technology adoption. Farmers who participated in FGDs also mentioned special situations influencing the models’ effectiveness. The factors AREA identified are used below as a framework to classify conditions that the stakeholders identified as influential to technology adoption. According to KIIs with USAID and local institutions, as well as an FGD with FFS farmers, the perceived benefits associated with the use of the seven technologies (e.g., user-friendly, low cost, less time￾consuming, market-driven, and high return on investment) affect farmers’ adoption of these technologies. These benefits can be classified into the following three categories: 1. Enabling factors are the conditions that either foster or inhibit an individual’s ability to engage in a behavior, regardless of whether s/he concludes that the benefits of doing so would outweigh the costs. User-friendly, less time-consuming or low-cost technologies can be enabling factors. For instance, farmers in FGDs across all models cited the availability of inputs or water as factors preventing the use of technologies such as seeds for leafy green vegetables. 2. Reinforcing factors are conditions that support a behavior. Associations with moderate resources could provide additional assistance to farmers and could be considered a reinforcing factor. High return on investment technologies such as the PICS bags cited by farmers in FGDs can also be a reinforcing factor. Non-adaptation of a technology such as the lighter hoe is a reinforcing factor supporting widespread non-adoption of the technology, regardless of the model or the strength of associations. 3. Predisposing factors are the pre-existing knowledge and perceptions that all of us bring to any consideration of behavioral change. Providing new knowledge may or may not change established perceptions but is required for any behavioral change model. KIIs with local institutions noted that farmers are slow to change behavior or adopt new technologies. If they are accustomed to an intense MF interaction before adopting technologies in prior situations, they may not be convinced to adopt. For example, according to KIIs with IPs, most of the farmers in the Kenskoff region have been exposed to the MF model in a prior USAID project, and they might be reluctant to adopt if they are only exposed to the technologies during the AREA experiment. According to the farmers in the FGDs, farmers will pick up new technologies if they perceive potential benefits, such as higher production, more revenues, and a market. The opposite also holds true: for instance, farmers in FGDs reported that the light hoe was not adopted across all extension models because it is not suited for the type of soil in the region. CONCLUSIONS Considering the experimental condition, ceteris paribus, the MF and FFS models triggered farmers’ utilization of technologies in the Kenscoff region. Farmers appreciated MF and FFS, saying they met their need for information to make informed decisions about adopting new technology. FFS and MF brought the information and support that the farmers needed to adopt the technology, and such support was reinforced by the activities of the farmers’ association. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 34 A strong association contributed significantly to triggering farmers’ utilization of technologies, this support being more efficient with the MF model. In other words, the performance of the models varied with association strength in terms of adoption rate. The association provided much-needed additional follow-up and support for farmers to adopt new technologies. The association leaders worked in synergy with the MFs to support utilization, although the ET could not determine the magnitude of this support. Ultimately, as shared by FGD participants, farmers’ behaviors towards the new technologies were dictated by the perceived benefits they potentially yield. EQ 4: TO WHAT EXTENT AND IN WHAT WAYS DID AREA’S INTERVENTIONS AND TRAINING (BOTH LONG AND SHORT TERM) INCREASE/IMPROVE TRAINEES’ ABILITY TO CONDUCT APPLIED AGRICULTURAL RESEARCH AND PROMOTE EFFECTIVE AGRICULTURAL EXTENSION ACTIVITIES? AREA supported the training of scientists at the undergraduate level. According to quarterly reports, AREA partnered with seven agricultural colleges across Haiti to develop an innovative higher education program to better prepare the next generation of university undergraduate students who can improve Haiti’s agricultural sector. Most of the students are now in the job market. In its 2018 annual report, AREA indicated that faculty at four partner universities implemented seven curriculum changes following their participation in AREA’s higher education program. For example, an FAMV faculty member changed her syllabus to better define the objectives and improve the pedagogy of her introductory to chemistry course, and faculty members at the American University of the Caribbean (AUC) improved the syllabi of two courses—one class is the introduction to agriculture, the other entomology—by adding sections on learning objectives and course policies. In FGDs with trainees, they indicated that they appreciated several trainings that AREA conducted, including training about agricultural extension, climate change, characterization and fertilization of soil, food technology, soil improvement, agricultural economics, gender, and social inclusion. An FGD participant said, “training on climate change, management, and soil fertility proved to be necessary and very useful, especially with regard to the working conditions of farmers (rudimentary tools), the topography of the areas (mountain), the different types of crops (rooted crops) and fallow management.” The participant added that “as young agronomists, they would be delighted to replicate all the trainings to farmers as extension agents.” However, these FGD participants suggested that some trainings were too short. AREA supported training of scientists at the graduate level. The activity also worked with land grant universities in the United States to provide master’s degree scholarships to future agricultural leaders. AREA funded 25 Haitian graduate students at UF and LSU to build a cadre of agriculture professionals to contribute to the sector’s development. Most of them conducted their research in Haiti, addressing country-specific agricultural challenges. According to AREA staff, all young professionals returned to Haiti initially to work as government officials, researchers, or in the private sector, but some returned to the United States to complete a Ph.D. They received hands-on training while doing research during their master’s degree and addressed real problems facing agriculture in Haiti. According to the reports, AREA-supported graduates became experts in specific fields in the agriculture sector and have published papers in several peer-reviewed journals. AREA hired some of them after their graduation (based on the Activity’s interest in their research topics) to participate in the research programs that AREA was implementing in Haiti. 35 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV However, KIIs participants from USAID, IPs, and universities noted that AREA lacked a plan for connecting these scholarship students with professional jobs after graduating. Upon completion of the program, all the recipients came back to Haiti, then after a while, most of them returned to universities abroad, pursuing a Ph.D. degree. University representatives indicated in KIIs that some of the graduates did not want to accept offers because the university salaries were too low. Some KII participants from universities expressed that there is a lack of qualified human resources to conduct research in their institution and still hope that the Ph.D. candidates will come back after receiving their degrees. A university KII participated recognized that “it's a good thing that [a student] can go up to the doctorate level because to do research, you have to have people who have the doctorate level,” and noted that a Master’s degree is not enough to tackle all the challenges researchers could face while conducting research in the agricultural sector. However, other university representatives who participated in KIIs, thought that the universities suffer from resources for both teaching and research, and the graduate students should remain at local universities to provide these services. AREA supported short-term training in a variety of fields. According to AREA program reports, the activity trained nearly 2,000 students, agricultural professionals, and farmers in limiting postharvest losses, food safety, conducting agriculture research and extension, and pest management and safety. For each research program that AREA designed, the activity implemented training to transfer skills and competencies related to the research program. Trainings were tailored to meet the needs of specific audiences. The ET identified the short-term training topics by research program in project documents, as summarized in Table 6 below. TABLE 6. TRAINING SESSIONS PER INTERVENTION (RESEARCH PROGRAMS) RESEARCH PROGRAM IMPLEMENTER PARTNERS TRAININGS BENEFICIARIES SOURCE Soil management AREA local staff FAMV Bas Boen CRDD Soil fertility management University students, professionals, civil society Q2Y3 report Q4Y3 report Post-harvest and food safety AREA local staff - Hazard analysis, critical control plan, post￾harvest loss management Q3Y3 report Climate Smart Solutions AREA, University of Reading - Analysis of historical climate data Haitian data analyst Q3Y3 report Plant pathology AREA, UF - Plant disease diagnostic, microcosm care, R MoA, CRDDs Q3Y3 report Q2Y4 report Maize Seed System CIMMYT, AREA CRDDs (Bas boen, Montrouis, Duvier) Corn seed production, standard experimental procedure Farmers, extension agents, CRDD staffs, MARNDR, private sector staff Q4Y3 report Gender assessment and intervention Cultural Practices - Gender disparities in program design Q1Y4 report AREA reported that several Haitian scholars benefited from a short-term online one-year certificate degree program through AREA. Beneficiaries also acquired skills and knowledge to better tackle the USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 36 agriculture problem, increasing their competencies in conducting applied agricultural research, according to AREA’s 2019 annual report. The ET’s survey of trainees asked respondents to indicate the type of training they received. Figure 6 below shows these responses. FIGURE 6. TYPES OF TRAINING RECEIVED BY RESPONDENTS25 In the sample, relatively more trainees participated in the legume breeding and nutrition program (41 percent), the plant pathology program (31 percent), climate change (30 percent), soil management program (27 percent), and maize seed production and management (16 percent). More details about these trainings and related outcomes are below. LEGUME BREEDING AND NUTRITION TRAINING AREA promoted local scientists’ training and capacity building as interns through the legume breeding program. In addition, AREA promoted capacity building in plant breeding and improvement by hiring two interns to support the Legume Breeding Program. The AREA FY2018 annual report noted that AREA built the capacity of these interns in various aspects of plant breeding and improvement, including genomic selection, marker-assisted breeding, and other state-of-the-art technologies. AREA training also built capacity at the CRDD level, according to CRDD KIIs. KII participants noted that at Bas Boen CRDD, CRDD staff, along with the University of Puerto Rico, are conducting trials on new bean varieties to select varieties resistant to high heat, drought, and pests, including insects and diseases. In partnership with Gollub under the legume program, trainings were conducted on household nutrition and the nutritional benefits of Asian spinach. The key audience of these trainings was female members of farmer organizations in the FtF-West region. AREA introduced Asian spinach and provided cooking 17 4 41 31 30 27 16 2 1 1 1 1 0 5 10 15 20 25 30 35 40 45 Other, please specify No answer/Don t know legume breeding / Nutrition Plant pathology Climate change Soil science Maize seeds Extension Agronomy Teachings / Methodology Marketing Management Type of training respondents received (%) 25 Names used in the survey to identify the trainings are more related to the name of the research programs, regrouping trainings that are alike. AREA used technical terms to differentiate trainings within programs that respondents tend to forget. 37 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV demonstrations to 132 women farmers and gardeners during nutrition training held at Haiti's two locations (AREA annual report, 2019). The ET observed that Asian spinach can still be seen in some backyards in the FTF-West region. PLANT PATHOLOGY TRAINING (PARTNERING MAINLY WITH CRDD AND MARNDR) AREA worked with CRDDs as a partner through several programs, including the plant pathology program. KIIs with CRDD staff indicated that through this partnership, they participated in many training courses, including training on writing and scientific rigor; training on professional ethics; training on taking data on the plant material tested; training on the use of data processing software, for example, R software; and training at UI on setting up trials. They also participated in research on maize, beans, and soil fertility, together with local universities, including the FAMV, UNIQ, the UNEPH, and the American University of Les Cayes. Many tests took place at CRDDs. As part of these trials, several undergraduate students in agricultural sciences could complete their final theses. According to KIIs with representatives of the CRDD and GoH groups, AREA provided technical training support for CRDD and Ministry of Agriculture personnel under the Plant Pathology Program, focusing on laboratory operations, isolation techniques, and identification and diagnosis of major plant pathogens. For instance, according to its 2019 annual report, AREA provided support to improve the capacity of the Bas Boën CRDD to operate and maintain a functional plant disease diagnosis laboratory. This has been accomplished through staff training, improving the plant pathology lab area, creating an isolation room, and supporting a lab technician. The project also provided the soil laboratory with reagents and equipment, particularly a flame photometer. FGDs indicated that AREA promoted local scientists’ training and capacity building, but laboratories are still under-equipped. For instance, in the Phytopathology Research Program, AREA selected four candidates for a 4–6 month internship. Trainings were conducted for laboratory staff in several universities and at CRDDs. However, in FGDs, trainees claimed they still lack materials and equipment to conduct research. This is the case at the direction of plant protection at the MARNDR and the FAMV soil laboratory. AREA established a fully equipped mycotoxin unit at FAMV and trained laboratory workers to use the equipment and provide services. Materials and trained scientists are still at FAMV at the time of data collection. Still, demand for services is low due to Haiti's socioeconomic, political, and sanitary situation, according to a university representative. CLIMATE CHANGE TRAINING (PARTNERING WITH MARNDR/UHM, NGOS) According to several KIIs with local institutions, PICSA training was a success and was appreciated by farmers. AREA leveraged other institutions established in Haiti to disseminate knowledge about PICSA. AREA partnered with UHM to disseminate information relating to climate data because farmers generally did not know how to use the data. AREA had workshops with farmers for climate solutions in agriculture in its extension program. AREA partnered with CRS to conduct training about PICSA in south Haiti. UHM has benefited from other types of training on installation and assembly of devices, both in Haiti and Miami, and is better positioned to generate data and provide information to farmers, according to GoH representatives. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 38 SOIL MANAGEMENT TRAINING (PARTNERING MAINLY WITH CRDDS/FAMV/UNIQ) According to IPs, AREA disseminated soil management technological training packages, including advice on soil preparation and fertilization operations to improve the yields of certain crops, particularly beans and maize. Experimentation and demonstration plots were set up in the communes of Croix-des￾Bouquets, Cabaret, and Kenscoff, associated with training seminars and field days for farmers. Sometimes trainings lasted up to a week and a half, and farmers learned a lot, according to IPs. Farmers received quality seeds and equipment, allowing them to set up test plots, and they were all supervised by the agricultural extension section of the project. AREA worked with UNIQ and the FAMV, using their soil laboratories for some analyses; however, due to a lack of laboratory capacity from these universities, some of the samples were sent to UF for analysis, according to KIIs with IPs and universities, because local laboratories often face problems with reagent availability (they place orders in the Dominican Republic, and the waiting time is very long). To have more reliable and timely results, AREA had to send samples to Florida for analysis, said one IP. In addition, some analyses cannot be carried out locally owing to a lack of equipment. IPs and universities indicated that human resources were trained, with training sessions conducted abroad for laboratory personnel. However, some equipment is lacking to address some of the agriculture problems in Haiti, and much remains to be done in the area of research in the country. MAIZE SEED PRODUCTION AND MANAGEMENT TRAINING According to universities representatives and IPs, AREA partnered with AUC and CIMMYT to conduct several trainings on the production of maize seeds and varietal improvement both in Mexico and Haiti. MARNDR staff and private sector and CRDD representatives also participated in maize training in the United States, according to CRDD and GoH representatives. University partners indicated that as part of its collaboration with AREA, AUC’s Plant and Soil Protection laboratories received sophisticated equipment for analysis. The equipment includes microscopes of different types, drying ovens, equipment for collecting data in the open field, including phenotyping meters for data on plants’ photosynthetic activities. These kits allow tests to be carried out very fast. The project performed short-term training (3 to 5 days) for laboratory work staff. The first training was based on microscopy and the second on plant disease diagnostic steps. Respondents in KIIs indicated that it was practical training intended for both laboratory staff and students. An undergraduate plant pathology class at AUC was upgraded to include a practicum, improving a new generation of students’ capacity to conduct agricultural research. The trainings were relevant. The survey probed the extent to which training was relevant to key stakeholders and institutions. To measure this, the ET collected information about not only the respondent’s perceptions about the relevance of the training but also the extent to which respondents think that that the training was related to their job and that they had hands-on training enabling them to apply new skills and knowledge effectively. 39 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV 70 75 86 28 0 20 40 60 80 100 Said they had hand-on training Agreed that the training was relevant Claimed the training was related to their work Heard about the theme before the training Training relevancy (%) FIGURE 7. RELEVANCE OF THE TRAINING RECEIVED BY RESPONDENTS The training was relevant to their jobs, according to 75 percent of the respondents. The relevance of the training stands out because 86 percent of the respondents claimed that the training was related to their work. It stands out even more because, despite the relevance of the theme to their work, only 28 percent claimed that they had heard about the theme before the training. A comparison between the percentages of respondents who think that the training is related to their jobs (86 percentage) and the percentage who had never heard of the theme before the training (28 percent) indicates how innovative the themes were. According to university and IP KIIs, considering the weakness of extension services in Haiti (see EQ2 findings above), AREA conducted specific training about extension, and GoH representatives were invited to participate, with the hope of seeing more extension services activities in the country. A striking example is a respondent’s testimony about the training on extension service: “For extension activities, I had participated in a very well-developed training which enabled me to develop agricultural extension programs. We were already in research and extension before the AREA project. After the project we continue our work, we have trials on bean and maize varieties, and we are also working in seed multiplication.” While some key informants from universities and GoH and survey respondents thought that the trainings were relevant and AREA provided trainees with adequate skills for their jobs, trainees who participated in FGDs believed that trainings were too short. The trainings were useful. The ET asked trainees several questions regarding the usefulness of the training. The questions pertained to their perception of the benefits of the skills they learned and their effective use. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 40 69 94 97 44 57 0 20 40 60 80 100 120 Claimed that supervisors asked them to use new knowledge Claimed they have used what they learned at work Think they learned something useful Had new task requiring going back to training materials Could enumerate two new skills that they used after the training Training usefulness (%) FIGURE 8. TRAINING USEFULNESS Nearly all the respondents thought that the trainings were useful and almost all them claimed to have been using what they learned at work. In fact, of 462 respondents, 447, or 97 percent, thought they learned something useful, and 433, or 94 percent, claimed they have used what they learned from the training at work. The ET inquired about the need for the new knowledge and skills at work by asking about supervisors’ requests as an outcome of the trainings. Some 321, or 69 percent, indicated that supervisors asked them to use the newly acquired knowledge, and 205, or 44 percent, had new tasks that required going back to the training materials. The survey asked respondents to list two new skills they acquired and used from the training. The majority of the respondents, 262, or 57 percent, could enumerate two new skills that they used after the training. This indicator speaks both to the usefulness of the training and the novelty of the themes and the level of capacity improvement at the workplace that AREA achieved. The trainings triggered behavior change. To have a distinct approximation of the trainings’ contribution to the trainees’ capacity to conduct research and promote effective extension activities, the ET also surveyed trainees on changes in their behaviors as a result of the training. 41 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV 24 89 94 98 92 0 20 40 60 80 100 120 Would hire someone without these skills for the job Think the new skills are required for the job Agreed that they are better researcher after the training Would recomment the training to peers Claimed to do things differently at work after the training Trainings effect on behavior (%) FIGURE 9. EFFECT OF THE TRAINING ON BEHAVIOR CHANGE High impact trainings trigger behavior changes. The ET measured to what extent AREA trainings pushed trainees to do things differently at work. A very high percentage of the 462 respondents, 426, or 92 percent, claimed to do things differently at work after the training. Among respondents, 89 percent thought that the new skills are required for their jobs, and only 24 percent claimed that they would hire someone without these skills for the job. About 94 percent of the trainees agreed that they are better researchers due to the training, and 453 respondents, or 98 percent, would recommend the training to peers. Social turmoil and security issues affected training-related AREA activities. In its FY2020 quarterly report, AREA reported that a plant pathology lab technician collaborated with an AREA master’s degree student to train farmers in Montrouis and students and faculty at the American University of the Caribbean in Les Cayes, but that the training was ultimately canceled due to civil unrest. In February of the same year, AREA reported that the project suspended interventions temporarily at the Bas Boën CRDD plant pathology laboratory because of increasing security issues, particularly in La Tremblay. AREA postponed two trainings on nematodes diagnostics that an AREA￾funded graduate student and the AREA plant pathology technician had scheduled for February 2020 at the American University in Les Cayes and at the Montrouis CRDD. KIIs with CRDDs confirmed the above claims, and the situation at the La Tremblay continued through the data collection period. CONCLUSIONS AREA performed well in identifying relevant topics and identifying appropriate recipients to the short￾term training, which also appears to have increased trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities in Haiti. The training proved helpful in addressing the needs for better skills at research institutions and triggered trainees’ behavior change. The topics were thoughtfully chosen and deemed relevant according to trainees. In some cases where farmers were involved, they perceived the training to be too short. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 42 AREA trained graduate students to address the lack of human resources in local research institutions. While some of them were able to secure a contract with AREA (two out of twelve surveyed students), other had returned to a university abroad to further pursue higher education. For several reasons, many long-term trainees did not end up joining the local institutions. AREA provided some equipment at local laboratories and trained technicians on their use. But laboratories at local institutions are still under-equipped: At the Activity’s end, assisted laboratories still did not have the equipment necessary to support research on critical issues, such as low crop yields for many crops, the impact of environmental deterioration on production systems, or the low adoption of recommended agricultural innovations in Haiti agriculture. LESSONS LEARNED The following table summarizes what worked well and what did not work well in AREA’s interventions: TABLE 7. LESSONS LEARNED WHAT WORKED WELL WHAT DID NOT WORK WELL EQ 1 – AREA SUPPORT TO APPLIED AGRICULTURAL RESEARCH Mini-grants to support ad hoc research in agricultural value chains had some impacts. According to many stakeholders, neither the government nor the private sector invests enough. The support provided by AREA to local researchers through the provision of mini-grants was a success, looking at all the ad hoc solutions that those researchers identified. Provision of equipment to laboratories and research institutes: Haiti's agricultural laboratories are under￾equipped. The provision of lab equipment is a foundation for successful research in the sector. AREA distributed essential lab equipment that included microscopes to detect the most common plant diseases. Support to laboratories through their equipment was a good choice. Local laboratories, universities, and research institutes still need equipment and additional support. Under EQ 1, the ET noted that the equipment provided to laboratories was essential but technology gaps to perform adequate lab testing still exist. Future support could focus on equipping one reference lab at least to run those tests locally if need be. Enlisting public and private support to AREA activities was not a success. AREA did not attract more government and private-sector funding in applied agricultural research and training. The lack of an exit strategy is not a good practice. Apart from the routine research undertaken by universities and some local partners, most of the interventions AREA supported have ceased after the funding stopped. AREA did not have an exit strategy. EQ 2 – DISSEMINATION OF AREA KNOWLEDGE PRODUCTS Provision of support to publish scientific papers and articles: AREA succeeded in supporting the publication of scientific articles in peer-reviewed journals. Interviewees from the IP, universities, USAID, and CRDDs agreed that the body of knowledge generated through AREA is tremendous and believe it will contribute to further developing the sector. 43 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV WHAT WORKED WELL WHAT DID NOT WORK WELL EQ 3 – EXTENSION MODELS IN THE AGRICULTURAL SECTOR The MF and FFS models fit well with farmers’ needs. Farmers who participated in the extension models experiments recognized that the MF and FFS models, as opposed to the ISD model, were adapted to Kenscoff region The ISD model was not as effective as the other models. Research on the extension models showed that the ISD model, which relied only on exposure to technology, was not adapted to the needs of the farmers in the Kenscoff region. The major outcome of the AREA experiment was that the strength of the associations influences the performance of the models. Strong association leaders influence farmers’ decisions to adopt sound technologies, regardless of the model. Higher synergy takes place with the MF model. The availability of inputs for farmers to continue to use the technologies is critical and was disrupted at the end of the project. While AREA’s main objective was not technology dissemination, effective implementation of the AREA’s proposed approaches was grounded in the availability of certain promoted technologies. Irrespective of the technology, the availability of inputs in local markets is key to adoption on a broader scale. EQ4 – AREA SUPPORT TO TRAINING OF AGRICULTURAL STAKEHOLDERS Short-term trainings did help improve trainees’ capacities. Trainees from all stakeholders’ groups agreed that short-term training helped improve their capacity and provided them with the needed skills to perform their duties better. Support to the sector through short￾term training of agricultural professionals appears to be an excellent option. Sending graduate students overseas is not the best option to immediately improve the sector within Haiti. An alternative option could have been to reinforce the local universities graduates’ programs to provide the same type of degrees locally. RECOMMENDATIONS To USAID, the evaluation recommends to: 1. Provide ad-hoc institutional support to local institutions and universities that agree to contract the trainees for at least a year following their studies. 2. Strengthen local universities sustainably by supporting their graduate program and research activities, developing adequate training curricula, and upgrading laboratories in local universities. 3. Further, support the equipment of laboratories at local research institutions to address critical issues in Haitian agriculture. 4. Ensure that a plan to disseminate research results to local audiences, not just the scientific community, is rolled out for future similar programs. 5. Support the implementation of an exit strategy for future research projects to ensure that research continues after the funding ends and that relevant input (technology) is available to those who want to utilize and later adopt the technology, even if they are a second or third-line actor. 6. Support the utilization of the MF and FFS models in the Kenscoff region to disseminate agricultural technologies and work with Haitian extension services to determine the best extension model for the other regions. 7. Strengthen farmers' associations to enable them to play a more substantial role in extension activities USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 44 ANNEXES ANNEX A. STATEMENT OF WORK STATEMENT OF WORK Performance Evaluation of Feed the Future Appui a la Recherche et au Developpement Agricole (AREA) I. PURPOSE OF THE EVALUATION The purpose of the evaluation is to inform USAID/Haiti on potential future similar agriculture programming in Haiti. Specifically, USAID/Haiti seeks information on how to better engage higher education institutions in agriculture, how to support a local system to conduct applied research and how to support an extension system that would contribute to a more productive agricultural sector. To provide this information, the evaluation should shed light on the extent to which the activity approach to building capacity in applied research was successful in increasing the effectiveness of the research community and farm productivity as well as the extent to which the research-based approach was effective at identifying extension models that contribute to a more productive agriculture sector. Lessons learned and recommendations provided through the final report at the end of the evaluation process should indicate the long-term impacts, including activities that are deemed to be sustainable, identify existing gaps in the area of long-term capacity building in the agriculture sector. The primary stakeholders for this evaluation include: USAID/Haiti, University of Florida, Louisiana State University, University of Illinois, local universities and other institutions which participated in the implementation of the project, and the Ministry of Agriculture. II. SUMMARY INFORMATION Strategy/Project/Activity Name Feed the Future Appui a la Recherche et au Developpement Agricole Implementer University of Florida Cooperative Agreement/Contract # AID-OAA-A-15-00039 Total Estimated Ceiling of the Evaluated Project/Activity(TEC) 13.7 Million Life of Strategy, Project, or Activity May 2015-December 2020 45 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV Active Geographic Regions AREA’s target area includes Kenscoff, Cul-de- Sac, and Matheux corridors. Capacity building activities had a national coverage Development Objective(s) (DOs) Food and Economic Security USAID Office EGAD III. BACKGROUND The goal of AREA was to build the capacity of public and private institutions and organizations to provide results-oriented agricultural research and extension programs that are inclusive of all farmers, promote gender equity, and are appropriate for Haiti. AREA improved and expanded existing country￾driven approaches to address food insecurity and undernutrition in Haiti. By strengthening and supporting Haitian public and private institutions operating in the agricultural sector, the project aimed to design, implement, and evaluate research and extension activities that support the agricultural sector and pave a path to self-reliance. In this regard, AREA funded 25 Haitian graduate students at the University of Florida and Louisiana State University to build a cadre of agriculture professionals to contribute to the sector’s development. All young professionals returned to Haiti and now work in the sector as government officials, researchers, or in the private sector. AREA also funded research activities to develop high-yielding varieties of key food security crops, identified the most effective way to introduce new production technologies, and ways to improve soil fertility in the Haitian context. Recognizing the critical role women play in the agricultural system, the project incorporated women into activities and developed new avenues for women’s participation. Lastly, AREA partnered with local universities and research centers to conduct research on corn, bean, and other critical food crops to increase yields. The research components also covered phytopathology, climate modeling, soil management, and postharvest technologies. A. DESCRIPTION OF THE PROBLEM, DEVELOPMENT HYPOTHESIS(ES), AND THEORY OF CHANGE Food insecurity has been a long-standing challenge in Haiti, exacerbated by natural disasters, a lack of resources to increase farm productivity and a myriad of other factors. For example, yields of critical food security crops such as the common bean falls short of the country’s yield potential, partly because traditional varieties are not optimally adapted for Haiti’s diverse agroecological zones. Plant diseases and pests cause significant crop losses, yet Haiti has few laboratories to diagnose and address them. The adoption of new technologies, such as improved seeds and production methods, suffers from ineffective or limited formal agricultural extension and technology transfer programs. Under the Feed the Future (FTF) goal to “sustainably reduce poverty and hunger,” the AREA program contributed to the objective of “Improved Agricultural Productivity” through Intermediate Result 1 “Improved Agricultural Sector Productivity” (see Figure 1 and Table 1 below for the indicators and activities in three major sections): 1. Sub-Intermediate Result 1.1: Enhanced Human and Institutional Capacity Development for Increased 2. Sub-Result 1.2: Enhanced Technology Development, Dissemination, Management, and Innovation USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 46 3. Sub-Result 1.3: Reduce gender disparities in access to, control over and benefit from resources, wealth, opportunities, and services – economic, social, political, and cultural. Figure 1 presents a visual overview of the relationship between the overarching goal of FTF and the target objective, sub-intermediate result, and sub-results of the AREA program. This framework describes the process by which the AREA program will achieve these sub-intermediate results in its area of intervention by providing illustrative activities and the relevant indicators. Further information on program activities can be found in the Work Plans. The Summary of Results and Indicators in Table 1 and Performance Indicator Reference Sheets (PIRS) in Annex 1 of the PMP provide details on the Monitoring and Evaluation (M&E) strategy for these indicators. Summary Strategy/Project/Activity/Intervention to be evaluated. The overall goal of this project is to build the capacity of Haitian institutions to increase the availability of improved production technologies to farmers and the private sector through effective extension and development of an agricultural innovation system. The development of functional and sustainable systems to generate knowledge, create process and product innovation, and extend information to farmers requires building on existing models of success and forging linkages to the national and international efforts 47 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV already in place in Haiti. The project worked to build and extend the capacity of government institutions and private organizations to improve their ability to serve the agricultural sector with research and extension programs that increase productivity in the target region. The project includes three major components: Research: AREA worked to increase the effectiveness of the research community in Haiti by improving the access and availability of scientific knowledge and technological innovations that lead to increased farm productivity. The Project’s three-stage approach to building capacity in applied research leveraged existing resources, increased the number of scientists engaged in agricultural research, and provided research-based diagnostic services to producers. Extension: Area investigated models that have been used to transfer technologies and information to Haitian farmers. The project implemented a research-based approach to identify effective extension models for different groups of farmers, including women farmers, that are essential to designing an equitable, sustainable and pluralistic system that can increase technology adoption rates. Effective extension services play a key role in farmer’s adoption of improved technology, which translates into higher yield and income for the farmers. Capacity Building through Higher Education: The objective of the higher education activity is to increase the capacity of Haiti’s agricultural universities and improve the curricula so graduates can better achieve the needs of the country’s agricultural sector. AREA partnered with seven agricultural colleges across Haiti to develop an innovative higher education program to better prepare the next generation of university students who can improve Haiti’s agricultural sector. The project also worked with Land Grant Universities in the U.S. to provide master’s degree scholarships to future agricultural leaders. B. SUMMARY OF THE PROJECT/ACTIVITY MONITORING, EVALUATION, AND LEARNING (MEL) PLAN This project is based on a strategic framework to enhance and expand existing country-driven approaches to address food insecurity and under-nutrition by strengthening and supporting Haitian public and private institutions that work in the agricultural sector. The major focus is increasing the capacity of the public and private sector to design, execute and evaluate research and extension activities. AREA has defined a set of performance indicators including a mix of outcome and output indicators to monitor project progress at all levels of the results framework. Output indicators track the immediate products of project activities and provide feedback to managers on project performance to identify areas where implementation strategies may need to be adjusted. Outcome indicators measure the effects, or results, of project activities, at the higher levels of the project results framework. Indicators were selected based on the overall strategic approach of the project and closely reflect the work plan to capture the main project activities. Indicators at the project objective and result levels help us monitor progress against targets, capture and communicate impact, provide input for USAID’s reporting needs, and support ongoing improvements. Indicators for the M&E system were selected based on the overall strategic approach to the project and closely reflect the work plan, capturing the main activities of the project, taking into account the required FtF, F and custom indicators. The AREA M&E plan includes 2 Feed the Future (FtF) and F indicators and 9 project level indicators. The Indicator Performance Tracking Table (IPTT) with the name of the indicator, unit of measure, disaggregation, data source, baseline data and targets; and the Performance Indicator Reference Sheets (PIRS) contains name of the indicator, the description of the indicator, justification and management utility of each indicator, the frequency of reporting, plan for data acquisition, data quality issues, and plan for data analysis, review and reporting are also part of the M&E plan. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 48 Quarterly and annual technical reports of the activity and are among other documents that outlines the accomplishment of the project. C- Key Achievements of the project ● Provided scholarships to 25 Haitians to earn Master of Science degrees from UF and LSU in many agricultural disciplines and supported them as they returned to Haiti to apply their new knowledge and skills to improve the agricultural sector and food security in Haiti. ● Involved thousands of farmers in the Kenscoff region in a large-scale research project to evaluate three models of extension and determine the best ways to transfer new technologies, tools and high-yielding varieties of crops to Haitian farmers. ● Developed and provided to local partners four high-yielding lines of beans so they can step up production of seeds for distribution to farmers. ● Upgraded three plant pathology laboratories with microscopes and other enhancements to provide disease diagnosis to farmers. ● Developed and installed five wireless and internet-enabled weather stations at key locations to provide farmers with local meteorological data to better manage their agricultural enterprises. ● Trained nearly 2,000 students, agricultural professionals and farmers in subjects such as limiting postharvest losses, food safety, how to conduct agriculture research and extension, and pest management and safety. ● Trained nearly 100 farmer advisers and three state meteorologists in a new climate training program that resulted in more than 500 farmers receiving critical information and decision-making tools they need to increase their resilience and incomes in the face of more variable weather and climate conditions. ● Launched a Faculty Development Academy at seven higher education institutions in Haiti, which sparked educators to make 27 improvements in curricula and teaching practices. ● Wrote more than 100 technical publications on a wide range of agricultural topics, all aimed at helping researchers, extension providers and farmers improve agriculture and food security. ● Provided funds to 31 university researchers and leaders at partner organizations to enhance their teaching, research and farm advisory service capabilities. ● Tested 588 new technologies, practices and approaches to identify those that have the greatest potential to improve agricultural productivity and food security in Haiti. IV. EVALUATION QUESTIONS 1. To what extent and in what ways did the AREA project improve applied agricultural research at local universities? In answering this question, the ET should examine the immediate effectiveness and long-term sustainability of the approaches AREA used to build capacity in applied research. 2. AREA conducted and facilitated a variety of research, how was this information disseminated, and to what extent and how did various stakeholder’s take-up the research? 3. On the basis of available research and information, what are the key determinants that drive the adoption of new agricultural technologies by Haitian Farmers? 4. To what extent and in what ways did AREA’s interventions and training (both long and short term) increase/improve trainees’ ability to conduct applied agricultural research and promote effective agricultural extension activities? 49 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV V. EVALUATION DESIGN AND METHODOLOGY It is expected that the evaluation team proposes an overall research design to address the four evaluation questions and a plan for collecting and analyzing the data. The evaluation design will be based on a mix￾method, a combination of quantitative and qualitative techniques. Key informant and focus group interviews, quantitative data collection and in-depth review of projects reports are among the techniques that the evaluation team should consider when addressing the evaluation questions. ESS and the evaluation team should also ensure that gender is appropriately and adequately addressed. VI. DELIVERABLES AND REPORTING REQUIREMENTS 1. Evaluation Work plan: Upon receipt of this Activity Request, Social Impact (SI) shall submit within four weeks a draft work plan to the Contracting Officer’s Representative (COR). The work plan will include: (1) the anticipated schedule and logistical arrangements; (2) a list of the members of the evaluation team, delineated by roles and responsibilities with their level of effort; (3) the identification of other required personnel and relevant local subcontractors, their LOE, roles and responsibilities and qualifications; and (4) the deliverable schedule. 2. Inception Report/Evaluation Design: Within two weeks of approval of the work plan, SI must submit to Contracting Officer’s Representative (COR) a combined inception report and evaluation design document (which will become an annex to the Evaluation report). While the inception report will identify information gap to address the evaluation questions, the evaluation design will include: (1) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan; (2) draft questionnaires and other data collection instruments or their main features; (3) the list of potential interviewees and sites to be visited; (4) known limitations to the evaluation design; and (5) a dissemination plan. USAID/Haiti will take up to 10 business days to review and consolidate comments through the COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within 10 business days. 3. In-briefing: Prior undertaking field work, the evaluation team will have an in-briefing with the the Economic Growth and Agriculture Development (EGAD) Team and the Evaluation and Survey Services (ESS) COR to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and to clarify any questions or logistic needs. 4. Evaluation Briefing/Presentation: The evaluation team is expected to hold a final presentation in person to discuss the summary of findings and recommendations to USAID within 10 business days after the conclusion of fieldwork. 5. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section IX: Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the Governance Office will have 10 working business days in which to review and comment on the initial draft, after which point the ESS COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report 5 business days hence, and again the Governance Office will review and send comments on this final draft report within 5 business days of its submission. 6. Final Evaluation Report: The evaluation team will be asked to take no more than 15 business days to respond/incorporate the final comments from the Governance Office. The evaluation team leader will then submit the final report to the COR. All project data and records (FGD and KII summary reports) shall be submitted in full and should be in electronic form in easily readable format, organized, and USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 50 documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. VII. EVALUATION TEAM COMPOSITION The Evaluation Team shall be comprised of three two Personnel positions: (i) a Team Leader, and (ii) an Assistant Team Leader. The Team Leader (TL) is ultimately responsible for the overall management of the evaluation team, coordinating the implementation of the evaluation, assigning evaluation responsibilities and tasks, and authoring the final evaluation report in conformity with this Statement of Work. The TL must be an experienced evaluation expert, with a documented track record of 10 years of experience in the field of evaluation. S/he should have a strong background in the agriculture research field. S/he should be fluent in French and English. S/he should have at least a Master’s Degree in Economics, Statistics, or Agriculture. The Assistant Team Leader (ATL) helps the TL in the overall management of the evaluation team and the final products, in conformity with this Statement of Work. The ATL should be familiar with the agriculture field. S/he must possess excellent writing and interpersonal skills and must be familiar with USAID programs, objectives, and reporting requirements. S/he should have experience in designing and implementing surveys and conducting FGDs. Fluency in French is required. English and Haitian Creole are highly desirable, as is significant prior work experience in Haiti. A Bachelor's degree in Experimental Research or Agriculture Research is required to ensure that all areas of technical expertise required for the evaluation are effectively covered. VIII. EVALUATION SCHEDULE Timing (Anticipated Weeks or Duration) Scheduled Activities 4 Weeks Preparation of the work plan 2 Weeks USAID review and approval of the workplan 4 Weeks Preparation of the Inception report and evaluation design 1 Week USAID review of the Inception Report and evaluation design 2 Weeks In-briefing and Data collection 2 Weeks Data analysis and Evaluation Briefing 3 Weeks Draft Report writing 3 Weeks USAID review of Draft Report 2 Week Incorporate USAID comments and prepare Final Report 51 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV IX. FINAL REPORT FORMAT The evaluation final report should include an abstract; executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see “How-To Note: Preparing Evaluation Reports” and ADS 201mah, USAID Evaluation Report Requirements. An optional evaluation report template is available in the Evaluation Toolkit. The executive summary should be 2–5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The annexes to the report shall include: ● The Evaluation SOW; ● Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; ● All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; ● All sources of information, properly identified and listed; and ● Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of. ● Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. ● Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. X. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.26 ● Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ● Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ● The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ● Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. 26 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 52 ● Evaluation methodology should be explained in detail and sources of information properly identified. ● Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ● Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ● If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. If recommendations are included, they should be supported by a specific set of findings, be action￾oriented, practical, and specific. 53 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV ANNEX B. DATA COLLECTION PROTOCOLS INFORMED CONSENT AGREEMENT—KEY INFORMANT INTERVIEWS (KIIS) Purpose. Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from an organization called Social Impact, a company that is based in the United States. Our team is in Haiti to conduct a study about the Appui à la Recherche et au Développement Agricole (AREA) project. AREA’s goal was to build the capacity of public and private institutions and organizations to provide results￾oriented agricultural research and extension programs that are inclusive of all farmers, promote gender equity, and are appropriate for Haiti. AREA improved and expanded existing country-driven approaches to address food insecurity and undernutrition in Haiti. By strengthening and supporting Haitian public and private institutions operating in the agricultural sector, the project aimed to design, implement, and evaluate research and extension activities that support the agricultural sector and pave a path to self-reliance. We would like your honest impressions, opinions, and thoughts about various issues related to this activity’s implementation and outcomes. We are independent consultants who have no affiliation with those who implemented AREA nor do we represent the Government of Haiti (GoH). Procedures. If you agree to participate, we ask you to discuss your experience and opinion of the activities and services implemented under the AREA project. The interview will take about 1 hour of your time. Although we will publish our findings in a public report, all your answers will be kept confidential in the report; we will not associate your name with anything you say. However, all team members and USAID will have access to the data, and although we will keep your personal identifying information separate from what you say, USAID will have access to interview notes and the document that identifies your interview notes, which could be linked to them. Nothing you tell us will be attributed to any individual person in the report. Rather the report will include only a composite of all the answers received by all of the individuals we interview. Although we may use quotes, none of the individuals interviewed will be named in the report. Risks/Benefits. There are no significant risks to your participation in this study. You will not receive any direct benefit or compensation for participating in this study. Although this study will not benefit you personally, we hope that our results will help improve potential future programs to support agricultural value chains in Haiti. Voluntary Participation: Participation in this interview is completely voluntary. You do not have to agree to be in this study. You are free to end the interview at any time or to decline to answer any question that you do not wish to answer. If you decline to participate in the interview, no one will be informed of this. Do you have any questions at this time? [Interviewer should answer any questions] Permission to Proceed I understand the purpose of the interview as outlined above and understand that I can withdraw from the interview at any time and for any reason. I agree to participate in the interview. (Evaluator records) Yes No Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 54 INFORMED CONSENT AGREEMENT—FOCUS GROUP DISCUSSIONS (FGDS) Purpose. Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from an organization called Social Impact, a company that is based in the United States. Our team is in Haiti to conduct a study about the Appui à la Recherche et au Développement Agricole (AREA) project. The goal of AREA was to build the capacity of public and private institutions and organizations to provide results￾oriented agricultural research and extension programs that are inclusive of all farmers, promote gender equity, and are appropriate for Haiti. AREA improved and expanded existing country-driven approaches to address food insecurity and undernutrition in Haiti. By strengthening and supporting Haitian public and private institutions operating in the agricultural sector, the project aimed to design, implement, and evaluate research and extension activities that support the agricultural sector and pave a path to self-reliance. You have been asked to participate today so that we can learn more about your involvement in the AREA program/learn from your knowledge and expertise on agricultural research and extension services in Haiti. We are meeting with more than 200 individuals (AREA managers, agronomists, farmers, and other research and extension services actors) who participated in the program either as implementers, recipients of program services, and/or as independent experts. You have been invited to participate in this discussion because you participated in at least one AREA activity. We would like your opinions and thoughts about AREA services and activities. We are independent consultants who have no affiliation with AREA or the GoH. Any information collected will be kept confidential in the report; we will not associate your name with anything you say. However, all team members and USAID will have access to the data, and although we will keep your personal identifying information separate from what you say, USAID will have access to interview notes and the document that identifies your interview notes, which could be linked to them. Procedures. If you agree to participate, we ask you to discuss your experience and opinion of the activities and services implemented under the AREA project. The FGD will take up to two hours of your time. Although we will publish our findings in a public report, all of your answers will be kept confidential. Nothing you tell us will be attributed to any individual person. Although we may use quotes, none of the individuals interviewed will be named in the report. However, as this is a group setting, to preserve confidentiality, we ask you not to share anything we discuss here today with anyone outside of this group. Risks/Benefits. There are no significant risks to your participation in this study. You will not receive any direct benefit or compensation for participating in this study. Although this study will not benefit you personally, we hope that our results will help improve potential future institutional capacity building programs for government institutions in Haiti. Voluntary Participation: Participation in this interview is completely voluntary. You do not have to agree to be in this study. You are free to leave the FGD at any time or to decline to answer any question that you do not wish to answer. If you decline to participate, no one will be informed of this. Do you have any questions at this time? [Interviewer should answer any questions] Permission to Proceed I understand the purpose of the interview as outlined above and understand that I can withdraw from the interview at any time and for any reason. I agree to participate in the interview. (Evaluator records) Yes No Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ 55 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV PROTOCOL FOR KIIS WITH USAID COR, USAID TECHNICAL OFFICE, AND M&E SPECIALISTS A. Preliminary identification: 1. Date: 2. Interview location: 3. Interviewer name: 4. Primary notetaker name: 5. Sex of respondent: B. KII questions for USAID 1. What has AREA accomplished over its life? (EQ1, EQ2, and EQ4) 2. With what did AREA face the most challenges in achieving its objectives? 3. What factors (external or internal) best facilitated AREA’s achievement of the activity goals and objectives? 4. Which factors (external or internal) most constrained AREA’s achievement of the activity goals and objectives? (EQ1, EQ2, and EQ4) Probe. To what extent did environmental factors or in the general political and economic environment affect AREA? To what extent did the state and capacity of local universities influence the project’s outcome? To what extent did internal factors such as the effectiveness of the implementing organization affect achievement? 5. What unintended outcomes did you see, if any? (crosscutting) 6. What was the main result of the AREA experiments about the extension models? What worked and what did not about the experiments? 7. What did AREA do to transition from an externally financed, project-oriented research-activity to a local research project within local entities? (EQ2) 8. What has worked/not worked in that approach? Probe. What are the likely changes that will occur once external support for AREA ends? What more needs to be done to ensure sustainability? In what areas, if any, do you think additional USAID assistance makes sense? 9. To what extent do you think that AREA was effectively working with local entities? What were the approaches’ strengths? Weaknesses? What, if anything, might have been done better? (crosscutting) (EQ1) Probe. What are the advantages and disadvantages of working with local universities? How effective was the collaboration with local universities? Were the roles and responsibilities of local universities for research projects clearly defined? Were activities carried out in a timely manner? To what extent were AREA managers able to adapt and find solutions to unexpected challenges? 10. What did AREA do to target different groups in its dissemination activities? 11. What worked/did not work? How did AREA coordinate, if at all, with other USAID- and/or USG activities? 12. What worked/did not work? (crosscutting) 13. What lessons were learned in AREA’s implementation? (crosscutting lessons learned) USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 56 PROTOCOL FOR KIIS WITH AREA IMPLEMENTATION TEAM (INCLUDING SUBCONTRACTORS) A. Preliminary identification: 1. Date: 2. Interview location: 3. Interviewer name: 4. Primary notetaker name: 5. Sex of respondent: B. Informed Consent Protocol Use the informed consent procedures; read informed consent script and ask KII if they agree to participate. Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ C. KII protocol for AREA national implementation team 1. What was your role in AREA implementation? Over which period did you perform this role? 2. How did AREA try to increase research at local universities? 3. What factors contributed to its success or lack of success? (EQ1) 4. How did AREA increase capacities of local organization in conducting research? 5. What factors contributed to its success or lack of success? (EQ1) 6. How did AREA collaborate with local organizing to promote applied research and extension services? 7. What worked/did not work? (EQ1) 8. What are the most important challenges that AREA faced during its implementation? 9. What did AREA do to overcome those challenges? 10. How did AREA identify and select local university partners? 11. How did outcomes related to AREA’s interventions vary among local university partners? 12. Were there significant differences in AREA outcomes among some types of universities versus others? Probe. Were there significant differences in AREA outcomes between public vs. private producers? A university with equipped laboratories vs. a university without laboratories? A university with Ph.D.-level professors? Did AREA propose effective solutions to overcome the above factors? If so, what are some examples? (EQ1) 13. How did AREA encourage smallholder (and medium- and large-scale) farmers to be part of the Alliance? What were the principle strategies AREA used? Which strategies have been the most and least effective to date? Probe. What non-price-related factors affect smallholder farmers’ decisions about what they plant, in what quantity, and to whom they sell their produce? 14. How have AREA interventions addressed key stakeholders concerns through research and extension programs? (EQ2) Probe. What affected AREA’s ability to disseminate results to key stakeholders in Haiti? How did AREA disseminate research results to key stakeholders? What worked/what did not? 15. How did AREA’s interventions affect farmer’s technology adoption and yield increase? Probe. What, if any, benefit have recent developments in the research programs offered to farmers? What more needs to be done to improve the benefits to farmers? 57 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV 16. How effective was the extension models experiment in identifying the best model for farmers’ technologies adoption? What are the factors affecting the models’ effectiveness/sustainability? To what extent have these been studied, and what more needs to be done? (EQ3) Probe. Do you think that the best model is likely to trigger a better adoption rate for all technologies? In what time frame? What are the main issues affecting its success? Do you think the model can be up-scaled nationwide? Why or why not? 17. How effective and sustainable were the short- and long-term training programs? What are the factors affecting AREA trainings’ effectiveness/sustainability? To what extent have these been addressed, and what more needs to be done? (EQ4) 18. How successful has AREA been in improving the overall efficiency of the extension services in Haiti? (EQ3) Probe. Are there significant bottlenecks affecting the efficiency of extension services in addition to those already being addressed by AREA? 19. To what extent, if at all, do policy/regulatory issues affect research development in Haiti? (EQ3) Probe. For example, do agricultural policies and macro-economic policies concerning agricultural imports, landownership law and practices, or food safety regulations have a significant effect on research? How? And how have these policies/regulations changed, if at all, since the inception of AREA? 20. To what extent has AREA successfully transitioned research programs from an externally financed, project-oriented implementation strategy to a locally conducted research program? (EQ3) Probe. How do local stakeholders perceive the transition? What are the likely changes that will occur once external support for AREA ends? What more needs to be done to ensure sustainability? USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 58 PROTOCOL FOR KIIS WITH LOCAL INSTITUTIONS REPRESENTATIVES A. Preliminary identification: 1. Date: 2. Interview location: 3. Interviewer name: 4. Primary notetaker name: 5. Sex of respondent: B. Informed Consent Protocol Use the informed consent procedures; read informed consent script and ask the KIIs if they agree to participate. Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ 1. How long have you been with [government agency/organization]? What are the roles and responsibilities of your institution with respect to overseeing/regulating/conducting/promoting research and extension activities in Haiti? What are your specific roles and responsibilities within your institution? 2. What have been the main challenges faced by your institution in its efforts to improve research and extension services in Haiti in recent years? Has there been any progress and/or worsening of the situation in since 2016? If so, what are some examples? To what factors do you attribute these changes? (EQ1, EQ2, and EQ4) 3. To what extent, if at all, do policy/regulatory issues affect research and extension services development? (EQ1) Probe. For example, what effect do agricultural policies and macro-economic policies concerning agricultural imports, landownership law and practices, climate change, or food safety regulations have on research? 4. Have you been involved in any activities organized by AREA or collaborated with the program in other ways? Please describe your collaboration. (EQ1) 5. Have AREA activities (training, research programs, extension services) been effective in increasing available solutions to agriculture problems in Haiti? How or how not? What, if anything, might have been done differently? What more needs to be done? (EQ1) 6. Have AREA activities (training, research programs, extension services) been effective in involving local universities in addressing agriculture problems in Haiti? How or how not? What, in anything, might have been done differently? What more needs to be done? (EQ2) 7. Which type of university, if any, faces additional challenges to increasing their involvement in research and extension services? Probe. Were there significant differences in AREA outcomes between public vs. private producers? A university with equipped laboratories vs. a university without laboratories? A university with Ph.D.-level professors? Did AREA propose effective solutions to overcome the above factors? If so, what are some examples? (EQ1, EQ2, and EQ3) 8. How have AREA trainees been involved in research and extension services in your institution? What was the outcome of the training in your point of view? To what extent has the training provided new knowledge and skills? If so, what are some examples, and how were they used? 9. How did your institution contribute to disseminating AREA-promoted research? To what extent will activities continue after AREA? 59 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV PROTOCOL FOR FGDS WITH FARMERS A. General Information 1. Date: 2. Number of participants: 3. Sex of participants: __ male __ female 4. Location of FGD: 5. Interviewer name: 6. Primary notetaker name: B. Informed Consent Protocol Use the informed consent procedures; read the informed consent script and ask FGD participants if they agree to participate. Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ C. Focus group questionnaire for participating farmers 1. Were you involved in the extension model experiment with AREA? What technologies were you exposed to, and what happened? What did AREA do with you regarding the technologies? (EQ3) 2. Have you received training and other support services to adopt the technology in recent years? If so, what types of support did you receive? Who provided the support? (EQ3) 3. To what extent have you been able to adopt the suggested technologies? Were some techniques easier to apply than others? Which ones? In your opinion, what factors prevent you or any other farmers from applying the new technologies? (EQ3) 4. To what extent do you feel that you will be able to continue to apply the techniques you learned in the future? Why or why not? (EQ3) 5. What have been the trends in your community in terms of adopting the technologies? What factors have influenced the adoption of the technologies? (EQ3) 6. What are the difficulties/problems that farmers in your community face in receiving and adopting technologies? (EQ3) 7. Have there been any improvements or deterioration in farmer adoption of technologies? If so, what are examples of changes? What factors influenced these changes? (EQ3) 8. What type of technologies work best and are better adopted and why? What recommendations would you offer to better assist farmers in the adoption of new technologies? (EQ3) USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 60 PROTOCOL FOR DIRECT OBSERVATION OF AREA INTERVENTIONS 1. Date 2. Primary observer: 3. Description of the Activity intervention being observed: 4. Location: Observation of AREA Field Experiments Ask stakeholder to explain what is going on in the field. Request to see evidence of features that are used – for experimental design and how the data will be collected and used. Ask for the person/persons in charge of system management, providing user support, or performing data entry to be identified. Ask about training they received. 1. To what extent is the person who was asked to explain able to provide the details? 2. To what extent are the main features of the AREA experiment understood? 3. To what extent was the protocol for the field experiment available? Where? 4. To what extent is the field being properly maintained? Is there evidence of maintenance occurring? (no evidence, some evidence, extensive evidence) Observation of Laboratories Visit the laboratories set up by AREA. 1. Observe the type of instruments and equipment available at the laboratories to conduct research. 2. If possible, observe the process of using the instruments in the laboratory. 3. Ask questions about instruments and their use in the laboratories, and what type of research for which the instruments are needed. 4. Were user manuals provided? Are they in a language that is easily understood? (no user manual, user manual not in local language, user manual in local languages) 5. To what extent are the instruments being properly maintained? Is there evidence of maintenance? (no evidence, some evidence, extensive evidence) 61 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV ANNEX C. DOCUMENTS REVIEWED Quarterly Reports, AREA Background Documents USAID/Haiti. AREA Program Quarterly report Oct – Dec 2015 USAID/Haiti. AREA Program Quarterly report Jan – March 2016 USAID/Haiti. AREA Program Quarterly report Oct – Dec 2016 USAID/Haiti. AREA Program Quarterly report Q1 2018 USAID/Haiti. AREA Program Quarterly report Q2 2018 USAID/Haiti. AREA Program Quarterly report Q3 2018 USAID/Haiti. AREA Program Quarterly report Q4 2018 USAID/Haiti. AREA Program Quarterly report Q1 2019 USAID/Haiti. AREA Program Quarterly report Q2 2019 USAID/Haiti. AREA Program Quarterly report Q3 2019 USAID/Haiti. AREA Program Quarterly report Q4 2019 USAID/Haiti. AREA Program Quarterly report Q2 2020 USAID/Haiti. AREA Program Quarterly report Q3 2020 USAID/Haiti. AREA Program Quarterly report Q4 2020 Annual Work Plans, AREA Background Documents SARD Workplan Oct 2015- Sept 2016 SARD Workplan Oct 2016- Sept 2017 USAID/Haiti. Annual Workplan for the AREA Program. 2018- 2019 USAID/Haiti. Annual Workplan for the AREA Program. Y3 USAID/Haiti. Annual Workplan for the AREA Program. 2019 USAID/Haiti. Workplan for the AREA Program. Oct – Dec 2020 USAID/Haiti. Annual Workplan for the AREA Program. Y5 AID-OAA-A15-00039 University of Florida Cooperative agreement FtF West Haiti RFA Monitoring & Evaluation Plans, AREA Background Documents FY 2020 MEL Workplan final USAID/Haiti. AREA Program M&E Plan Y2. USAID/Haiti. AREA Program M&E Plan Y3 USAID/Haiti. AREA Performance Indicator Tracking Table 2016 USAID/Haiti. UF SARD draft MEP 2105 USAID/Haiti. UF SARD draft MEP resubmission 2016 Other Sources Caroline G. Staub, Graham Clarkson. Farmer-led participatory extension leads Haitian farmers to anticipate climate-related risks and adjust livelihood strategies, Journal of Rural Studies, 2020. Caroline Staub, Anne Gilot, Molene Pierre, Gerald Murray, Rosalie Koenig. Coping with climatic shocks: local perspectives from Haiti’s rural mountain regions. Population and Environment, June 2020. J. C. Fulton, S. Bec, J. Fayette, R. C. Ploetz, K. A. Garrett, and C. L. Harmon. Department of Plant Pathology, University of Florida, Gainesville, FL, et al. First Report of Plantain Soft Rot Caused by Klebsiella variicola in Haiti. Plant Disease. April 7, 2020. Joubert Fayette, Sladana Bec, Stephanie Loubeau, James Fulton, Karen Garrett, Carrie Lapaire Harmon. Department of Plant Pathology, University of Florida. First report of Lasiodiplodia hormozganensis causing fruit rot of eggplant in Haiti. Plant Disease. Aug. 26, 2019. James C. Fulton, Jeannie M. Klein, Sladana Bec, Joubert Fayette, Karen A. Garrett, Jeffrey B. Jones, Sujan Timilsina, Carrie L. Harmon. Draft Genome Sequences of Plant-Pathogenic Klebsiella variicola Strains Isolated from Plantain in Haiti. Microbiology Resource Announcements, July 2020, 9 (29) e00336-20. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 62 Geoffrey Meru, Dayana Leyva, Vincent Michael, Riphine Mainviel, Marie Dorval, Yuqing Fu. Genetic Variation among Cucurbita pepo Accessions Varying in Seed Nutrition and Seed Size. American Journal of Plant Sciences (2019). Geoffrey Meru, Dayana Leyva, Vincent Michael, Riphine Mainviel, Marie Dorval. Seed and oil pumpkin: a potential specialty crop for the southeast. Proceedings of the Florida State Horticultural Society (2018). Li, J., Demesyeux, L., Brym, M. et al. Development of species-specific molecular markers in Vanilla for seedling selection of hybrids. Molecular Biology Reports (2020). Demesyeux, M. L. Mendes, W. T. Crow, A. H. Chambers. Plant-Parasitic Nematodes Associated with Eight Banana Cultivars in Southern Florida. Nematropica (2020). R. Mompremier, Y. Her, G. Hoogenboom, K. Migliaccio, R. Muñoz-Carpena, Z. Brym, R. W. Colbert, W. Jeune. Modeling the response of dry bean yield to irrigation water availability controlled by watershed hydrology. Agricultural Water Management (January 2021). Franky Celestin, Rao S. Mylavarapu, George J. Hochmuth, Yuncong Li, Wesly Jeune & van Santen Edzard. Black bean (Phaseolus vulgaris L.) response to potassium fertilization in two different soils in Haiti. Journal of Plant Nutrition (2020). Bertrhude Albert, Grady Roberts, Amy Harder. Faculty perceptions of students’ extension competency needs within Haiti’s agricultural universities. Journal of International Agricultural and Extension Education. Vol. 24(3) December 2017. Bertrhude Albert, Grady Roberts, Amy Harder. Barriers Faced by Small Scale Farmers in the North Department of Haiti. Journal of International Agricultural and Extension Education. Vol. 24(3) December 2017. Absalon Pierre, Marie Christelle Calixte, Kelly Moore, J. C. Bunch, Rosalie Koenig, Lemâne Delva and T. Grady Roberts. Haitian Agricultural Faculty Preparation for Their Academic Roles. Journal of International Agricultural and Extension Education. Vol. 25(1) May 2018. Calixte, M. C., Roberts, T. G., & Bunch, J. C. (2020). Understanding the Context for Agricultural TVET in Haiti. Journal of International Agricultural and Extension Education. Calixte, M. C., Roberts, T. G., & Bunch, J. C., (2019). Employment Opportunities for Graduates of Agricultural TVET Schools in Haiti. Journal of International Agricultural and Extension Education. Calixte, M. C., Roberts, T. G., & Bunch, J. C., (2019). Exploring the purpose of agricultural technical schools in Haiti. Journal of International Agricultural and Extension Education. Calixte, M. C., Roberts, T. G., & Bunch, J. C. (2018). The balance of theoretical and practical skills in agricultural technical schools in Haiti: An exploration of the curriculum. Journal of International Agricultural and Extension Education. Marie Christelle Calixte, Understanding Haitian Agricultural Technical and Vocational Education and Training: Review of Four Schools, 2018 Wilfrid Calvin, Assessment of intercropping and biological insecticide applications on Melanaphis sacchari infestations in sorghum, and identification of potential natural enemies and alternate hosts, 2019 Franky Celestin, Black beans (Phaseolus vulgaris L.) response to phosphorus and potassium in two different soils in Haiti, 2019 Wanita Dantes, Getting a Jump on Invasive Pathogens Geographic Risk Assessment for New Banana Diseases in the Caribbean and a Rapid Detection Method for Raffaelea lauricola in Environmental Samples, 2020 Rochteau Daréus, Screening Cowpea (Vigna unguiculata L.Walp) germplasm for root knot nematodes resistance (Meloidogyne sp.) and morphological, physiological, and agronomic traits, 2019 Lynhe Demesyeux, Nematodes’ effect on banana and plantains, and the production of miracle fruit in South Florida, 2019 Stéphanie Camelia Docteur, Factors affecting Haitian farmers’ perception and willingness to adopt the black sigatoka disease prevention methods in plantain and banana plantations, 2019 Marie Darline Dorval, Genomic prediction of sweet sorghum agronomic performance under drought and irrigated environments in Haiti, 2019 Vovener Edmond, Development of biotechnology tools to improve precision breeding in vanilla, 2019 Cassandre Feuillé, Evaluation of light quality and light levels for in vitro production of ornamental bananas (Musa sp.), 2019 63 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV Jean Ribert François, Critical Influences on the Aspirations of Caribbean-American Emerging Adults, 2019 Jhonson Leonard, Biological control activity of rice rhizosphere bacteria and their interactive effects with silica treatment against sheath blight of rice, 2019 Marie Rachèle Lexidort, Determining the presence or absence of the Determining the presence or absence of Euscepes postfasciatus in Haiti and use of molecular markers for the distinction of Cylas formicarius elangantulus biotypes, 2019 Carina Theodore, Increasing fresh-market broccoli production in Florida, 2019 Other theses from UF master of science students supported by AREA: Robyn Adair, Field evaluation of cover crops for nematode and weed management in Florida and Haiti, 2018 USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 64 ANNEX D. QUANTITATIVE SURVEYS A. Demographics 1. Date: 2. Name of farmer: 3. Sex of: __ male __ female 4. Commune and locality: 5. Department: 6. Interviewer name: B. Informed Consent Agreement Purpose: Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from an organization called Social Impact, a company that is based in the United States. The goal of AREA was to build the capacity of public and private institutions and organizations to provide results-oriented agricultural research and extension programs that are inclusive of all farmers, promote gender equity, and are appropriate for Haiti. AREA improved and expanded existing country-driven approaches to address food insecurity and undernutrition in Haiti. By strengthening and supporting Haitian public and private institutions operating in the agricultural sector, the project aimed to design, implement, and evaluate research and extension activities that support the agricultural sector and pave a path to self￾reliance. You have been asked to participate today so that we can learn more about your involvement in the AREA project and learn from your knowledge and expertise on agricultural research and extension services in Haiti. We are speaking with more than 200 individuals (AREA managers, agronomists, farmers, and other value chain actors) who participated in the project either as implementers, recipients of project services, and/or experts. You have been invited to participate in this discussion because you participated in at least one AREA activity or because you farm sorghum in a region serviced by the AREA Alliance. We would like your opinions and thoughts about the AREA services. We are independent consultants who have no affiliation with AREA or the GoH. Procedures. If you agree to participate, we ask you to discuss your experience and opinion of the activities and services implemented under the AREA project. Although we will publish our findings in a public report, all of your answers will be kept confidential. Nothing you tell us will be attributed to any individual person. None of the individuals interviewed will be named in the report. However, as this is a group setting, to preserve confidentiality, we ask you not to share anything we discuss here today with anyone outside of this group. Risks/Benefits. There are no significant risks to your participation in this study. You will not receive any direct benefit or compensation for participating in this study. Although this study will not benefit you personally, we hope that our results will help improve potential future institutional capacity-building programs for government institutions in Haiti. Voluntary Participation. Participation in this interview is completely voluntary. The interview should take about 20 minutes of your time and you will not be contacted in the future. We would greatly appreciate your help in responding to these questions. You do not have to agree to be in this study. You are free to leave the survey at any time or to decline to answer any question that you do not wish to answer. If you decline to participate, no one will be informed of this. Do you have any questions at this time? [Interviewer should answer any questions] Permission to Proceed I understand the purpose of the interview as outlined above and understand that I can withdraw from the interview at any time and for any reason. I agree to participate in the interview. (Evaluator records) Yes No Initials of evaluator to indicate receipt of verbal consent: _____________________ Date _________________________ 65 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV AREA Project Final Performance Evaluation Trainees Survey # QUESTIONS ANSWERS Q1 Interview number Q2 Interviewer initials Q3 Location Q4 Date of Interview Q5 Interview start time (circle AM or PM). If needed, we can also add “interview end time.” READ: The above informed consent Q6 Are you willing to answer these questions? 1. Yes, 2. No Q7 How old are you? a) Under 30, b) 31-40, c) 41-50, d) Over 50 years old Q8 What is your gender? Male, 2. Female Q9 The highest level of education? No school, 2. Primary, 3. High school, 4. College, 5. Post-graduate Q10 Do you currently have paid employment? 1. Yes, 2. No Q11 Where was your last employment? 1. State university, 2. Private university, 3. AREA, 4. GoH, 5. CRDD, 6. Other, please specify (______) Q12 Where you involved in agricultural research? 1. Yes, 2. No Q13 Where you involved in extension activities? 1. Yes, 2. No Q14 What type of training did you received from AREA? 1. Short term, 2. Long term Q15 Where was the training conducted? 1. Port-au-Prince, 2. Northern Haiti, 3. Southern Haiti, 4. Center Haiti. 5. USA, 6. Other, please specify (______) Q16 What was the theme of the research? (circle all that apply) 1. Soil science, 2. Plant pathology, 3. Maize seeds, 4. Climate change, 5. legume breeding, 6. Other, please specify (______) Q17 Have you heard about the theme before? 1. Yes, 2. No Q18 In what language was the training? 1. Creole, 2. French, 3. English, 4. Other, please specify (_______-) Q19 Did you complete the training? 1. Yes, 2. No If no, why not_______________________ USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 66 # QUESTIONS ANSWERS Q20 What is your proof of completion of the training? 1. Certificate of completion, 2. Degree certificate, 3. Nothing Q21 Do you know someone at work with that knowledge? 1. Yes, 2. No Q22 How long was the training? 1. Two years, 2. Few days, 3. Few weeks, 4. Other, please specify (______) Q23 Was the training related to your work? 1. Yes, 2. No Q24 Did you have hands-on training? 1. Yes, 2. No Q25 What type of practicum did you have? Q26 How relevant was the training? 1. Very, 2. Somewhat, 3. No Q27 Did you use what you learned from the training? 1. Yes, 2. No Q28 How long after the training did you use new skills/knowledge? 1. Between 1 and 3 months, 2. Between 4 and 12 months, 3. After 12 months Q29 Who recommend you for the training? 1. Boss, 2. Co-workers, 3. AREA, 4. Other Q30 Were you ever asked to use the new skills by supervisors? 1. Yes, 2. No Q31 Is there someone else in your institution with this knowledge? 1. Yes, 2. No Q32 Did you train someone else after receiving the training? 1. Yes, 2. No Q33 Did you receive training materials? 1. Yes, 2. No Q34 Did you go back to these materials after the training? 1. Yes, 2. No Q35 Do you think you have learned something useful from the training? 1. Yes, 2. No Q36 Would you agree that you are better at what you do thanks to the training? 1. Yes, 2. No Q37 Did you have new tasks that require you to go back to the training materials? 1. Yes, 2. No Q38 Can you enumerate two body of knowledge you mastered through the training? If Yes...Name them? 1. Yes, 2. No Q39 Can you enumerate two new skills that you have used after the training? If Yes...Name them? 1. Yes, 2. No Q40 Did you publish research results using knowledge from the training? 1. Yes, 2. No, 3. N/A (N/A if farmers) Q41 Did you disseminate results or information to farmers or others thanks to the training? 1. Yes, 2. No 67 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV # QUESTIONS ANSWERS Q42 Were you able to work better in extension activity thanks to the training? 1. Yes, 2. No Q41 Is there something you do differently at work after the training? 1. Yes, 2. No Q42 Would you recommend the training to your co-workers as something useful to do their work? 1. Yes, 2. No Q43 Did your boss acknowledge your new skills after the training? 1. Yes, 2. No Q44 Would you agree that you are a better researcher after the training? 1. Yes, 2. No Q45 At work, is the new skill required? 1. Yes, 2. No, Q46 Would you hire someone without these skills to replace you? 1. Yes, 2. No Q47 How satisfied were you with the training? 1. Very, 2. Somewhat, 3. No Q48 Do you feel well informed now about the topic of the training? 1. Yes, 2. No Q49 Do you wish you could get more information about the theme of the training? 1. Yes, 2. No Q50 What are the sources of information someone can access? 1. Source 1 2. Source 2: 3. Source 3: USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 68 ANNEX E. PRELIMINARY LIST OF KEY INFORMANTS CATEGORY FUNCTION NAMES TEL/EMAIL USAID COR Reginald Toussaint AOR M&E Gandhi Dorival Implementing Partner Chief of Party Deputy Chief of Party Lemane Delva University of Florida representative University of Illinois representative University of Louisiana representative University of Readings Representatives Maize Seed System Program Principal Invewtigator (PI) Legume breeding program PI Climate Smart Solutions PI Plant Pathology Program PI Soil Science Research program PI Post-Harvest and Food Safety PI Small grant program PI CIMMYT PI Ministry of Agriculture Former Director of innovation Garry Augustin Fractyl Mertilus garryaugustin@yahoo.fr fractyl007@yahoo.fr Actual Director of Innovation Beaubrun Valoune Ministry of Agriculture Representatives TBD Plant Protection Directorate Dakson Sanon Pascal Constant sdakson@hotmail.com Unite Meteorologique Nationale Marcelin Esterlin esterlinm@yahoo.fr Local CRDDs Montrouis CRDD Representative Jane Wynne Jean Junior Auguste jane@wynnefarm.org jean.junior_auguste@gmail.c om Robin CRDD Representatives Pierre Paul Jules sohaderk@yahoo.fr Bas Boen CRDD Representatives Kenel Cadet Freud Lucas kenelcadet@gmail.com flucas@fonhdad.org 69 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV CATEGORY FUNCTION NAMES TEL/EMAIL Local universities FAMV Dean Jocelyn Louissaint jlouissaint2000@yahoo.fr FAMV Vice Dean of Research Nicolas Carvil nicarvil@yahoo.com FAMV professor beneficiary UNIQ FSAE Dean UNIQ Vice Dean of Research Gael Pressoir gael.pressoir@uniq.edu UP Centre Dean UP Centre Vice Dean of Research UNDH Dean Benedique Paul benediquep@yahoo.fr UND Vice Dean of Research UNEPH Dean Lucien Jean Bernard lucienjeanbernard@yahoo.fr UNEPH Vice Dean of Research American University of the Caribbean Paul Touloute Arsene Similien touloutepm@yahoo.com arsenesimilien@gmail.com Université d'état d'Haïti, campus Henri Christophe de Limonade Emerson Louis Small grants beneficiaries FAMV Myriam Hyppolite R. Pierre Tescar Romain Exilien Elda Deronnette UPC UNEPH Locally hired researchers Maize Seed System Program Legume breeding program Climate Smart Solutions Plant Pathology Program Soil Science Research program Jeune Pierre Post-Harvest and Food Safety Small grant program CIMMYT Levael Eugene USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 70 ANNEX F. LIST OF AREA PUBLICATIONS IN PEER-REVIEWED JOURNALS CLIMATE CHANGE ● Farmer-led participatory extension leads Haitian farmers to anticipate climate-related risks and adjust livelihood strategies, Journal of Rural Studies, 2020. ● Coping with climatic shocks: local perspectives from Haiti’s rural mountain regions. Population and Environment, June 2020. PLANT PATHOLOGY ● First Report of Plantain Soft Rot Caused by Klebsiella variicola in Haiti. Plant Disease. April 7, 2020. ● First report of Lasiodiplodia hormozganensis causing fruit rot of eggplant in Haiti. Plant Disease. Aug. 26, 2019. To our knowledge, this is the first report of L. hormozganensis causing fruit rot in eggplant. Chemical disease management is limited in Haiti, therefore investigation of the host range and disease incidence in eggplant growing areas is necessary to determine if removal of inoculum is a viable strategy. ● Draft Genome Sequences of Plant-Pathogenic Klebsiella variicola Strains Isolated from Plantain in Haiti. Microbiology Resource Announcements, July 2020, 9 (29) e00336-20. IMPROVED SEEDS AND CROPS ● Genetic Variation among Cucurbita pepo Accessions Varying in Seed Nutrition and Seed Size. American Journal of Plant Sciences (2019). ● Seed and oil pumpkin: a potential specialty crop for the southeast. Proceedings of the Florida State Horticultural Society (2018). ● Development of species-specific molecular markers in Vanilla for seedling selection of hybrids. Molecular Biology Reports (2020). This is the first report using species-specific, qRT-PCR-based molecular markers in Vanilla. These markers are inexpensive, simple to develop, and can rapidly screen large populations. These methods will enable the further development of species-specific molecular markers when creating Vanilla interspecific hybrid populations. HORTICULTURE ● Plant-Parasitic Nematodes Associated with Eight Banana Cultivars in Southern Florida. Nematropica (2020). This is the first report of nematodes associated with a diverse banana accessions in both histosol and limestone sites in southern Florida. HYDROLOGY ● Modeling the response of dry bean yield to irrigation water availability controlled by watershed hydrology. Agricultural Water Management (January 2021). SOIL ● Black bean (Phaseolus vulgaris L.) response to potassium fertilization in two different soils in Haiti. Journal of Plant Nutrition (2020). 71 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV ● Response of black bean (Phaseolus vulgaris L.) to phosphorus fertilization in acidic and alkaline soils in Haiti, Journal of Plant Nutrition (2019). This study contributes to the first step in optimizing black bean production through better interpretation and nutrient recommendation of soil tests for Haitian soils. ● Cover crop options for nematode and weed suppression in Haiti. Proceedings of the Florida State Horticultural Society (2018). HIGHER EDUCATION ● Faculty perceptions of students’ extension competency needs within Haiti’s agricultural universities. Journal of International Agricultural and Extension Education. Vol. 24(3) December 2017. The purpose of this research was to determine Haitian faculty’s perception of what competency areas should be focused on to ensure that their students are prepared to be successful extension workers upon graduation. ... An overall recommendation is to create a council consisting of faculty, students, extension personnel, farmers, and government officials who will be tasked with determining necessary competency areas for students. ● Barriers Faced by Small Scale Farmers in the North Department of Haiti. Journal of International Agricultural and Extension Education. Vol. 24(3) December 2017. Haiti is one of the economically poorest countries in the world. As a result, billions of dollars have been invested into agricultural initiatives in Haiti, however, less emphasis has been placed on building the capacity of farmers through training and educational programs. Limited research has been conducted on barriers faced by Haitian farmers. The purpose of this study was to explore the barriers and faced by small-scale farmers in the North Department of Haiti. ● The tertiary agriculture institutions have the great challenge of educating the future professionals who will help in meeting the growth needs of the Haitian agriculture sector. The extent that faculty are prepared for the teaching roles is unknown. This study examined this topic using a basic qualitative study that consisted of interviews with 37 lecturers working at five leading agricultural universities in Haiti. ● Social capital of faculty within Haiti’s agricultural education and training system. Journal of International Agricultural and Extension Education. Journal of International Agricultural and Extension Education. Vol. 25(2) August 2018. The purpose of this research was to identify how teaching at multiple institutions impacts social capital of faculty within the Haitian agricultural education and training system. ● Haitian Faculty Perceptions of Students’ Competence at Graduation: An Opportunity for Curricula Modification. Journal of International Agricultural and Extension Education. Vol. 25(3) December 2018. The results of this revealed that (a) the employment conditions for university graduates were challenging; (b) faculty did not believe it was important for their students to graduate proficient in all the GFRAS New Extensionist competency areas; (c) faculty believed students should have research skills, the ability to be self-directed, and the ability to work in teams; and (d) faculty generally believed their institutions were producing graduates with the competencies needed for employment, although several lecturers identified specific areas for improvement. Recommendations for curricula reform and additional research are suggested. GENDER AND EXTENSION ● Understanding the Context for Agricultural TVET in Haiti. Journal of International Agricultural and Extension Education. USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 72 ● Employment Opportunities for Graduates of Agricultural TVET Schools in Haiti. Journal of International Agricultural and Extension Education. ● Exploring the purpose of agricultural technical schools in Haiti. Journal of International Agricultural and Extension Education. ● The balance of theoretical and practical skills in agricultural technical schools in Haiti: An exploration of the curriculum. Journal of International Agricultural and Extension Education. ● Gender Integration in Agricultural Extension Services: A literature review, April 2019. AREA also supported the publication of many booklets and manuals for Haitian agriculture stakeholders in English, French, and Creole: ● Evaluation of the behavior of 24 lines of beans (Phaseolus vulgaris L.) (French), 2020 ● Constructing a Solar Grain Dryer (English) (Creole), 2019, 2020 | Video presentation on building a solar grain dryer (Creole) ● Study Of Climatological Stations and Networks In Haiti (English) (French), 2020 ● How to Improve Soil Fertility in Haiti (French), 2020 ● Manual on Using PhotosynQ (English) (French), 2020 ● Weather Station Maintenance and Repair Manual (English) (French), 2020 ● Manual on how to safely operate a lab (French), 2020 ● Feed the Future Haiti AREA Project: Master of Science Students, August 2019 73 | PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE USAID.GOV ANNEX G. LIST OF ARTICLES AREA TRANSLATED INTO CREOLE 1. Monica Elliott, Ken Pernezny, Aaron Palmateer, Nikol Havranek. Guidelines to identification and management of plant disease problems: Part I. Eliminating insect damage and abiotic disorders. PP248. 2. Ken Pernezny, Monica Elliott, Aaron Palmateer and Nikol Havranek. Guidelines for Identification and Management of Plant Disease Problems: Part II. Diagnosing plant diseases caused by fungi, bacteria and viruses. PP249. 3. Aaron Palmateer, Ken Pernezny, Monica Elliott and Nikol Havranek. Guidelines for identification and management of plant disease problems: Part III. Managing plant diseases. PP250. 4. Monica Elliott, Ken Pernezny, Aaron Palmateer, and Nikol Havranek. Guidelines for identification and management of plant disease problems: Part IV. Plant health questions to ask the client. PP251. 5. Carrie Harmon, Philip Harmon, Aaron Palmateer, Mattews Paret, and Natalie Peres. Sample Submission Guide for Plant Diagnostic Clinics of the Florida Plant Diagnostic Network. RFSR007 USAID.GOV PERFORMANCE EVALUATION OF APPUI A LA RECHERCHE ET AU DEVELOPPEMENT AGRICOLE | 74 ANNEX H. LIST OF ARTICLES PUBLISHED WITH THE MARNDR COLLABORATION 1. J. C. Fulton, S. Bec, J. Fayette, R. C. Ploetz, K. A. Garrett, and C. L. Harmon. Department of Plant Pathology, University of Florida, Gainesville, FL, et al. First Report of Plantain Soft Rot Caused by Klebsiella variicola in Haiti. Plant Disease. April 7, 2020. 2. Joubert Fayette, Sladana Bec, Stephanie Loubeau, James Fulton, Karen Garrett, Carrie Lapaire Harmon. Department of Plant Pathology, University of Florida. First report of Lasiodiplodia hormozganensis causing fruit rot of eggplant in Haiti. Plant Disease. Aug. 26, 2019. James C. Fulton, Jeannie M. Klein, Sladana Bec, Joubert Fayette, Karen A. Garrett, Jeffrey B. Jones, Sujan Timilsina, Carrie L. Harmon. Draft Genome Sequences of Plant-Pathogenic Klebsiella variicola Strains Isolated from Plantain in Haiti. Microbiology Resource Announcements, July 2020, 9 (29) e00336-20.