Midterm Evaluation YEAC VIII: Youth Emergency Action Committee Phase 8 March 2022; Dominica, Grenada, Jamaica, Saint Lucia BHA Disaster Risk Reduction Program in the Caribbean Caritas Antilles, Caritas Dominica, Caritas Grenada, and Saint Patricks’ Rangers Roseau, Dominica; Saint George’s, Grenada; Kingston, Jamaica; Castries, Saint Lucia Evaluators – Hannah Kyle/CRS Department of Humanitarian Response; Cassandra Bissainthe/CRS Department of Humanitarian Response 2 Contents Acronyms............................................................................................................................................................ 3 Introduction........................................................................................................................................................ 4 Background..................................................................................................................................................... 4 Table 1. Sector Table, YEAC Phase VIII ........................................................................................................ 5 Methodology .................................................................................................................................................. 7 Table 2. General questions.......................................................................................................................... 7 Table 3. Focus Groups with YEAC participants ............................................................................................ 8 Table 4. Key Informant Interviews with Relevant Stakeholders .................................................................. 9 Data Collection and Analysis......................................................................................................................... 10 Conclusions................................................................................................................................................... 10 1. Relevance ......................................................................................................................................... 10 2. Impact............................................................................................................................................... 12 3. Efficiency........................................................................................................................................... 13 4. Coordination..................................................................................................................................... 15 5. Sustainability.................................................................................................................................... 16 Annex 1: Recommendations and the Action Plan............................................................................................. 18 Annex 2: PowerPoint Presentation of the findings........................................................................................... 28 Annex 3: Sources of Data and Data Collection Methods................................................................................... 29 3 Acronyms BHA: U.S. Government Bureau of Humanitarian Assistance CA: Caritas Antilles CD: Caritas Dominica CDEMA: Caribbean Emergency and Disaster Management Authority (Saint Lucia) CG: Caritas Grenada COVID-19: SARS-CoV-2 CRS: Catholic Relief Services DRR: Disaster Risk Reduction EMPOWER: Empowering Partner Organizations Working on Emergency Responses FGDs: Focus Group Discussions KSAMC: Kingston & St. Andrew Municipal Corporation (Jamaica) MEAL: Monitoring, Evaluation, Accountability, and Learning MoE: Ministry of Education (Saint Lucia) NaDMA: National Disaster Management Authority (Grenada) NEMO: National Emergency Management Organization (Saint Lucia) SPR: Saint Patricks’ Rangers USAID: U.S. Agency for International Development YEAC: Youth Emergency Action Committees 4 Introduction Background The Latin America and Caribbean region faced an average of 17 hurricanes per year between 2000 and 2019, 23 of which were Category 5 hurricanes.1 Storms are becoming more powerful and more frequent, giving countries less time to recover between events as evidenced by Dominica’s experience with Tropical Storm Erika in 2015 and Hurricane Maria in 2017, which affected the entire estimated population of the island (71,293).2 Of the top 10 countries in the region most exposed to tropical cyclones, 8 of them are in the Caribbean.3 While effects of other natural disasters such as flooding, earthquakes, and tsunamis have been felt less in the Caribbean in relation to South America, coastal areas are vulnerable to these disasters as well and climate change, including the El Niño and La Niña weather phenomena, 4 may increase the Caribbean’s vulnerability. Further, in discussing the Caribbean’s vulnerability to disasters, recovery from large events is challenging: “Disasters are often more devastating in small island nations like the Bahamas, which have more limited resources to cope, including small populations and modest economic bases, competition for limited finances, limited pools of technical specialists and inadequate institutional measures.”5 In 2020, the COVID-19 pandemic exploded, creating emergency conditions in each country, as well as severe travel restrictions between countries. Given shorter recovery times and increased uncertainty as to weather patterns, as well as the sudden onset of the COVID-19 pandemic, emergency preparedness in the Caribbean is a unique opportunity to prepare before disaster strikes so that resources, both human and capital, are pre-positioned in case of an emergency. The mandate of the Youth Emergency Action Committees (YEACs) is to build capacity within the four countries (Dominica, Grenada, Jamaica, and Saint Lucia) to empower youths as change-agents particularly in disaster risk reduction and emergency response, while creating pathways to leadership development, for longer-term sustainability and resilience. With support from USAID's Bureau for Humanitarian Assistance (BHA), Caritas Antilles (CA) and its partners have been building the capacity of marginalized urban communities since 2009 by working with at-risk youth through the development and support of the YEACs. The aim of Phase VIII is to implement a three-year program to build on the lessons learned and achievements of the previous phases to strengthen the network of 30 YEACs (12 communities in Jamaica, 8 communities in Grenada, 7 communities in Saint Lucia, and 3 communities in Dominica). 1 UNOCHA.Natural Disasters in Latin America and the Caribbean, 2000-2019; 1. 2 UNOCHA.Natural Disasters in Latin America and the Caribbean, 2000-2019; 5. 3 UNOCHA.Natural Disasters in Latin America and the Caribbean, 2000-2019; 6. 4 UNOCHA.Natural Disasters in Latin America and the Caribbean, 2000-2019; 16. 5 Crystal Romeo Upperman, Gillian Marcelle and Carole Excell. Strengthening Disaster Preparedness in the Caribbean, 2019. 5 Table 1. Sector Table, YEAC Phase VIII Sector Name: Risk Management Policy and Practice Objective: Objective 1: YEACs further fulfills and strengthens their roles in disaster preparedness and response Objective 2: Vulnerable communities have increased capacity in disaster preparedness. Dollar Amount Requested: US$ 1.8 Million Number of People Targeted: 709 YEAC members; 16 991 community members Number of IDPs Targeted: 0 Geographic Area(s): • Jamaica: St. Andrew- Mavis Bank, Gordon town; St. Thomas-Port Morant, Grants pen; St. Catherine-Waterford, Greater Portmore; Kingston - Drewsland; New Haven; Waterhouse; Olympic Gardens; Callaloo/Riverton; Seaview Gardens; Seivwright Gardens; Waltham Gardens. • Grenada: Grand Anse (Mont Tout), Gouyave, Victoria, Sauteurs, Grenville, Carriacou, Petit Martinique and a community in the Parish of St. David. • Saint Lucia: Ti Rocher, Marchand, La Clery, Bexon, Forestierre, a community in the north and a community in the south. • Commonwealth of Dominica: Two communities in Roseau and Environs and Pointe Michel Keyword(s): Climate; First Aid Training and or Search and Rescue Training; Information Systems/Geographic Information Systems (GIS) Sub-sector Name: Capacity Building and Training Indicators: Indicator Target Indicator 1 (BHA): Percentage of people trained who retain skills and knowledge after two months (Output 1.1.1.) 80% Indicator 2 (BHA): Number of people passing final exams or receiving certificates (output 1.1.1.) 319 (128 Males, 191 females) Indicator 3 (BHA): Number of people trained in First Aid, Search and Rescue, or health related Disaster Risk Reduction activities (output 1.1.1.) 395 (158 males, 237females) 6 Indicator 4 (BHA): Number of people trained in disaster preparedness, risk reduction and management (outputs 2.1.1. and 2.1.2.) 2672 (1069 males, 1603 females) Indicator 5 (Custom): Number of Trainings 159 Indicator 6 (Custom): Number of new active members by sex (output 1.1.2 and Output 1.2.3.) 369 (148 males, 221 females) Indicator 7 (Custom): Percentage of household and community entities that implement at least one action of their emergency disaster plans (IR2.1.) 80% Indicator 8 (Custom): Number of people trained in Leadership and Life skills relevant to Disaster Risk Reduction 325 (130 males, 195 females) Indicator 9 (Custom): Number of Teambuilding and Social Events 76 Indicator 10 (Custom): Number of Stakeholder engagements 72 The project management team included Midterm Evaluation plans in the proposal to BHA. CA requested Catholic Relief Services (CRS) to be the external evaluator for the process which was conducted at month 18 during the 30-month project. CA, with partners Caritas Dominica (CD), Caritas Grenada (CG), and Saint Patricks’ Rangers (SPR), is committed to evaluating and improving the quality of its disaster risk reduction and emergency preparedness programs. The overall goal of the midterm evaluation is to enable the team to learn from the implementation of the program thus far, and to make improvements so that the program is effective in meeting the needs of people potentially affected by future disasters. The objectives of the Midterm Evaluation are:6 1. To verify that project progress is in line with reported/expected project planning and outcomes. 2. To identify different approaches that partners are undertaking to realize deliverables, and whether approaches/methodologies are contributing to planned capacity strengthening outcomes. 3. To review the strengths and weaknesses of YEAC’s approach to date and to identify ways to strengthen it. 4. To identify lessons learned and adaptation for the remainder of the project period and for future programs. 5. To identify how the YEAC project team has worked together within and across country programs. 6. To analyze the importance of the emergency response and leadership and life skills trainings to the youths’ self-development and the development of their communities. 7. To gain an appreciation of organizations’ emergency response mechanisms and possible gaps in general emergency response 8. To understand stakeholders’ perspective on volunteerism and the role of youths in emergency response. 6 Outlined in the Evaluation Scope of Work included in the application for funding. 7 Methodology The CRS technical advisors carried out the midterm evaluation through March 2022 (March 5-31) in the 4 countries (in order of evaluation): Saint Lucia, Grenada, Dominica, and Jamaica. The Monitoring, Evaluation, Accountability, and Learning (MEAL) Technical Advisor acted as lead evaluator responsible for evaluating the program and producing the final report. The Technical Advisor for Partnership and Capacity Strengthening in the Caribbean Zone coordinated the interviews with Caritas Antilles management and conducted interviews in Saint Lucia and Jamaica. Project teams in all 4 countries supported the CRS technical advisors in logistics, coordination, and through sharing their experiences of the program with the evaluation team. The Midterm Evaluation team performed the exercise as follows: The evaluation team had an initial discussion with the Regional Coordinator of CA, the prime organization for this project, to ensure that expectations, results, and process were clear to both parties. The evaluation methods for the evaluation were discussed, verifying the activities and objectives for the midterm evaluation. The evaluation team used the five criteria, as set out below, to structure data collection and reflection sessions. The evaluation team held discussions with the relevant team in each country and other stakeholders to reach conclusions based on the criteria. The evaluation team consulted project participants through semi-structured Focus Group Discussions (FGDs). The evaluation team consulted the team at various levels - management and community outreach teams - to understand what they feel allows them to achieve the planned results and what they feel the barriers are that prevent them from achieving the results. The evaluation team presented and discussed the findings with the CA management team in a preliminary meeting before the workshop to give an opportunity for initial feedback before presenting to the larger team. This meeting was followed by a series of virtual discussions to discuss the evaluation findings and to create an action plan that encompasses both this project and future projects. The presentation can be found in Annex 2. The CRS EMPOWER team leader facilitated discussions around a specific action plan, complete with the names of the person(s) responsible and a timeline, allowing the action plan to be immediately implementable after the reflection session. The action plan can be found in Annex 1. The Midterm Evaluation was created based on the following questions, which were identified taking into account the evaluation criteria defined by CA. Table 2. General questions Criterion Question Relevance Are the specific activities and outputs of the program consistent with the YEAC overall goals and indicators? Impact What has happened as a result of the project? What real difference has the activity made to the beneficiaries? 8 Efficiency Were activities cost-efficient; were objectives achieved on time? Was the project implemented in the most efficient way, compared to potential alternatives? Coordination How effective is the current coordination? Sustainability To what extent are the benefits of the project likely to continue after OFDA funding ceases? What major factors may influence the achievement or non-achievement of sustainability of the program or project? Methods Study site: The study was carried out both virtually and in-person in Roseau, Dominica; Saint George’s, Grenada; Kingston, Jamaica; and Castries, Saint Lucia Population: The Midterm Evaluation focused on relevant stakeholders from government, civil society, and project staff as well as on participants of the YEACs in each country, all working in disaster risk reduction and emergency preparedness in their respective countries. Data collection and handling The tools for the data collection process were developed by the Midterm Evaluation leader. Multiple data collection tools were used, including: Table 3. Focus Groups with YEAC participants No. of the Focus Group Discussion Type of participants Number of participants Location of participants Date 1 YEAC members 15 Caritas Antilles office; Castries, Saint Lucia March 7, 2022 2 YEAC members 10 Catholic Church Youth Center; Gouyave, Grenada March 10, 2022 3 YEAC members 11 Caritas Dominica office; Roseau, Dominica March 15, 2022 4 YEAC members 12 Saint Patricks’ Rangers office; Kingston, Jamaica March 21, 2022 Each Focus Group Discussion (FGD) was carried out with current YEAC members. FGDs were mixed-gender, held between 12 and 15 people, and covered the ages of 16-35. During these discussions, a semi-structured protocol was followed to evaluate the participants' perceptions of the program based on the criteria: relevance, impact, efficiency, coverage, and sustainability. There were 4 FGDs in total, 1 per country. 2. Interview with relevant stakeholders Semi-structured interviews were conducted with the relevant stakeholders from government and civil society, as well as project staff in each country. The evaluation team conducted 30 interviews with 32 people: 9 Table 4. Key Informant Interviews with Relevant Stakeholders Name Organization Position Date Ms. Maria Medard National Emergency Management Organization (NEMO), Saint Lucia Deputy Director March 7, 2022 Mr. Mabius Francis Saint Lucia Fire Service Station Officer March 7, 2022 Mr. Bernez Khodra Caribbean Emergency and Disaster Management Authority (CDEMA) and former School Safety Officer for Ministry of Education (MoE), Saint Lucia Safety Consultant March 7, 2022 Mr. Arthur Scott Entrepot Secondary School, Saint Lucia Principal March 7, 2022 Ms. Proann Francis Caritas Antilles Project Officer March 7, 2022 Mr. Villai Edward Caritas Antilles Project Assistant March 7, 2022 Ms. Terencia Gaillard Saint Lucia Red Cross Director General March 8, 2022 Mr. Glenn Wilson St. John's Association Chief Commissioner/Training Officer Castries March 8, 2022 Ms. Cynthia Fontenelle Caritas Antilles Regional Program Manager March 8, 2022 Ms. Alaine Weeks Caritas Antilles Community Promoter March 8, 2022 Mr. Francis Darius Caritas Grenada Caritas Coordinator March 9, 2022 Mr. Sylvan McIntyre Government of Grenada/Port of Grenada Superintendent March 9, 2022 Mr. John Amonie Caritas Grenada Project Assistant March 9, 2022 Mr. Benedict Peters National Disaster Management Authority (NaDMA), Grenada National Disaster Coordinator March 9, 2022 Ms. Rose-Anne Redhead Caritas Grenada Program Manager March 10, 2022 Ms. Natasha James USAID, Grenada Trainer March 10, 2022 Ms. Chanda Stafford Caritas Grenada Community Promoter March 11, 2022 Ms. Annette Henry Caritas Dominica Finance Officer March 13, 2022 Deacon Curtis Victor Caritas Dominica National Coordinator March 14, 2022 Mr. Anthony Williams A&D Instructors Instructor March 14, 2022 Ms. Mahalia Paul Caritas Dominica Project Manager March 14, 2022 Mr. Donalson Frederick A&D Instructors Instructor March 14, 2022 Ms. Tamara Bernard Caritas Dominica Project Assistant March 14, 2022 Ms. Dailah Felix Caritas Dominica Community Promoter March 14, 2022 Mr. Mandela Christian Office of Disaster Management, Dominica Program Officer March 14, 2022 10 Ms. Sandra Charter￾Rolle Dominica Red Cross Director General March 15, 2022 Ms. Nicole King IsraAid, Dominica Country Manager March 15, 2022 Mr. Dilton Pike Jamaica Fire Brigade Assistant Superintendent March 21, 2022 Mr. Terry Forrester Kingston & St. Andrew Municipal Corporation (KSAMC) Disaster Coordinator March 21, 2022 Ms. Kathlene Arnold Jamaica Red Cross Resource Mobilization Administrator March 22, 2022 Mr. Christopher Constantine Waterhouse CDC, Jamaica President March 22, 2022 Ms. Beverley Bailey Seaview Gardens SDA Church Center Manager March 22, 2022 3. Reflection workshop A series of reflection workshops was carried out with CA, CD, CG, and SPR staff involved in the program, using post-action review methods after interviews and focus groups. The initial findings of the evaluator were shared with the team, discussed, and adjusted according to feedback. The detailed action plan to address the areas of improvement was developed in a large group with key staff from each partner following the presentation of initial findings. Data Collection and Analysis Data collection was led by CRS Technical Advisor for Monitoring, Evaluation, and Learning (MEAL) in coordination with the CRS Technical Advisor for Partnership and Capacity Strengthening – focal point for the Caribbean. Each interviewer took notes. The detailed notes contain everything as said. Notes were either taken in Microsoft word or by hand and then transferred to Microsoft Word. Data analysis focused on the outline of the Midterm Evaluation questions (table 2). The MEAL Technical Advisor analyzed them using qualitative methods according to the categories defined in table 2. The reflection workshop was held virtually over three weeks in three sessions, the first one to present the findings and discuss, the second one to begin the detailed action planning, and the third session to finalize the action plan. Representatives from each partner were present. The findings were presented by the CRS MEAL Technical Advisor and the action planning was facilitated by the CRS EMPOWER Team Leader. Conclusions 1. Relevance 1.1 The project is responding with flexibility to the changes that happened during the pandemic. The project is in the 8th phase, which began in 2020. Given the long-term and far-reaching nature of the YEAC program, activities based on in-person interactions were very well established. External stakeholders, project staff, and project participants feel that the program has adapted well to the “new normal,” maintaining ties through virtual interactions (“We did virtual trainings”) and leaning on existing members to sustain the program. Staff mentioned that the curriculum was adapted during the pandemic to respond better to the reality: “the curriculum we adapted to reflect the disaster preparedness approach.” 11 Current members cited the work they continue to do as the pandemic evolves, highlighting the social ties that form the engine of YEACs: “During peak of covid, we lost a lot of members. Members are slowly coming back now. We individually called each one of them.” Though the project has successfully adapted, the project was affected by COVID and multiple staff members expressed the challenging environment and delays in implementation given stringent containment measures in each country. “The YEAC project is good but we have had some delays and challenges due to COVID.” 1.2 The project is engaging and unique, compared with other actors especially regarding the youth component. The project knows youth needs and interests but is not acting on them in some cases, likely due in part to lack of scope and budget. Overall, participants, staff, and external stakeholders feel that the YEAC program engages well with youth and is important to preparing youth to act as adults in the disaster risk reduction (DRR) space. “The youth were really vulnerable but through this program they feel empowered and much more educated and aware of the hazards.” However, the project feels limited in its activities, with a few respondents citing elements that they either hoped would be included or that had been eliminated but that they would like to see again: “Youth are faced with many challenges that the project does not address.” “’Job readiness’ is one of the trainings we’ve wanted to do but haven’t been able to this year – it would be virtual training.” “Add the edutainment components again. The youths loved that. DRR in the form of arts.” Many respondents cited limited financial and human resources as the main reason for the limitation in programming, represented by the following quote: “To some extent we are meeting the needs. Resources (financial and human) are not sufficient to meet all of needs. Additional human and financial resources would be needed for greater impact.” 1.3 The project could benefit from innovations related to curriculum and project activities. The project’s objectives are very relevant in all countries, however there were many ideas of improvements members, staff, and external stakeholders would like to see as an already successful program continues to grow: “We should include more components on raising awareness on impact of climate change especially with flooding.” “Assist schools and engage them more with drills (earthquake, fire, flooding etc...)” “Volunteerism is no longer relevant, and these youths need incentives to devote their time to the program especially since they do not have any source of income or job.” “Other skillsets are needed (how to prepare for an interview, how to become marketable with the DRR skillsets)” “Consider some trainings on Counseling services, Psychological First Aid, Radio Communication, Mental Health and Psychosocial support.” 12 Some partners have been implementing this program for 8 years while the most recent began implementation 3 years ago. There is variation in overall attitude towards the activities, with shorter-term staff and YEAC members expressing more enthusiasm for the program’s activities. A long-term respondent related that “the program can come off as boring and repetitive sometimes.” Another respondent noted that one barrier to achieving innovation could be the project targets, that targets sometimes became the focus of the small teams which took away time and space to innovate: “We are very focused on achieving targets rather than impact at times.” 1.4 The project is capturing informal feedback through consultation but could improve formal ways to document feedback or a confidential way to report feedback. Respondents in each country noted that there was a very close relationship between participants and staff, with multiple staff and members stating, “We are a family. YEAC is a family.” Staff described hearing and responding to feedback from youth members, and youth members reiterated that they feel heard by staff. However, the project could benefit from a formal way to document feedback and a confidential channel through which feedback may reach the team. A feedback system with multiple channels including an anonymous channel and a confidential channel is rapidly becoming a requirement across donor agencies. During the reflection workshop, the team noted that before COVID, the project had been working with community suggestion boxes but that COVID had made the travel necessary to securely manage the suggestion boxes impossible. The following quotes are examples of things that may have come through to project staff in real time in a more formal system: “There are a lot of needs in the communities that we are not able to meet such as social issues and mental health.” “The packages that are given to the family’s needs to be reviewed and readjusted. Include maybe (flashlights, small radios, batteries etc.)” 1.5 The project could specifically consider stronger partnerships with stakeholders to create employment opportunities in the DRR sector. For many youths, employment was both a priority and a worry. External stakeholders and youth in particular feel that the project could play a larger role in developing local DRR capacities through private sector connections, with one respondent noting, “Establish connection with the private sector.” Other respondents built on this idea, noting the importance of employment to the youth and the potential YEAC has as a project to support youth in the employment space. “Make members more employable with more appropriate skills. We could align them with DRR.” “Members are very satisfied but maybe in job opportunities and job placement they want more. They want the YEAC project to facilitate these opportunities a bit more.” “There seems to be some recruitment or retention challenges...perhaps finding ways to engage with stakeholders and create incentives for the youth could be something they could consider.” 2. Impact 2.1 The project transforms lives, even with limited resources. Overall, in every country, YEAC is a popular program among stakeholders, both internal and external. Youth continue to engage with YEAC, and many respondents cited tangible examples of changes that they witnessed, 13 such as a shy member in Dominica becoming a talented public speaker or a closed-off member in Saint Lucia who is now the first person to share information during meetings – evidenced by the following: “The transformation of youth in the program is felt and witnessed.” A long-term member discussed the role YEAC had in his/her life, stating that “Some of us came back and left but what we learn from the YEAC was good.” And another member highlighted the long-term commitment members feel toward the project: “Some of us have been members for 10-12 years.” During the Caribe Wave 2022 Tsunami drill in Grenada, the lead evaluator met and informally interviewed a long-term YEAC member who had had to step back from YEAC involvement due to a new job. However, even as he was no longer an official YEAC member, he continued to participate as he was able, with his new commitments, supporting the Caritas Grenada YEAC leadership in coordination of the Caribe Wave 2022 Tsunami drill along with NaDMA. 2.2 The project could consider geographic expansion in all countries. In many countries, there was an appetite among all stakeholders, especially members, to expand geographically to communities that have need but are not currently covered. “There are other communities that want to join but we are not in those places. The rural communities where there are needs are disconnected from the project. We need to find a way to expand and engage them.” Overall, respondents recognized the importance of the project’s activities, represented through the following quote: “Reach in this project is definitely important.” Many pointed to the Caribbean’s vulnerability to natural disasters and expressed anxiety about the preparedness of those communities that have not worked with YEACs, with many pointing specifically to the trainings for members and the family disaster preparedness plans as tools essential to emergency preparedness in all communities vulnerable to natural disasters. 3. Efficiency 3.1 The project should improve two-way communication especially around liquidations, submission of cash forecasts and reports that may lead to delays of payment. Project staff pointed to communication as a challenge internally. Though payments were specifically called out in the interviews through the following quotes: “sometimes there are delays which would prevent us from making payments on time” and “some delays in receiving the funds which has delayed project activities to a certain extent,” in discussion during the reflection workshop the team agreed that the delayed payments had improved in recent months but that overall, communication between all four organizations could be improved. During the discussion, the team highlighted that this recommendation addressing two-way or multi-way internal communication. 3.2 The project could develop a completion process to better assist YEAC members aging out. The age limit for YEACs (16-26) was a concern for both youth and staff, with a respondent noting that “the older persons older than 35 would like to be in it [YEAC] too.” Another respondent suggested that younger people would also like to participate, saying that there could be roles for both younger and older individuals outside of the interval of 16-26 years of age: “You could have a junior group – or a youth advisor.” There is an appetite for further coverage of the program through preparing youth for the age-out, with the popularity of the programming pushing stakeholders, including external stakeholders, to recognize the program’s potential in empowering youth for adulthood service in DRR: “The only drawback that I am seeing with the overall aspect of the program it has to define more clearly the pathway of a young person into 14 adulthood – when you are no longer considered in the age range, what is the hinge that takes you somewhere else.” 3.3 Communication products could be developed centrally and shared more widely in communities. The project should consider innovative ways to share communication products through social media and events. Members and staff would like to share the impact they feel their work is having more widely, with one respondent expressing “[the project] has impact but not visibility.” Another respondent suggested that YEAC “work on their visibility more to get persons to understand their work better.” An external stakeholder noted, “It isn’t very clear what the goals and objectives are as they [YEAC project] are not sharing widely with their stakeholders so that they [the stakeholders] all have an overall understanding in their [YEAC project] role in disaster management.” Many ideas were shared for how the YEAC project and each organization could support greater recognition of the program and its benefits, including the following: “Maybe an app, website or a page that shows all the different YEACs and their work and promote that app and share updates.” “Meet people where they are…they like to party. We could do a beach clean-up and ask everyone to wear a white shirt or something so it’s visible.” “We [could] send a blast message” “Aspect of visibility should be more proactive activities – rather than response activities.” 3.4 In some areas, recruitment would be more effective if the YEAC brand were stronger. YEAC members in particular felt that the project is in a position to improve brand recognition and that would aid in recruitment. YEAC members felt that much of the work they do is compelling but is not used to its full potential in recruiting members, evidenced by the following quote: “There is a need to document the success and the work of the organization…which would help with recruitment.” Youth members also reported being misidentified in some places as members of other active civil society groups. A respondent suggested taking advantage of the strong social ties YEAC fosters, and many members mentioned that they joined with friends or because of friends. “[The] social aspect is important to consider and include [for recruitment]. This is what attracts them. Ways to maintain face-to-face engagement or opportunities for them to contribute (beach cleaning, support to other stakeholders’ activities, cleaning gutters, etc.)” 3.5 The project could consider a conversation with the donor about revising the current budget and increasing the budget for the next phase, taking into account salary scales that consider cost of living and additional human resources. In each country, there were concerns about the budget. Staff members do not feel that the current budgeted salaries reflect the cost of living, do not feel that there are enough staff members to achieve the targets, and do not feel that the salaries are competitive enough to attract and retain qualified staff members. Insufficient funding impacts the possibility of geographical expansion: 15 “Funding is not appropriate to reach the needs of the communities we serve. Funding has significantly decreased over recent years and so we are not able to have the impact we once had. We even had to reduce the number of camps because of funding issues. We now have less YEAC members because we don’t have enough resources to expand.” Staff members also feel that the budget should be reviewed and revised, noting the following regarding retention challenges with current low salaries and volunteerism and the possibility of reviewing the training schedule to re-allocate resources to under-funded categories: “in certain areas I think the budget could improve...CERT training on an annual basis is expensive” “The budget needs to be consistent and aligned with the objective especially when it comes to the YEAC leaders. If we are asking someone to devote their time and train the Youth, there should be some sort of compensation for them, so they continue to value their contribution to the work.” “Volunteerism is no longer relevant, and these youths need incentives to devote their time to the program especially since they do not have any source of income or job.” 4. Coordination 4.1 The project is doing an excellent job of ensuring relationships that are locally relevant. Each partner ensures that they are known to the relevant stakeholders in each country. As seen in table 4, there are many similarities between countries around the type of organization (the disaster management authority, the Red Cross), however there are also locally unique relationships developed that help YEAC remain relevant in each country. The project staff is very aware of the importance of this, relating that: “Success is based on relationships.” Another staff member stated the fluidity of the relationships, noting that “the relationship with these stakeholders even go beyond the project.” 4.2 The project could integrate and involve other actors in a more structured, strategic, and active way. While the project fosters locally relevant relationships, some external partners do not feel involved with the program. They are aware of the program, but do not feel invested. This is illustrated by the external stakeholders who said, “Beyond the trainings we don’t have much of a relationship.” and “We do not have many details on the YEAC so wouldn’t be able to elaborate much.” From the YEAC project side, there is an appetite to participate in the larger DRR structure, with one member stating, “we want to participate in government things.” One respondent suggested that the project look for partnerships to complement the trainings currently being offered as a point for strategic partnership: “Look for partnerships that could complement some of the trainings that are being offered by the YEAC.” 4.3 The project coordinated a regional exchange between partners and staff, and would like to see this happen more often. There is a sense in each country that they are operating in silos. Many stakeholders, both internal and external, felt isolated (“YEAC is isolated in each island.” and “They are operating in silo.”) There was a recognition that this isolation was exacerbated by the COVID pandemic, however all agreed that the in-person events organized by the project were positive and should happen more often, but virtual events should be considered depending on the COVID context going forward. Someone brought up the idea 16 of a partnership exercise, facilitated by someone external in order to strengthen coordination and cohesiveness among the teams. “We should be looking for more ways and opportunities to include cross learning engagement amongst YEAC islands. The groups want to exchange amongst each other.” “…could be virtual” “More regional collaboration is needed particularly for the trainings. Other islands have interesting things that they are doing which could be shared and beneficial to the others. We should think about opportunities for the members to also exchange with the other members from other islands.” “Good to have a partnership exercise amongst the partners…maybe have an external person facilitate this exercise.” 5. Sustainability 5.1 The project is seeking to understand youth perspectives and to include their voice in the project. YEAC is lauded for engaging youth, capacitating youth, and creating a safe space for youth to productively develop life-saving skills (“The methods used by the youth is effective now and allow the project to meet its objective and gives a sense a structure and belonging to members.”) The project is youth-centered, with a YEAC member noting, “everybody has a voice.” The project is popular, with one respondent noting: “Hoping this program will continue as it is needed.” 5.2 The project equips youth with needed and marketable skills, but the youth require additional assistance to market themselves and/or the skills that they have learned. The project is responding to youth needs through adapting training offered (“We have added additional life skill courses for members which will help with employability and eventual job placement.”), however members are not always confident in applying those skills outside of YEAC or an emergency context, with a YEAC member asking, “Where can we use our skills outside of an emergency?” Stakeholders see this assistance as a core component of the mission of the project: “I believe that YEAC should be able to recommend these young people going into the work.” A positive example of applying skills was described in Grenada, in which YEAC partnered with the government to have YEAC members work with medical professionals as assistance in first aid during a football tournament. YEAC members were highlighted as part of the program and the football association is looking to further include YEAC members in its activities: “with the football association they’ve already highlighted the training for the YEAC team going forward.” 5.3 The project should consider strategically engaging relevant ministries in each country to improve pipelines to employment for trained youth as well as coordination for emergency responses. In line with other partnership recommendations and the recommendations shared by respondents to strengthen YEAC’s ability to support a pathway into adulthood, a respondent suggested that the project “consider strategic partnerships that can contribute to the projects objective and expand its reach and facilitate connections between the different emergency response actors,” with another respondent expressing that “YEAC isn’t very integrated at the parish level mechanisms.” An external stakeholder, in discussing YEAC’s role in regional and national emergency response, noted that “you need a framework, you need a structure for the long haul.” 17 Another respondent shared a vision of what an integrated YEAC may look like, acknowledging that this would be something that requires buy-in and initiative from larger structures: “Government could create policy or protocol for successful graduates of YEAC – getting a job or assignment within a national structure – you’d want to be able to prioritize these people – even over university – someone who comes from ground up who has practical experiences.” 5.4 The project could find ways to engage donors in future proposals to reintroduce components of the project that supported sustainability such as the DRR grants. Respondents look forward to future phases of YEAC and hope to include components that have been removed from previous phases such as the mitigation grants: “we have removed a lot of what made the project impactful (mitigation grants etc.). More livelihoods options should be considered in the program.” Respondents did not feel that the project should end yet, with one respondent saying, “I am afraid that if the project funding ended all their work would end too and there wouldn’t be any activities continuing.” Others discussed new ideas, such as the following, for future phases: “Consider including opportunities for the members to have access to small funding opportunities or mechanisms within the YEAC groups to implement some of the needs they have identified for their communities and reach greater levels of acceptance (early warning systems, small DRR project in the community, repairs to a shelter, hazard mitigation funding opportunities and others).” 18 Annex 1: Recommendations and the Action Plan # Theme Result/Conclusion Recommendations/Su ggestions Activities Date to complete activity Responsible Resources/Support desired to complete 1.1 Relevance The project is responding with flexibility to the changes that happened during the pandemic. Caritas Antilles - partner by partner review on timeline (check ins with each partner to see where they are and what they can do in this time) This is already happening, no need to discuss. - - - 1.2 Relevance The project is engaging and unique, compared with other actors especially regarding the youth component. The project knows youth needs and interests but is not acting on them in some cases, likely due in part to lack of scope and budget. Within scope and budget - edutainment, next phase proposal budget - job readiness. Other human resources other geographic areas? 1. Discussion with all partners on how to include and integrate job readiness and life skills and an edutainment component into awareness raising; upload recommendatio ns from evaluation to the ART platform and schedule a meeting with the donor (prepare meeting, discuss what we want to change 17-May-22 Needs assessment conducted by June 13, 2022; Analysis and setting up conversation with donor by July 5, 2022. Cynthia Fontenelle (Partner discussions); Each Finance Officer (prepare budgets and ___leads donor meeting preparation) *N/A, will use current regional space for this discussion 19 (prioritize, be clear on justifications for expansion) and how much more budget different scenarios would need) 2. job fairs with our stakeholders in the area of disaster preparedness 3. Needs Assessments to justify expansion/progr am edits Cynthia Fontenelle Proann and all PMs for each island, CRS support in formulating needs assessment template, Review of analysis 4. Budget for needs assessment PMs in each country Potential resources needed in travel/logistics and time of PMs 1.3 Relevance The project could benefit from innovations related to curriculum and project activities. Map other services available in other social services in order to develop a referral mechanism. Some are already doing this (i.e. life skills courses for members); Within scope and budget, look for things (i.e. raising awareness of 1. Create service map (who does what, contact numbers (i.e. a directory)); use checklist for each service provider, produce pocket guide to services 29-Apr-22 Mahalia Paul (complete template to share with all partners) Daniela Suarez (to discuss with Abdiel service mapping examples) 20 climate change) and look to begin conversations with donors for what is not within this scope (i.e. job readiness). Look for strategic partnerships in communities so that they could start some of this now (i.e. qualified community trainers?, plan joint events, start discussions about job fairs/job pipelines, find synergies) 2. Identify and meet with environmental organizations working on climate change to explore synergies 30-Apr-22 Marcia Haywood Identify YEAC member who is interested in climate change (climate change champion?) to support 3. Create a training around job readiness that would include resume writing, interview preparation and etiquette (incl. attire) 6-May-22 Dwayne Francis (will share curriculum materials with whole network to adapt) 21 4. Develop curricula (training guide) to address certain areas - such as conflict resolution 6-May-22 Proann Francis Marcia Haywood general support, Rose-Anne Redhead, Anna/Daniela to share conflict resolution examples 1.4 Relevance The project is capturing informal feedback through consultation, but there is no formal way to document feedback nor a confidential way to report feedback. Lack of feedback systems to be able to gauge needs - build a feedback system that accepts anonymous feedback (system to classify sensitive v. not sensitive feedbacks - and pathways to channel those) 1. Workshop/work ing session - go over basic components of a system, go over SOPs for each country, need dedicated time for PMs to do these things; in-person Week of June 27 Daniela Suarez, Abdiel Cabrera (CRS)/Cynthi a Fontenelle Diane to share SVG SOPs 1.4 Relevance The project could specifically consider stronger partnerships with stakeholders to create employment opportunities in the DRR sector. within scope and budget - establish private sector connection now, bring concrete proposals to the government, look at more specific MOUs for youth with government stakeholders for responses, develop SOPs for that; for next phase, identify job opportunities, help youth connect with them, look at training gaps, engage private sectors for internships 1. Mapping exercise of potential employment opportunities or internships with DRR stakeholders April 30, 2022 (Template); June 17, 2022 (complete mapping, all islands) Proann Francis, develop template Daniela Suarez, provide templates (virtual survey?) 22 2.1 Impact The project transforms lives, even with limited resources. No action needed - - - 2.2 Impact The project could consider geographic expansion in all countries. With additional scope and budget, expand curriculum, activity, and geographically. Already covered - - - 3.1 Efficiency The project should improve two￾way communication especially around liquidations, submission of cash forecasts and reports that may lead to delays of payment. Consider a process for budget revision if the budget is not relevant, multi-way communication about delays; consider acreditation for in￾house training 1. Each partner develop ideal budget, keeping in mind cost categories (10% flexibility between cost categories, for more need to discuss with AOR) 13-May-22 Cynthia Fontenelle, Lintha Smith Will reach out if needed 3.2 Efficiency The project could develop a graduation process to better assist YEAC members aging out. Define a pathway for graduating youth (leaders with stipends for next phase?), junior group? Youth advisor?, job preparedness?, increasing leadership roles? Current phase - recognition ceremony, defining role of youth advisors clearly;;Next phase - Establish clear selection criteria, establish clear pathway for how we will select and when leader roles, determine stipend we would want to propose to BHA (Two components - 31-May-22 Cynthia Fontenelle + PMs None 23 pathway; stipend youth leaders and community service roles within the disaster response structures (once they've completed YEAC program)) 3.3 Efficiency Communication products could be developed centrally and shared more widely in communities. The project should consider innovative ways to share well-produced communication products through social media and events. Hold community events, further develop social media messaging, consider SMS messages, one pagers on project for external stakeholders, quarterly newsletter - centrally focused that includes success stories and testimonials from quarterly reports Current phase: update pages in domain - posting recent and upcoming activities, one pager;; Next phase: Hire dedicated communications person, increase budget for visibility May 6, 2022 - create schedule of social media; May 30, 2022 - one-pager Proann Francis, create schedule of social media; Cynthia Fontenelle - one-pager PMs, community promoters; Daniela Suarez (CRS) to send template for one￾pager 3.4 Efficiency In some areas, recruitment would be more effective if the YEAC brand were stronger. ID cards, flyers for recruitment, share local YEAC logos Current phase: create Google photo database - each country shares photos and logos and template for a flyer for recruitment - regional approach;; Next phase: include budget for ID May 31, 2022 - Flyer Cynthia Fontenelle and Marcia Haywood (Flyer + google drive) budget 24 cards (badges that identify members on accomplishment with regards to training - like scouts) and flyer printing 3.5 Efficiency The project could consider a conversation with the donor about revising the current budget and increasing the budget for the next phase, taking into account salary scales that consider cost of living and additional human resources. Review salaries based on cost of living, include more stipends for youth leaders (linked to completion process); review transportation budget lines (Transportation budget is not sufficient to meet current needs); additional budget for next phase, revision of current budget. Current phase - each team submit proposal to CA through email based on financial reports (looking at increasing budget on transportation where applicable);; Next phase: define method through budget review for next phase (specifically around salary scales) Financials are due May 5 (April expenses); Requests to Lintha by May 5 (request for revision) Lintha Smith PMs to request 4.1 Coordination The project is doing an excellent job of ensuring relationships that are locally relevant. - No action needed - - - 4.2 Coordination The project could integrate and involve other actors in a more structured, strategic, and active way. Create a more strategic relationship with the government, identify opportunities for YEAC visibility, for the next phase - Stakeholder mapping (incl collab groups), stakeholder engagement schedule - key May 2, share template; May 31, mapping complete Rose-Anne Redhead to coordinate Proann Francis to share template, Viviene Morris 25 involve government in design workshop workshops for collaboration beween agencies & YEAC (beginning of project and/or throughout project) 4.3 Coordination The project coordinated a regional exchange between partners and staff would like to see this happen more often. There is a sense in each country that they are operating in silos. Look at more virtual exchanges, for next phase look to budget for in-person exchanges Island-level trivia tournaments - regional level championship - meeting with each PM, PAs, Community Promoters to develop island￾level tournament June 10, 2022 - target for completing all island-level tournaments; Regional tournament - August 31, 2022 Viallai Edward For local tournament - prizes, live streaming, food, transportation (?);; For regional tournament - resources for travel (and lodging) and prizes, live streaming, food 5.1 Sustainability The project is seeking to understand youth perspectives and to include their voice in the project. - No action needed - - - 5.2 Sustainability The project equips youth with needed and marketable skills, but the youth require additional assistance to market themselves and/or the skills that they have learned. linking to rapid response funds (ELMA) for employment opportunities, showing youth where to apply these skills (humanitarian sector), review budget for eventual emergencies to move YEAC members between islands to support responses (i.e. when Finding 1.4 - employment mapping relevant here, Request ELMA rapid response funds - Proann (as in 1.4) - 26 St. Lucia & Grenada members went to Dominica) 5.3 Sustainability The project should consider strategically engaging relevant ministries in each country to improve pipelines to employment for trained youth as well as coordination for emergency responses. Including government in next design workshop? Already covered - - - 5.4 Sustainability The project could find ways to engage donors in future proposals to reintroduce components of the project that supported sustainability such as the DRR grants - small grants for communities that serve as incentives to complete activities (i.e. drainage). Prepare illustrative list of types of projects that could be included (CRS can support) and how many could feasibly be accomplished within a project period and approximate maximum budget per project (has been up to USD$6,000 in the past, but needs to be discussed) Prepare justification (why we want to include - what impact have we seen in the past, type of leverage it creates), prepare approximate budget - number of projects and budget per project, prepare an illustrative list of projects that could be projects 31-May-22 Dwayne Francis CRS - Anna Hrybyk 27 28 Annex 2: PowerPoint Presentation of the findings YEAC VIII Midterm Preliminary Findings P 29 Annex 3: Sources of Data and Data Collection Methods From the Evaluation Statement of Work submitted with application by Caritas Antilles. Data Collection Method and Frequency Key variables or areas of inquiry to collect to meet the objective Data Sources (Respondents) Sampling Method and Sample Size Evaluation objective(s) and question(s) assessed with this method – Section D above Desk review • Number of people trained in disaster preparedness, risk reduction and management • Number of people passing final exams or receiving certificates • Percentage of people trained who retain skills and knowledge after two months • Number of people trained in First Aid, Search and Rescue, or health related Disaster Risk Reduction activities • Number of people trained in Leadership and Life skills Project reports; IPTT and Monitoring Table Training tracking log n/a I. To verify that project progress is in line with reported/expected project planning and outcomes. II. To identify different approaches that partners are undertaking to realize deliverables, and whether approaches/methodologies are contributing to planned capacity strengthening outcomes III. To review the strengths and weaknesses of YEAC’s approach to date and to identify ways to strengthen it. 1. Relevance - are the specific activities and outputs of the program consistent with the YEAC overall goals and indicators 2. Effectiveness – 2(b) How could the project be managed more effectively?7 4. Impact – 4 (a) what has happened as a result of the project; 4 (b) what real difference has the activity made to the beneficiaries; 4 (c) how many people have been affected. Focus Group Discussion – Program Participants Learning and synergies related to capacity strengthening, collaboration YEAC Members 1 focus group with 12 program participants III. To review the strengths and weaknesses of YEAC’s approach to date and to identify ways to strengthen it. VI. to analyze the importance of the emergency response and leadership and life skills trainings to the youths’ self-development and the development of their community 4. Impact – 4 (a) what has happened as a result of the project; 4 (b) what real difference has the activity made to the beneficiaries. 5. Sustainability – 5 (a) to what extent are the benefits of the project likely to continue after OFDA funding ceases. Key informant interview – Learning and synergies related to capacity strengthening Stakeholders 2 to 3 main stakeholders per country VII. To gain an appreciation of organizations’ emergency response mechanisms and possible gaps in general emergency response 7 Please note that effectiveness was re-interpreted as coordination – how effective is the current coordination for the objectives of the project? 30 Data Collection Method and Frequency Key variables or areas of inquiry to collect to meet the objective Data Sources (Respondents) Sampling Method and Sample Size Evaluation objective(s) and question(s) assessed with this method – Section D above Stakeholders (participating organizations) VIII. To understand stakeholders’ perspective on volunteerism and the role of youths in emergency response. 2. Effectiveness – 2(b) How could the project be managed more effectively? 4. Impact – 4 (a) what has happened as a result of the project 5 Sustainability - 5 (b)what major factors may influence the achievement or non-achievement of sustainability of the program or project. Key informant interview – Project Staff Learning and synergies related to capacity strengthening Project Staff 3 project staff per country II. To identify different approaches that partners are undertaking to realize deliverables, and whether approaches/methodologies are contributing to planned capacity strengthening outcomes. III. To review the strengths and weaknesses of YEAC’s approach to date and to identify ways to strengthen it. IV. To identify lessons learned and adaptation for the remainder of the project period and for future programs. V. To identify how the YEAC project team has worked together within and across country program. 1. Relevance - are the specific activities and outputs of the program consistent with the YEAC overall goals and indicators 2. Effectiveness –2(a) to what extent were the stated objectives achieved and what were the major factors influencing the extent of achievement of the objectives; 2(b) How could the project be managed more effectively? 3. Efficiency - were activities cost-efficient; were objectives achieved on time; was the project implemented in the most efficient way, compared to potential alternatives. 4. Impact – 4 (a) what has happened as a result of the project; 4 (b) what real difference has the activity made to the beneficiaries; 4 (c) how many people have been affected. 5. Sustainability – 5 (a) to what extent are the benefits of the project likely to continue after OFDA funding ceases; 5 (b) what major factors may influence the achievement or non-achievement of sustainability of the program or project.