QUARTERLY PROGRESS REPORT HAITI EVALUATION AND SURVEY SERVICES (ESS) PROJECT Quarter 1, Fiscal Year 2022 (October – December 2021) Submitted January 31, 2022 Contract Number: AID-521-C-17-00002 DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of Social Impact, Inc. (SI) and do not necessarily reflect the views of USAID or the United States Government. This document was produced for review by USAID. It was prepared by SI for the USAID/Haiti ESS Project. CONTENTS ACRONYMS II INTRODUCTION 1 2 PROJECT STATUS REPORT 2 PROJECT DELIVERABLES 2 TECHNICAL TASKS 2 TECHNICAL TASKS UPDATE 2 ANNUAL WORK PLAN 4 PLANNED TASKS FOR NEXT QUARTER 5 IMPLEMENTATION CHALLENGES AND SOLUTIONS 5 KEY ACCOMPLISHMENTS 6 i | USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT USAID.GOV ACRONYMS AREA Appui à la Recherche et au Développement Agricole CDD Community Driven Development CLP Chanje La Vie Plante CO Contracting Officer COP Chief of Party COR Contracting Officer’s Representative DEC Development Experience Clearinghouse DFA Director of Finance and Administration ESS Evaluation and Survey Services ET Evaluation Team FCR Findings, Conclusions, and Recommendations FGD Focus Group Discussion FY Fiscal Year GOH Government of Haiti HQ Headquarters IAHBP Increased Accountability in Haiti’s Budgetary Process IFOS Institut de Formation du Sud IP Implementing Partner KII Key Informant Interview MOE Ministry of Education OAA Office of Acquisition and Assistance Q Quarter RERA Rapid Education Risk Assessment Santé Service Access, Nationwide, Technology, and Empowerment SI Social Impact, Inc. SOD Statement of Difference SOW Statement of Work TL Team Lead UF University of Florida USAID United States Agency for International Development WATSAN Water and Sanitation USAID.GOV USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT | ii INTRODUCTION This report documents Haiti Evaluation and Survey Services’ (ESS) activities, including achievements and challenges for Quarter 1 (Q1) of Fiscal Year (FY) 2022, covering project activities between October 1, 2021, and December 31, 2021. 1 | USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT USAID.GOV PROJECT STATUS REPORT PROJECT DELIVERABLES ESS submitted its FY21 Q4 Quarterly Progress Report on October 29, 2021. In addition, on December 13, 2021, ESS submitted a quarterly Accrual Report, including information on ESS’s expenditures, disbursements, and accruals. TECHNICAL TASKS TECHNICAL TASKS UPDATE During FY22 Q1, ESS continued working on the several ongoing tasks from the previous quarter, including the Santé evaluation, Haiti Rapid Education and Risk Analysis (RERA), the AREA (Appui à la Recherche et au Développement Agricole) final evaluation, the ATTEINDRE/ATTAIN activity baseline, the Community Driven Development (CDD) mid-term evaluation, and the Water and Sanitation (WATSAN) activity final evaluation. ESS details the progress and challenges related to each task below. SANTÉ MID-TERM EVALUATION ESS submitted the final Santé report on May 7, 2021, addressing all of USAID’s comments. USAID informed ESS that the Agency elected to submit a statement of difference (SOD) on the evaluation. USAID submitted the SOD on August 26, 2021. A meeting was organized with SI, USAID’s Program Office, and Health Team on September 24, 2021 to discuss the best way forward. ESS began preparing its response to the SOD at the end of September, which was submitted on October 15, 2021. ESS awaits USAID’s advice further actions. HAITI RAPID EDUCATION AND RISK ANALYSIS USAID approved the English version of the final report on September 24, 2021. During FY22 Q1, ESS submitted the French version of the out-briefing presentation alongside the English and French briefers to USAID on October 15, 2021, as dissemination materials for the stakeholders. ESS submitted the French version of the final report on October 19, 2021. During the weekly ESS-USAID check-in on December 10, 2021, USAID advised ESS to go ahead and prepare the approved final report for submission to the Development Experience Clearinghouse (DEC) and the final file to USAID. ESS awaits the schedule for the out-briefing presentation with the Ministry of Education (MOE). In the USAID.GOV USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT | 2 meantime, USAID provided the MOE’s feedback on the French version of the final report on December 15, 2021. At the end of FY22 Q1, the RERA team was addressing the MOE’s feedback on the French version of the final report. ESS awaits the schedule for the out-briefing presentation with the Education Working Group. AREA EVALUATION The evaluation team (ET) started the qualitative data collection on October 7, 2021. Meanwhile USAID coordinated the list of 2,000 short-term trainees for the survey component with the University of Florida (UF), the ET and ESS and continued to help gather contact information and followed up with the UF to identify missing contacts. These efforts resulted in a sampling frame containing only 751 short￾term trainees. As advised during the biweekly call, the ET revised the sample size based on the new total of 751 and submitted the revision on December 3, 2021, which USAID approved on December 7, 2021. The survey tools were translated from English to French and Creole and translated back into English before they were pretested. The qualitative data collection was completed on December 8, 2021, while the surveys were done on December 18, 2021. It is worth noting that data collection took longer than usual for the AREA evaluation because of difficulties in getting the short-term trainees’ list and accurate contact information. At the end of FY 22 Q1, ESS had started data coding, data processing, and planning for data analysis. ATTAIN/ATTEINDRE During FY22 Q1, IFOS was onboarded and secured 128 enumerators in the 64 commune sections. ESS submitted a revised timeline to USAID on October 14, 2021. ESS shared a detailed Listing Plan with USAID on October 19, 2021. IFOS organized an orientation for enumerators on October 20, 2021 and piloted the listing exercise in the south and the north on October 21, 2021, for which the Assistant Team Lead and ESS staff were present for. Debriefing for the pilot took place on October 22, 2021 among the pilot team. The ET and ESS met with IFOS on October 29, 2021 and discussed the analysis of the pilot data and the way forward. IFOS was advised to provide further training/orientation to their team and conduct a second pilot of the listing prior to launching the full listing exercise. The second orientation for the listing team was held on November 5-6, 2021. The second piloting followed on November 8, 2021. The out-briefing with IFOS and ESS took place on November 10, 2021. The actual listing exercise started on November 12, 2021 and ended in most of the communal sections on November 24, 2021, except in the four “big” cities—Cap-Haitian, Les Cayes, Ouanaminthe, and Hinche—where it was completed on December 8, 2021, because the ET and ESS had instructed IFOS to continue after revising the data. In the week of November 22, 2021, the ET onboarded the notetakers and planned for the KIIs, and started the KIIs the week of December 6, 2021. IFOS shared the complete listing database with ESS on December 15, 2021. ESS submitted the revised timeline to USAID on December 7, 2021, which the Agency has approved. The biweekly calls were scheduled regularly, which provided the platform to address challenges and confirm a shared understanding for agreed-upon next steps throughout this period. The ET started translating the interview script and interviews with the key informants on December 10, 2021, and were still in progress at the end of FY22 Q1. 3 | USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT USAID.GOV The ET finalized translations of the survey and IFOS uploaded a version onto the digital platform. Using stratified data to measure the impact of enterprise income-related questions from the listing, IFOS conducted a pretest of the questionnaire on December 17, 2021. Since receiving listing data, the ET worked to identify key stratifying variables to construct the survey sample and expects to finish by the end of the FY22 Q1. WATSAN ACTIVITY FINAL EVALUATION USAID shared the final SOW on September 22, 2021. ESS submitted the Work Plan on October 21, 2021, and the consultant rate approval package for the TL and Assistant Team Lead was submitted on October 26, 2021. USAID approved the ET members on November 16, 2021, and shared activity documents including an introduction letter with ESS. The ET and the WATSAN team at USAID and DAI, the implementing partner (IP), have had several discussions on the mWater platform. The kick-off meeting between the ET, USAID, IP, and ESS took place on December 15, 2021. At the end of FY22 Q1, the ET members began the desk review in preparation for the in-briefing meeting. CDD ACTIVITY USAID shared the draft SOW for the CDD mid-term evaluation on August 20, 2021. A technical meeting between USAID and ESS was organized on August 30, 2021. ESS provided feedback on the draft SOW on September 3, 2021. USAID provided the final SOW on September 17, 2021. ESS submitted the Work Plan including the ET members for USAID feedback/approval on October 8, 2021. On October 20, 2021, USAID approved the suggested ET members and the Work Plan, contingent on changing the methodology to a mixed method approach. ESS submitted the revised Work Plan the same day. ESS submitted the consultant rate approval request for TL candidate Xavier Bardou on October 21, 2021, which was approved by the Office of Acquisition and Assistance (OAA) on November 16, 2021. While onboarding the ET, ESS learned that the approved TL operates as an LLC based in Hong Kong. ESS submitted a Request for Waiver of Source/Nationality requirements for Mr. Bardou on November 29, 2021, but OAA denied the request for a waiver to sub-contract Mr. Bardou. ESS submitted a new TL candidate, Isabelle McMahon, on December 16, 2021. At the end of FY22 Q1, ESS was waiting for USAID technical concurrence upon which ESS would issue her a firm-fixed price sub-contract well below the threshold required to seek consent to sub-contract through her U.S.-based LLC. ANNUAL WORK PLAN As a result of USAID internal consultations and prioritization, the ESS Contracting Officer’s Representative (COR) shared an official list of desired Year 5 Work Plan activities with ESS. WATSAN and CDD are the two new evaluations to be added to the ongoing tasks. Relevant draft SOWs were shared with ESS in August 2021 to ensure completion before the end of the ESS contract. However, these tasks were initially anticipated to be conducted through May 2022. On October 6, 2021, ESS submitted a request for a one month no-cost extension as well as a budget realignment considering the activity end date and current budget ceiling. On October 25, 2021, USAID provided salary adjustment approval for ESS staff included in the submitted budget realignment package. Further, on November 9, 2021, USAID provided approval of the no-cost extension and Year 4 and Year 5 fees as well as severance pay for the local staff. USAID.GOV USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT | 4 All the tasks are launched and ESS submitted the Year 5 Work Plan on October 29, 2021. Due to delays in the implementation of the WATSAN evaluation and approval of the Team Lead for the CDD evaluation, these two evaluations are now anticipated to be concluded by August 2022; ESS had to revise the request for budget realignment and no-cost extension accordingly. ESS submitted the revised budget realignment and a three-month no-cost extension to USAID on December 16, 2021. ESS awaits USAID’s feedback or approval as well as a contract modification reflecting the approvals provided to date. PLANNED TASKS FOR NEXT QUARTER In FY 22 Q2, ESS intends to finalize and upload the final Santé and RERA reports to the DEC, contingent on USAID approval, and present the Haiti RERA results to the Education Working Group including the MOE. In addition, ESS intends to submit draft reports for the AREA and ATTAIN activities. ESS will also continue working on the WATSAN and CDD activities. ESS will start implementing its close-out plan contingent on USAID approval. I 5 | USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT USAID.GOV KEY ACCOMPLISHMENTS During FY22 Q1, ESS had several key accomplishments to highlight. • The ATTAIN Listing Exercise for the ATTAIN Baseline evaluation and the data collection for the AREA final evaluation were completed. − The Listing Exercise for the ATTAIN Baseline: During the week of December 13, 2021, the listing activity ended. IFOS completed data cleaning and provided the ET with a final copy of the listing data, containing 24 observations, on December 15, 2021. − Data Collection for the AREA final evaluation: Overall, data collection went well despite hardships such as lack of farmers’ lists, lack of project staff in the field, fuel shortages, and communication difficulties in Haiti. A thorough cleaning of the database and removing all duplicates revealed that the trainees' list had only 751 unique training participants. Therefore, the sample size for short term trainees was adjusted to be 442 using the formula included in the design report. Overall, the ET completed the KIIs (58), FGDs (12), and the short-term (446) and the long￾term trainees (16) surveys. Data collection is complete; the ET focused on data coding and cleaning and plans to identify all the critical responses obtained under each evaluation question. At the end of the exercise, the ET will know what reasonable inference could be based on the responses obtained from the different stakeholders. Data obtained with the qualitative tools will be combined with the results from the quantitative ones to make a final determination concerning each evaluation question. • All other tasks assigned to ESS (Santé, RERA, AREA, ATTAIN, CDD, and the WATSAN final evaluation) are moving forward despite major health and security concerns such as COVID-19, USAID.GOV USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT | 6 kidnappings, and fuel shortages. 7 | USAID/HAITI EVALUATION AND SURVEY SERVICES REPORT USAID.GOV