Ethiopia Development Food Assistance (DFAP) Commodity Management Review Final Report November 16, 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared for USAID/Ethiopia for contract AID-603-TO-16-00002, by International Business and Technical Consultants, Inc. (IBTCI). Authors include: Faheem Khan, Team Leader, Mesfin Beyero, Temesgen Adnew Burka, Benti Erena, Solomon Woldeab and Temesgen Wario, and Katherine Labombarde. USAID/ETHIOPIA DFAP COMMODITY MANAGEMENT REVIEW FINAL REPORT December 5, 2016 Original Submission: November 16, 2016 AID-603-TO-16-00002 International Business & Technical Consultants, Inc. (IBTCI) 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. International Business & Technical Consultants, Inc. i Table of Contents Acronyms...........................................................................................................................................................ii Glossary .............................................................................................................................................................iii Executive Summary................................................................................................................................. 1 1. Background and Problem Statement.......................................................................................... 5 2. Methods .............................................................................................................................................. 6 3. Findings .............................................................................................................................................10 Efficiency in Distribution.........................................................................................................................10 Food Management Practice and Internal Control Systems....................................................................20 Record-keeping and Reporting ...............................................................................................................27 4. Conclusions ......................................................................................................................................30 5. Recommendations..........................................................................................................................32 Annex A: Scope of Work from the USAID Request for Task Order Proposals .............................36 Annex B: Team and Timetable ..........................................................................................................42 Annex C: Warehouse Questionnaire................................................................................................46 Annex D: Questionnaire Employed at Food Distribution Points .....................................................52 Annex E: End-Use Beneficiary Questionnaire ...................................................................................55 ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. ii Acronyms CRS Catholic Relief Services DFAP USAID’s Development Food Assistance Program DPPO Disaster Preparedness and Prevention Office DSB Direct Support Beneficiaries GoE Government of Ethiopia FGD Focus Group Discussion FDP Food Distribution Point FIFO First-In-First-Out HCS Hararghe Catholic Services IP Implementing Partner Kg Kilogram KII Key Informant Interview MBL Master Beneficiary List MCS Meki Church Secretariat MT Metric Ton NGO Non-Governmental Organization PLW Pregnant and Lactating Women PSNP Productive Safety Net Program QWICR Quarterly Web-Interfaced Commodity Reporting REST Relief Society of Tigray SC/SCI Save the Children International SoW Scope of Work TL Team Leader US United States USAID United States Agency for International Development WFP World Food Program YSP Yellow Split Peas Glossary of Key Terms Kebele – The smallest administrative unit in Ethiopia, a ward or neighborhood Rubhall – A large, relocatable tent-like structure often used in humanitarian emergencies Woreda – A District of Ethiopia ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 1 Executive Summary The USAID Development Food Assistance Program (DFAP) is a significant mechanism through which USAID supports the Ethiopian Government’s Productive Safety Net Program (PSNP) to achieve sustained changes to food security and the livelihoods of the chronically poor and reduce their vulnerability to hazards, disasters, and malnutrition. During the period under review, Fiscal Year (FY) 2015 and FY 2016. USAID’s DFAP has been implemented by Catholic Relief Services (CRS), Save the Children, Food for the Hungry, and the Relief Society of Tigray (REST). This review was commissioned to review the commodity management and operations in the Oromia and Somali regions by CRS and Save the Children. The findings of the evaluation are framed against the key research questions as set forth by USAID, from which the three primary research questions were designed to address: • Efficiency in Distribution of Food Aid • Food Management Practices and Internal Control Systems • Reporting The review team found that the two DFAP Implementing Partners (IP) managed and accounted for commodities in accordance with Regulation 11. Save the Children developed a strong working relationship with the GoE and has been actively engaged in the process of beneficiary selection under the leadership of the Kebele Task Force, a governmental entity in each local area comprised of a few villages. They were also a member of the Appeals Committee in case of disputes. Although CRS guidelines recommended that its Ethiopian partners under the Meki Church Secretariat (MCS) participate in the targeting process by working with the Kebele and Woreda Task Forces, responses from partner staff in all CRS Food Distribution Points (FDPs) reviewed and indicated that they did not participate in targeting efforts. To ensure commodities were channeled to eligible public works program beneficiaries, the Woreda food security desk prepared the monthly Payroll List and provided these to implementing partners (IPs). Eligible beneficiaries presented their Client Card at the FDP, where the distribution staff verified their eligibility to receive commodities by confirming their names on the payroll list. Direct support beneficiaries appeared on a separate direct support list and were eligible to receive commodities with no work having to be performed. Save the Children adopted an additional internal control system by referencing the Master Beneficiary List (MBL), thereby enabling a further verification of eligible beneficiaries between the Payroll List (PL) and the MBL. Direct support beneficiaries were aimed at people unable to work (disabled, aged) and the extreme poor. Based on a review of both the distribution of FDPs and the non-distribution FDP seen in the Sire Woreda, the review team discovered that CRS beneficiaries were not issued Client Cards that should have been required for the collection of rations. When asked if they had Client Cards for the regular DFAP commodities under the response from beneficiaries was negative. This may have posed a risk as food aid distributors were unable to verify the identification and eligibility of beneficiaries. Instead, they were expected to distribute the appropriate commodities based on presentation of a paper issued by the Woreda describing the specific commodities and amount the bearer was entitled to. Across the majority of FDPs reviewed, the review team found that the time devoted by beneficiaries for the collection of rations was lengthy and needed to be more efficient. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 2 Additionally, on average, the distances beneficiaries were required to travel to reach FDPs indicated that several of the FDP locations were difficult to reach. Community participation in determining more accessible FDP distribution points could have been strengthened and would have resulted in less disruption and discontent among them. More importantly, the efficiency of commodity distribution could have been improved with better planning. As an alternative, rather than scheduling all beneficiaries to come to the FDP at the same time, the distribution process could have been staggered to reduce waiting times and facilitate more efficient distribution activities. For example, family and individual rations could have been split into different sessions. Waiting times were inordinately long due to the practice of holding the distribution until all beneficiaries were in attendance. This was surprisingly documented universally, but rationalized to organize into group distributions (bulk rations to groups of households). The team visited, observed and evaluated 18 different FDPs. None were seen to include basic provisions such as drinking water, shade to protect from sunlight and rain, or latrines. When considering the long wait times, and the distances beneficiaries were required to travel to reach these sites, FDPs needed to provide these basic amenities, and consider gender sensitivities, such as providing gender segregated latrines, rest areas for pregnant women, and lactation points. Interviews with beneficiaries and observations by the team found that beneficiaries of rations provided by Save the Children were largely satisfied with the received size of the ration. For CRS, it was found that distributors were not using the standard scooping measures provided to them for accurate measurement of the Contingency ration package. This tax or fee may have been a factor leading to many of the CRS beneficiaries complaining that they had not received their fair share of commodities, primarily pulses. A critical concern uncovered by the review team was that the majority of beneficiaries met with reported that they were required to pay 2-3 Birr (equivalent to between $US 0.09-$0.13) when receiving rations. This was collected by village leaders to pay for labor hired to move commodities, and for guards to protect commodities. Further, in the Ibseta FDP in Sire Woreda, where CRS beneficiaries were required to pay 5 Birr ($0.22) at each distribution as a contribution to a savings account. The review team found no evidence of this savings account and the parent organizations of Save the Children and CRS had not condoned these fees. For commodities distributed during FY 2015 the review team observed that all signboards at FDPs displayed the correct information and were visibly branded, including the commodities and amounts to be provided, together with the USAID logo. However, apart from the three FDPs reviewed for the Dhas woreda, and another FDP in Yabello Woreda, managed by Save the Children, all other FDPs for both IPs continued to display the ration mixes of the regular PSNP/DFAP and did not display the updated information for the new “transfer of food as additional month” allotment through the same IPs. The three FDPs in Dhas, and the Elwoya Magala FDP in Yabello, developed an innovative approach where they prepared an updated sign board printed on cloth, which they draped over the DFAP signboard at the time of distributions. Although the USAID logo was visible, there was a widespread perception shared among many of the beneficiaries interviewed that the donor was either the GoE, or the United States (US) Catholics, or the IP themselves, in tandem with USAID. Accordingly, a more well thought out ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 3 communication strategy would ensure that beneficiaries recognized that these commodities were a gift from the American people. With respect to warehouses, the review team found commodities were well managed and stored, when looking at both IPs. Warehouses were appropriately located, situated to allow easy access for trucks, fitted with wide doorways for easy transfer of commodities into and out of warehouses, and provided adequate space for reconstitution of commodities. The construction featured solid floors, walls and ceilings protecting from weather and pests. The only exception was the CRS rubhall in Dire Dawa that was found to be aged and in need of extensive renovation. All warehouses reviewed, including both primary and secondary, were equipped with the necessary equipment to adequately manage and protect commodities including weighing scales, rat traps, and communication systems. Fire extinguishers were maintained in all warehouses; and, guards demonstrated knowledge about their use. Other fire extinguishing material such as buckets, access to water, and sand were also present. All warehouses were observed to offer 24-hour security – either by directly employing and managing security guards, or hiring the services of external security firms. The one exception to this was the CRS warehouse in Dire Dawa where the landlord arranged security. Commodity storage was efficiently performed by two IPs following USAID’s Regulation 11. Commodities were stacked one meter between and away from walls and ceilings; and with plastic sheets and pallets used for stacks less than 25 levels high and with different shipments stacked according to dates adopting the first-in-first-out (FIFO) approach. Records indicated that regular fumigations were performed. “Reconstitution” is a common, required activity for repacking bags among those that have torn or broken, with correct weights ensured. CRS and Save the Children also use the term “reconditioning.” The review team observed that damaged bags were disaggregated and stacked separately in all warehouses reviewed under CRS and Save the Children. Warehouse staff who were interviewed at both IP’s warehouses stated that reconstitution was performed as per specific commodity guidelines. As per the IP guidelines, the review team found evidence from records that for both CRS and Save the Children, commodity supervisors provided advance notice to store keepers regarding the type and quantity of commodities, date of dispatch from central warehouses (e.g. Adama), and arrival, and the commodity shipment number. Records confirmed that the store keepers counted bags to verify that deliveries matched properly with the authorization documents, and against the completed and signed the waybill. Although not a requirement as per programmatic management guidelines, Save the Children did perform weighing a sample of bags which appeared to be damaged, and in some warehouses weighed every sampled twentieth bag. The review team considered this to be an effective initiative considering commodities are vulnerable during transportation where risks of pilferage are high. In Ethiopia NGO experience is that it tends typically to be only a portion of the bag that is stolen, not the whole bag. Both counting and weighing a sample of bags provides further security to minimize losses from such potential thefts, albeit after the fact. Both IPs have invested in their management and operational staffing structure to satisfy the requirements of Regulation 11. Senior management at each IP were proactive about ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 4 commodity management, and were supported by capable managers and technical staff at the central, regional, woreda, and FDP levels. In the case of CRS, FDPs were managed and operated by its partner Hararghe Catholic Services (HCS) and Meki Catholic Secretariat (MCS). CRS/HCS/MCS exerted adequate control throughout the supply chain to distribution points. Security guards were also used at specific FDPs Save the Children’s approach used a woreda team of distributors who rotated between FDPs to oversee distributions. By contrast, Save the Children relied upon community policing for guarding commodities at FDPs. Due to conditions of pervasive food insecurity and poverty within these same communities, the review team considers this a risk and recommends Save the Children manage security directly and with a professional staff in the future. There are several good practices to be noted. The review team was favorably impressed with the record keeping of Save the Children, where documents and records were efficiently maintained across the various sites from the central level, and throughout the transportation and distribution network to end point beneficiaries at the FDPs. The review team considered this a model of best practices. Save the Children was also successful at forging demonstrably strong relations with the GoE, being an active part of the targeting process, and an active voice in the appeals process. The CRS approach of requiring beneficiaries to obtain approval before they may delegate other persons to collect their rations added an additional layer of security to ensure that ineligible transfers of rations did not occur. Although beneficiaries were unhappy with this extra effort, it is valuable to ensure the integrity of ration distribution and further attention to streamline this process would be beneficial. Both Implementing Partners were found to adhere to their guidelines of relying on the Kebele and Woreda Task Forces to identify “Direct Support Beneficiaries” (DSB). However, beneficiaries interviewed were very appreciative of Save the Children’s approach where two lists were maintained – the first listing all “permanent” DSB such as the elderly and disabled and the second including persons who were temporarily unable to work due to illness, or for pregnant and lactating women. These individuals moved on to the temporary Direct Support (DS) list and received commodities during their eligibility period, without the need to perform public works. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 5 1. Background and Problem Statement The United States Agency for International Development (USAID) plays a lead role in food security programming in Ethiopia. They are the largest donor to the Government of Ethiopia’s (GoE) multi-donor funded Productive Safety Net Program (PSNP) as well as humanitarian assistance. USAID provides assistance through a consortium of NGO Implementing Partners and through the World Food Program (WFP) to address emergency food needs that arise from natural disasters, such as droughts. USAID’s Development Food Assistance Programs (DFAPs) are mechanisms that support the government’s PSNP to address chronic food insecurity with a goal to make sustained changes to the livelihoods of impoverished communities and increase resilience to economic decline and natural disasters. The DFAPs are implemented by four NGO partners namely, Catholic Relief Services (CRS), Food for the Hungry, Relief Society of Tigray (REST) and Save the Children. The purpose of this review was to assess the compliance and the management efficiency of approved Title II commodities by CRS and Save the Children in the Oromia and Somali regions, as per the USAID standards and regulations. The period of review included FY 2015 and FY 2016 commodities and examined whether the procedures and systems of commodity receipt, management, handling and utilization were consistent with USAID regulations and guidelines; and whether DFAP implementing partners (IPs) followed appropriate standards of best practice in commodity receipt, management, distribution and reporting. During FY 2015 implementation, USAID approved the distribution of 12,179 Metric Tons (MT) of wheat, 1,218 MT of yellow split peas (YSP) and 365 MT of vegetable oil under DFAP for CRS’ implementation. Of these approved quantities, CRS distributed 12,034 MT of wheat for a utilization rate of 98.8%; 1,101 MT of pulses at 90% utilization; and 330 MT of vegetable oil were distributed at 90% utilization. For FY 2016 implementation, USAID approved 16,063 MTof wheat and 4,282 MT YSP with utilization rates of 107% and 101%, respectively. This phenomenon of distributions exceeding 100% occurs when unused commodities are held over as a balance from the previous year. Of the DFAP commodities 86 percent were distributed as part of, or under, the governments PSNP public works (PW) transfers, while the balance of sixteen percent were used as a contingency resource, which CRS distributed at the end of the project cycle, during an additional month of transfers to PW beneficiaries in September 2016. Save the Children received the following FY2015 approved commodities: 11,401 MT of Wheat, 8,740 MT of Sorghum, 1,683 MT of yellow split peas, and 516 MT of vegetable oil; of which the review team found that SCI distributed commodities 10,638,929 MT of Wheat, 7,647 MT of Sorghum, 1,492 MT of YSP and 448 MT of vegetable oil. Then, in FY 2016 SCI received 11,810 MT of Wheat, 8,405 MT of Sorghum, 5,351 MT of YSP, and 69 MT of vegetable oil; of which, SCI distributed 10,469 MT of wheat, 7,361 MT of Sorghum and 4,755 MT of YSP. Title II food assistance programs are subject to USAID’s 22 Code of Federal Regulations 211, commonly known as Regulation 11. This, as well as other USAID directives, outlines commodity management responsibilities, reporting requirements, and claims handling practices that applies to NGO Title II programs. This includes a number of quarterly commodity and loss reports submitted through USAID’s Quarterly Web-Interfaced Commodity Reporting system (QWICR). ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 6 2. Methods Commodities for the period FY 2015 and FY 2016 through October were reviewed in this independent review. DFAP commodity distribution which continues through 2016 under the auspices of the GoE was also examined. The IBTCI review team comprised of a team leader, a commodity manager, a food assistance expert, and evaluation, monitoring and audit specialists traveled to selected warehouses and FDPs to answer key review questions sought by USAID, NGOs and the GoE. CRS and Save the Children both had “Through Bill of Lading” arrangements where shippers were responsible for ocean freight and the transportation of commodities from primary warehouses, Save the Children transferred commodities to secondary warehouses and on to FDPs for distribution to beneficiaries. CRS did not retain any secondary warehouses and supplied commodities to FDPs directly from their primary warehouses. Because the handover and management of commodities began from the time the commodities were received at the primary warehouses, to the point of actual hand-over to beneficiaries, this review focused on responding to the key research questions as detailed in USAID’s Scope of Work (SoW) pertaining to commodity management at primary and secondary warehouses, FDPs and beneficiary end-use monitoring. As CRS had completed both the regular DFAP and Contingency Program commodity transfers, this report reflects on their past performance. To accommodate both the regular DFAP and the Contingency program extension currently ongoing with Save the Children, this report assesses their past and ongoing performance. The review team captured information from primary and secondary warehouses, FDPs, and end-use beneficiaries. A “purposive” sampling approach was adopted to select warehouses and FDPs to be evaluated under this review. The review team visited a mix of FDPs scheduled to distribute food in the timeframe within this review, together with FDPs no longer scheduled to distribute food. Sites with distribution enabled the review team to observe directly how efficiently these activities were performed, and to assess the internal control systems at the FDP. Non-distribution FDPs were also important to assess the manner in which records were kept after the FDP had completed its distribution cycles. The sampling approach first selected only those woredas where distributions were scheduled in the timeframe of this review. From this list, three woredas under CRS and three woredas under Save the Children were randomly selected. Under each selected woreda, FDPs with distributions scheduled within the review timeframe were listed. Two FDPs were then randomly selected from this list, and a third non-distribution FDP within this woreda was also randomly selected. A total of nine FDPs from three woredas, per IP, was chosen using this approach to constitute the sample frame to be evaluated for this review. For Save the Children, three secondary warehouses in each of the selected woredas were selected to be included for review. Considering CRS supplied FDPs directly from their primary warehouses in Adama and Dire Dawa, there were no secondary warehouses for CRS falling into the sample frame for review. Primary warehouses were included in the sample frame to cover the one remaining operational primary warehouse of Save the Children in Adama, and the four primary warehouses used by CRS for DFAP commodities in Adama. Two additional primary warehouses of CRS in Dire Dawa were also included in the sample. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 7 Beneficiary end-use monitoring was conducted among a total of 239 beneficiaries (117 female respondents and 122 male respondents). Sample beneficiaries were randomly selected from FDPs undergoing distributions and interviewed directly at the FDP. For FDPs with no distributions, the evaluators worked with the FDP/regional IP staff, to identify all the villages served by the FDP, and then selected two villages at random. Beneficiaries were then randomly selected from the two selected villages and interviewed directly at their homes. If a beneficiary was not available, the evaluator went to the next selected beneficiary on the list. Although the review managed to evaluate a total of 18 FDPs under six woredas for both CRS and Save the Children, the unrest across the Oromia region beginning on October 2, 2016, required changes in the review schedule. Due to the violence across Oromia, Save the Children were forced to instruct trucks carrying commodities to halt transportation until they were confident it was safe for movement to resume. Unfortunately, trucks remained immobilized over the review period and so all FDPs which had distributions planned were unable to maintain the schedule. This resulted in all nine FDPs of Save the Children selected in the sample frame as being non-distribution FDPs and no active distributions could be observed. In the case of CRS, the review team accelerated the process and was able to capture two woredas which had their final distributions planned at the end of September 2016. Of these two woredas, as described earlier, two out of three FDPs in each woreda were distribution FDPs where the review team was able to observe on-going distributions, and the third FDP was a non-distribution FDP. Table 1 summarizes the sample frame reached to review the 2015/16 Contingency distribution. The 2014/15 distribution was entirely based on secondary documentation review considering the regular DFAP transfers had already come to an end. Table 1: Summary of the final sample frame evaluated at the field sites CATHOLIC RELIEF SERVICES SAVE THE CHILDREN Sample Site Number of Beneficiaries Interviewed Number & Names of Staff Interviewed Sample Site Number of Beneficiaries Interviewed Number & Names of Staff Interviewed Adama Primary Warehouses Alemu Dehaba (Storekeeper) + Degnachew (Commodity Accountant) + Terfefe Tilahun (Storekeeper) + Asrat Abera (Storekeeper) - Total 4 Adama Primary Warehouse Yosef Degefe (Commodity Officer) - Total 1 Dire Dawa Primary Warehouses Girama Zemariam (Warehouse Supervisor) + Aster Samuel (Storekeeper) + Berhanu Beyene (Field Office Manager) - Total 7 Woreda: Arisi Negelle Temesgen Tuna (Commodity Woreda: Dollo Ado Mohammed Jiro (Regional DFAP ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 8 CATHOLIC RELIEF SERVICES SAVE THE CHILDREN Sample Site Number of Beneficiaries Interviewed Number & Names of Staff Interviewed Sample Site Number of Beneficiaries Interviewed Number & Names of Staff Interviewed - FDP1: H/Basso (Distribution) Female=6 Male=5 Total=11 Supervisor, MCS) + Alemu Dibaba (Woreda Food Security Process Owner) - Total 2 FDP1: Aamino (NON￾Distribution) Female=3 Male=7 Total=10 Program Manager) + Kassahun Abayneh (Acting Store Keeper) + Abdurzak (Commodity Accountant) - Total 3 - FDP2: Mudi Arjo (Distribution) Female=7 Male=3 Total=10 FDP2: Shambal (NON￾Distribution) Female=5 Male=9 Total=14 - FDP3: Rafo Argesa (NON￾Distribution) Female=11 Male=9 Total=20 FDP3: Bowbow (NON￾Distribution) Female=3 Male=6 Total=9 Woreda: Sire Getachew Ayele (Commodity Supervisor) + Gebre Lesgesse (Storekeeper) + Lilia Gebo (Storekeeper) – Total 3 Woreda: Yabello Abdi Hamid (Woreda Commodity Coordinator) + Dange Negassa (Secondary Warehouse Storekeeper) + Ibesa Teshome (Quality Assurance Officer) + Shoko Chulle (Commodity Accountant) + Bulle Dhalan (Food Distributor) - Total 5 FDP1: Sire (Distribution) Female=15 Male=19 Total=34 FDP1: Hidi Ale (NON￾Distribution) Female=5 Male=5 Total=10 FDP2: Koloba Hawas (Distribution) Female=3 Male=5 FDP2: Elwoya Magala (NON￾Distribution) Female=5 Male=5 Total=10 FDP3: Ibseta (NON￾Distribution) Female=4 Male=9 Total=14 FDP3: Obda (NON￾Distribution) Female=5 Male=6 Total=11 Woreda: Kersa Gossa Dereje (Storekeeper, HCS) + Samual Yohannes (Storekeeper) + Yosef Adis (Commodity Supervisor, HCS) - Total 2 Woreda: Dhas Habib Kahdira (Roving Storekeeper) + Boru Dalecha (Roving Storekeeper) + Abduha Dhera (Commodity Coordinator) + Tesema Tadese (Storekeeper) + Boru Dida (Commodity Accountant) - Total 5 - FDP1: Dolu (NON￾Distribution) Female=8 Male=6 Total=14 FDP1: Gorille (NON￾Distribution) Female=7 Male=5 Total=12 - FDP2: Langhe (NON￾Distribution) Female=7 Male=5 Total=12 FDP2: Wolensu (NON￾Distribution) Female=7 Male=4 Total=11 - FDP3: Woter (NON￾Distribution) Female=6 Male=7 Total=13 FDP3: Gayo (NON￾Distribution) Female=10 Male=7 Total=17 Annexes C, D and E contain the tools used for the review of the primary and secondary warehouses, the FDPs, and the Beneficiary End-Use monitoring respectively. The tools were ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 9 developed consulting other large and successful Title II DFAPs, particularly the NGO SHOUHARDO DFAP in Bangladesh which managed one of the largest commodity portfolios globally. Technical SHOUHARDO experts were consulted in the process, drawing on their large Title II team’s prior experience, as well as the Ethiopian national review team members involved in this assessment to tailor the tools to the Ethiopian context. The map below shows where the review team visited for original data collection: While drafting, testing, and refining the research tools, an extensive training was conducted with the full review team, where important feedback led to further refinement of the tools. The team piloted the tools in the Sire woreda, working together to utilize different questionnaires at the primary and secondary warehouses, the FDPs, and interviews with beneficiaries. The full team came together on October 1, 2016 to discuss the functionality of the various tools, and to make final adjustments before continuing with the remaining schedule. Analysis was an on-going activity over the duration of data collection. As teams collected information, these were submitted in prescribed reporting formats to the team leader daily. The team leader collated and merged data and continued synthesis and analysis all throughout the data collection period. This approach allowed a continual interpretation of data, allowing the team leader to interact with the data collection teams to follow up on any areas that needed further investigation. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 10 Client Card in use with fingerprints At the same time, a comprehensive desk research was reviewing a broad range of commodity documents of both CRS and Save the Children. This included visiting their offices in Addis Ababa, and directly conversing and interviewing a range of management, administrative and technical staff of both IPs. After completing primary data collection and desk reviews, the review team discussed unique observations and impressions and then triangulated across sources observations, feedback from IP staff, beneficiaries and other key informants including government counterparts. 3. Findings Efficiency in Distribution Distribution of Commodities to intended beneficiaries Evidence from this review indicates that both CRS and Save the Children were compliant with Section 211.5.e. where the military were not involved with the distribution, handling, or allocation of the DFAP commodities; and also compliant with Section 211.5.g. where the review found no evidence of discrimination against eligible recipients. USAID Regulation 11, Section 211.5 states: e) “No military distributions. Except as AID may otherwise agree in writing, agriculture commodities donated by AID shall not be distributed, handled or allocated to any military forces.” f) “Determination of eligibility of recipients. Cooperating sponsors shall be responsible for determining that recipients are eligible in accordance with the operational plan. Commodities shall be distributed free of charge.” g) “No discrimination. Cooperating sponsors shall distribute to only with eligible recipients without regard to political affiliation, geographic location, ethnic, tribal or religious or other factors extraneous to need……..” With regard to determining eligible beneficiaries (Section 211.5.f.), as per the GoE regulation, the Kebele Task Force was responsible for leading the targeting process to identify eligible Ethiopian PSNP beneficiaries, who also comprise the beneficiaries falling under the DFAP. Lists were then forwarded to the Woreda Task Force for approval to develop the Master Beneficiary List (MBL). Save the Children was actively engaged as a member of the Kebele Task Force in identifying eligible beneficiaries and also served as a member of the Appeals Committee. The original MBL was retained by the Woreda and Kebele Task Force, and a copy maintained by Save the Children. CRS guidelines indicate that its partner, the Meki Church Secretariat (MCS), should have participated in the targeting process by working with the Kebele and the Woreda Task ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 11 Standard Issued Containers for Contingency Force, however responses from partner staff in all CRS FDPs indicated that they did not participate in the targeting effort. Conversely, CRS depended on the woreda to undertake the targeting process to select the PSNP beneficiaries. Payroll lists were prepared by the Woreda Disaster Preparedness Prevention Office (DPPO), including the Food Security Process officer based on public works performed by beneficiaries, which were forwarded to the IPs. Beneficiaries attended FDPs on the scheduled day, and the distribution staff verified eligibility of beneficiaries by checking Client Cards, and seeing whether their names appeared on the payroll list sent to them by the woreda. If the names matched, rations were provided to the beneficiary who then confirmed receipt by signing or thumb printing their own Client Card and the payroll sheet. In addition to the control system above, Save the Children had a further internal control system where they were able to check whether the beneficiary was also listed in the copy of the MBL retained at the FDP. In the Sire woreda, the review team found that none of the beneficiaries had Client Cards from the two FDPs with distributions, and the one FDP with non-distribution under CRS. Rather, these beneficiaries were issued paper demands from the woreda reflecting simply the amount of rations they were eligible to receive. On presenting these at the FDPs, beneficiaries were provided rations. Considering there were no Client Cards, beneficiaries only signed the payroll sheet. Evidence collected finds that between 92 – 98 percent of beneficiaries travelled to collect their rations on scheduled days. However, in the few cases that beneficiaries were unable to come to collect commodities, or delegate another to collect on their behalf, the commodities were stored at the FDP and beneficiaries were able to collect this ration, in addition to the next month’s ration, in the next distribution. Efficiency in distribution / Extent of control IPs exercise/maintain during food distributions / Effectiveness of distribution process from observation The review team found that there were variations across the FDPs reviewed of how distributions were managed. As part of the distribution management, beneficiaries were grouped together in cohorts of similar number of rations that a household was entitled to receive. Households eligible to receive the maximum number of rations for five individuals (i.e. households consisting of five or more people) were grouped together; households eligible to receive rations for four individuals were separately grouped; and so on. The review team directly observed in CRS’s FDPs where distributions were active, beneficiaries being grouped and provided intact wheat bags, each weighing 50 kg, as well as opened bags with enough wheat to meet the requirements of all members in a group, for self-distribution. In all other FDPs reviewed for both CRS and Save the Children, the team were informed by respective staff that a similar approach was taken for the 2014/15 regular DFAP commodities. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 12 FDP with active food distribution in Sire Woreda Observations and feedback from beneficiaries indicated that a uniform approach was not taken for distribution of pulses, where FDPs were found to be using both “scooping” and a “group” approach. The scooping process of Save the Children was found to be robust where they used the standard issued 5kg and 4kg containers to provide rations. For the FDPs reviewed under CRS, the review team observed that standard issued containers were not being used, and rather locally procured containers were used instead. This may account for the many complaints heard from beneficiaries that they were not receiving their entitled quantities of pulses. In terms of crowd control and management of food distributions, the review team observed effective systems in place of using ropes to organize queues at CRS FDPs at H/Bosso and Mudi Arjo in the Arisi Negelle woreda, and understood that a similar approach was taken in the Rafo Argesa FDP of Arisi Negelle. For the distribution FDPs under Sire woreda, the review team observed that both the Sire and Koloba Hawas FDPs needed better crowd management that directly impacted on the efficiency of the distribution. FDP Management Committees were observed to be operational in the Arisi Negelle distribution FDPs, comprised of four members from the community with one space reserved for a female. However, the Management Committees in the Sire distribution FDPs were all comprised of government representatives with no community members present. Unfortunately, the review could not observe other distributions as all other sample FDPs had no distributions scheduled during the review period. From beneficiary interviews, the review team found that beneficiaries across all three FDPs of CRS in Arisi Negelle, and the two FDPs in Sire, were required to contribute 2-3 Birr [$0.09 – $0.13] to cover costs of labor and guards working at the FDP, in addition to costs borne by the DFAP. For the two Sire FDPs, beneficiaries were required to pay 2 Birr [$0.09] for the printing of Client Cards. It was further found from interviews with beneficiaries at the Ibseta FDP in Sire that they were required to pay 5 Birr [$0.22] each to government officials to contribute to a savings account. The review team found no evidence that such a savings account existed. For Save the Children, the Shambal FDP in Dollo Ado, the Gorille FDP in Dhas, and the Elwoya Magala FDP in Yabello, beneficiaries were required to similarly pay for labor and guards at the FDP. This directly violates Regulation 11 Section 211.5.f. which states “Commodities shall be distributed free of charge…in no case will recipients be excluded from receiving commodities because of inability to make a contribution to the cooperating sponsor for any purpose.” ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 13 Complaint box which was relatively inaccessible inside Sire FDP The “complaint management system” was another area needing attention. Universally across all the sample beneficiaries interviewed, almost all respondents said they were unaware of how to lodge complaints. More specifically, they were not aware of the accountability systems put in place by CRS or Save the Children. Not having a “risk-free” complaint system established resulted in beneficiaries not being able to raise concerns they may have had on the distribution. Distributions to entitled beneficiaries per the approved ration size and mix To respond to this research question, two elements were considered – first, whether beneficiaries were aware of their entitlements, and second, according to beneficiaries whether they were receiving the correct quantity they were entitled to receive. Table 2 shows that beneficiaries were largely content across the Save the Children areas, with the exception of the Obda FDP in Yabello Woreda where 50 percent of interviewed beneficiaries believed they were receiving less pulses than their entitlement, as well as the Gayo FDP in Dhas Woreda where evaluators found that 50 percent of interviewed beneficiaries were unclear about their entitlement. In the case of CRS, 80 – 100 percent of beneficiaries responded that they did not receive their due entitlements for pulses across all three FDPs reviewed in the Sire Woreda, and 40 percent of respondents in the Mudi Arjo FDP in Arisi Negelle Woreda complained the same. In Kersa Woreda, 50 percent beneficiaries from two of the three FDPs assessed were found lacking knowledge on entitlements for pulses, and 55 percent of beneficiaries similarly lacked knowledge on entitlements in the H/Basso FDP in Arisi Negelle Woreda. Only in the Rafo Argesa FDP in Arisi Negelle Woreda and the Dolu FDP in Kersa Woreda were beneficiaries found to be content on entitlements and receiving proper ration levels. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 14 Table 2 – Summary of beneficiary responses on rations Catholic Relief Services Save the Children Woreda / FDP Beneficiary Response Woreda / FDP Beneficiary Response Arisi Negelle -H/Basso FDP 55% beneficiaries unclear on entitlements (pulses) Dollo Ado -Aamino FDP OK Arisi Negelle -Mudi Arjo FDP 40% beneficiaries complained receiving less rations Dollo Ado -Bowbow FDP OK Arisi Negelle -Rafo Argesa FDP OK Dollo Ado -Shambal FDP OK Kersa -Dolu FDP OK Dhas -Gayo FDP 50% beneficiaries unclear on entitlements (pulses) Kersa -Langhe FDP 50% beneficiaries unclear on entitlements (pulses) Dhas -Gorille FDP OK Kersa -Woter FDP 50% beneficiaries unclear on entitlements (pulses) Dhas -Wolensu FDP OK Sire -Ibseta FDP 100% beneficiaries complained receiving less rations Yabello -Elwoya Magala FDP OK Sire -Koloba Hawas FDP 80% beneficiaries complained receiving less rations Yabello -Hidi Ale FDP OK Sire -Sire FDP 95% beneficiaries complained receiving less rations Yabello -Obda FDP 50% beneficiaries complained receiving less rations Effectiveness of communication systems at FDPs The IPs’ effective communication method involved displaying “up-to-date” information on sign boards at FDPs. The review team discovered that all signboards used across CRS and Save the Children FDPs displayed the rations were a gift from the American people. However, there still remained a broad belief that rations were coming from other donors, including the GoE, US Catholics or the IPs themselves. Table 3 summarizes responses from beneficiaries interviewed receiving rations from different FDPs. Further effort was needed to ensure that all beneficiaries recognized that the donor was the US government and to comply with Regulation 11 Section 211.5.f. which states “public recognition will be given that the commodities have been provided through the friendship of the American People as food for peace.” ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 15 Table 3 – Beneficiary interpretation of the donor Catholic Relief Services Save the Children Woreda / FDP Beneficiary response about who is the donor Woreda / FDP Beneficiary response about who is the donor Arisi Negelle -H/Basso FDP 60% believe GoE + US Catholics Dollo Ado -Aamino FDP 40% believe USAID + SC Arisi Negelle -Mudi Arjo FDP USAID Dollo Ado -Bowbow FDP USAID Arisi Negelle -Rafo Argesa FDP 20% believe US Catholics Dollo Ado -Shambal FDP 50% believe USAID + SC Kersa -Dolu FDP USAID Dhas -Gayo FDP 30% believe USAID + SC Kersa -Langhe FDP USAID Dhas -Gorille FDP USAID Kersa -Woter FDP USAID Dhas -Wolensu FDP USAID Sire -Ibseta FDP 70% believe USAID + GoE Yabello -Elwoya Magala FDP USAID Sire -Koloba Hawas FDP USAID Yabello -Hidi Ale FDP USAID Sire -Sire FDP 10% believe USAID + GoE Yabello -Obda FDP 80% believe SC For the FY 2015 regular DFAP, all FDPs reviewed were observed to display and communicate the accurate commodities and quantities that were then disbursed. For the FY 2016 commodities from DFAP, only the Save the Children FDPs of Dhas and the Elwoya Magala FDP in Yabello reflected up-to-date information about their rations. Save the Children had produced a transferrable signboard made of cloth, which was draped over the permanent signboard at these FDPs to reflect the contingency rations being provided, and the quantities against each of these. All remaining signboards across the CRS FDPs and the remaining Save the Children FDPs did not reflect the latest ration mix that beneficiaries were entitled to, and continued to display the previous rations provided under the regular DFAP. This may have been a factor that contributed to confusion as entitlements to pulses had changed from1.5 kg under the DFAP, to 4 kg during the extended period, as well as the discontinuation of vegetable oil where a large portion of beneficiaries interviewed expressed frustration that they no longer received oil. Role of government in the distribution process The Woreda Food Security Expert was assigned to each of the FDPs, to help oversee the distribution and to ensure that eligible beneficiaries received the correct rations. The general feedback from beneficiaries about the effectiveness of government participation was mixed. Specifically, in all three CRS FDPs under the Sire Woreda, it emerged that the FDP Management Committee was wholly comprised of various government representatives – including the Food Security Expert, the Health Agent, a representative from the Women’s ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 16 Affairs department, and the Kebele Chairman. For the two FDPs observed with distributions, the review team found that these various government representatives were absent during the distribution, calling into question the effectiveness of these management committees. Further, the FDP Management Committees across the CRS and Save the Children FDPs were usually comprised of elected community representatives, which was not the case in Sire Woreda – thereby limiting community engagement. Time beneficiaries spend at distributions Both IPs operated in hard-to-reach areas of Ethiopia, where remote landscapes and the large geographic areas covered by the programs determined the trade-offs necessary to locate FDPs as best as possible where beneficiaries were able to reasonably travel to these sites for ration collection, together with FDPs offering the environment for a safe and efficient distribution. Two factors were reviewed to assess the efficiency of FDPs – firstly, the location of the FDPs and on average how long it took beneficiaries to reach these FDPs; and secondly, once at the FDP, how long it took on average for beneficiaries to receive rations. Time taken by beneficiaries to reach their FDPs Figures 1 and 2 summarize the average time taken for beneficiaries to travel to the FDPs, which in all cases is by walking. In these charts, red indicates high levels of concern, green indicates low levels of concern, and yellow indicates medium levels of concern. 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 ArisiNegelle-H/Basso ArisiNegelle-MudiArjo ArisiNegelle-RafoArgesa Kersa-Dolu Kersa-Langhe Kersa-Woter Sire-Ibseta Sire-KolobaHawas Sire-Sire Figure 1. Catholic Relief Services Average time taken for Beneficiaries to reach FDPs (in hours) ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 17 The majority of Save the Children FDPs sampled and visited appeared to be well located where beneficiaries could reach FDPs within an hour, with the exception of the Obda FDP under Yabello Woreda, and the Gorille FDP under Dhas Woreda. In contrast, CRS FDPs took more time on average to reach – requiring on average two hours travel time – and the Sire FDP under the Sire woreda taking up to four hours to reach. Locating FDPs within easy distances of beneficiaries’ homes is important to ensure that beneficiaries are able to not only easily walk to these FDPs, but also to make it easy for them to take back rations to their homes. Longer distances invariably incurred additional costs as beneficiaries were forced to pay transport fees for local transportation. The long distances also potentially put beneficiaries at risk, especially females, starting their return journeys home later during the evenings in the dark. Ideally, beneficiaries should have been able to reach FDPs within an hour, or a round trip of two hours. However, given the remote nature of program areas, it is understandable that certain FDPs took longer to reach. The FDPs shown in red in Figures 1 and 2 indicate that the locations of these FDPs were unsuitable considering the length of time beneficiaries were compelled to walk to reach these sites. Beneficiaries collecting rations from these FDPs essentially lost their entire working day, many of whom were forced to stay overnight given that they also had to wait a number of hours to receive rations as detailed in the following section. Time taken to collect rations The time CRS beneficiaries waited, on average, to receive rations after they reached FDPs was significantly long at most FDPs as summarized in Figure 3 below. In the H/Basso and Mudi Arjo FDPs under Arisi Negelle woreda, the Langhe FDP in Kersa woreda, and the Sire FDP in Sire woreda, beneficiaries on average waited between six to eight hours before they received rations. In the Rago Argesa FDP in Airisi Negelle woreda, and the Dolu FDP in Kersa woreda, beneficiaries waited up to three hours to receive rations. Only the Woter FDP in Kersa 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 DolloAdo-Aamino DolloAdo-Bowbow DolloAdo-Shambal Dhas-Wolensu Dhas-Gayo Dhas-Gorille Yabello-ElwoyaMagala Yabello-HidiAle Yabello-Obda Figure 2. Save the Children Average time taken for Beneficiaries to reach FDPs (in hours) ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 18 woreda, and the Koloba Hawas and Ibseta FDPs in Sire woreda, appeared to have had acceptable wait times of between one to two hours for beneficiaries to receive rations. For Save the Children (Figure 4), the Obda FDP in Yabello woreda, and the Gorille FDP in Dhas woreda had long wait times between six to eight hours for beneficiaries to receive their rations. Only the Bowbow FDP in Dollo Ado woreda, and the Wolensu FDP in Dhas woreda, appeared to have had acceptable wait times of between one to two hours. All remaining FDPs had a wait time of up to four hours. Wait times at FDPs can ensure that beneficiaries could collect their rations quickly and get back home in the shortest time. This enables beneficiaries to engage in other productive activities during the day including income generating activities and house chores which otherwise remain neglected. Considering the target beneficiaries were from the poorest and most vulnerable communities, it was essential for them to engage in productive activities to develop their economic and household capital and only invest the shortest time possible on non-productive activities such as the collection of safety-net food. 0 1 2 3 4 5 6 7 8 ArisiNegelle-H/Basso ArisiNegelle-MudiArjo ArisiNegelle-RafoArgesa Kersa-Dolu Kersa-Langhe Kersa-Woter Sire-Ibseta Sire-KolobaHawas Sire-Sire Figure 3. Catholic Relief Services Average time for beneficiaries to receive rations from arrival time at FDP (in hours) ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 19 Delays were experienced primarily due to beneficiaries being called all at one time, and the distribution only starting once all beneficiaries, or their delegates, were present at the FDP. Beneficiaries interviewed often complained that when they reached the FDP at the appointed time, they were frequently forced to wait several hours until the late comers arrived. From the perspective of the IPs, this approach was taken to enable a more efficient distribution where households were grouped based on the number of family members. Different groups were then provided group rations for self-distribution between the members in the group. Although in theory this sounds reasonable, in reality this approach turned into a major obstacle leading to significant delays. When taken together with the time taken for beneficiaries to walk to the FDPs, and the time taken to receive rations, beneficiaries interviewed complained this resulted in considerable hardship and risk and by the time many of them embarked on their return journey home it had already turned dark. This compelled them to either hire transportation on the return journey which was costly, or many of them were forced to store their rations and to stay overnight at a friend or neighbor’s home. The overnight stay was usually on a rental basis, where beneficiaries either paid by cash, or in kind, where they were forced to share a portion of their rations with the host family. The review team found that none of the FDPs reviewed, of either CRS or Save the Children, offered any basic facilities to beneficiaries such as shade for people to shelter under, drinking water, and latrine facilities. Not having such basic provisions, together with the long wait times to receive rations, resulted in severe discomfort for beneficiaries leading to frustration. 0 1 2 3 4 5 6 7 8 DolloAdo-Aamino DolloAdo-Bowbow DolloAdo-Shambal Dhas-Wolensu Dhas-Gayo Dhas-Gorille Yabello-ElwoyaMagala Yabello-HidiAle Yabello-Obda Figure 4. Save the Children Average time for beneficiaries to receive rations from arrival time at FDP (in hours) ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 20 Example of signed Payroll Sheet Existence and reliability of predictable transfer schedule (public works versus direct support) In each of the FDPs observed, payroll lists were observed to be issued by the woreda indicating eligible beneficiaries to collect rations against public works performed. The Payroll Lists were sent to each of the FDPs ahead of the planned distribution, and rations were provided to eligible beneficiaries whose identities were confirmed via Client Cards. The only exception to this was in the Sire woreda of CRS where none of the beneficiaries across the three FDPs reviewed had any Client Cards. With the exception of the Sire woreda, evidence shows that all other sample woredas were following due process for payment against public works. In the case of Direct Support, they were also reflected in the Direct Support list at each of the FDPs. On presentation of their Client Cards, these beneficiaries were provided their due rations. Food Management Practice and Internal Control Systems The review team managed to review four Primary warehouses of CRS in Adama, together with two primary warehouses of CRS in Dire Dawa (one rented from a private owner, and the other a rubhall owned by the organization). CRS did not maintain any secondary warehouses, rather supplying all their FDPs directly from the primary warehouses in Adama and Dire Dawa. For Save the Children, the only remaining primary warehouse with DFAP commodities in Adama was reviewed, as well as the secondary warehouses based in all the sample woredas of Dhas, Dollo Ado and Yabello. Review of food management, distribution and reporting system / Technical backing and support provided by regional experts At Save the Children, the Deputy Chief of Party was responsible overall for the management of commodities, and was supported at the central level by the Senior Commodity Manager, the Logistics Manager, the Logistics Officer, the Commodity Accountant and the Monitoring, Evaluation and Compliance Manager. This team was responsible for planning commodities and initiating requests for commodities from USAID. The central management team was the key liaison team with USAID. The Save the Children regional level management included the Regional Program Manager and the Commodity Coordinator, and at the woreda level there was the Woreda Program Manager, Secondary Warehouse Store Keeper, Commodity Accountant and Commodity Coordinator. At the FDP level there was a roving management team including store keepers, food distributors and quality assurance officers, who were responsible for distributions in all FDPs in a woreda. Food monitors and quality assurance officers conducted monthly monitoring visits during FDP distributions to ensure that ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 21 •Store Keepers •Food Distributors •Quality Assurance Officers FDP Level •Woreda Program Manager •Secondary Warehouse Store Keeper •Commodity Accountant •Commodity Coordinator Woreda Level •Regional Program Manager •Commodity Coordinator Regional Level •Deputy Chief of Party •Sr Commodity Manager / Commodity Accountant •Logistics Mgr / Logistics Off •M&E / Compliance Manager Central Level commodity transfers were appropriately being performed. They also visited warehouses and investigated FDP stocks. The IBTCI team was provided evidence of a checklist of findings, together with an action plan of how these were to be handled, by whom, together with timelines. At CRS, the Head of Operations was in overall charge of commodities and supported at the central level by the Logistics Manager, the Deputy Logistics Manager, Commodity Accountant and the Compliance Officer. This team was responsible for coordinating with USAID. At the regional level the organization employed the Field Office Manager, Commodity Accountant, Commodity Transport Officer, Warehouse Supervisor, and Store Keeper, and at the woreda level employed the Program Coordinator and Commodity Supervisor. At the FDP level, the partner Meki Church Secretariat (MCS) employed Store Keepers and food distributors, with monthly field monitoring visits from CRS Food Monitors, and quarterly visits from regional and central technical and management staff. For perceived risky areas, food monitors increased their visits to twice a month until the risk was considered to be contained. At the CRS central level, the Chief of Party (COP) spearheaded the Commodity Risk Monitoring and Compliance department, consisting of the Deputy Chief of Party and four Compliance Officers. The review team found evidence that the Compliance Officers conducted quarterly field visits and reported back to the COP. The review team observed that this department had developed an easy checklist to manage and follow up on issues identified during the monitoring visits which included what the finding was, how this was to be addressed, who was responsible, together with timelines of when actions would be performed and evidence of closure. Evidence suggests that both IPs adopted and implemented their FY 2015 and 2016 commodities via a robust system with sufficient commodity technical and management personnel satisfying Regulation 11 Section 211.5.b which states that “the Cooperating Sponsor shall provide adequate supervisory personnel for the efficient operation of the program in planning, organizing, implementing, controlling, and evaluating programs involved in food distributions.” Reports flowed up from the FDPs, where FDP staff prepared weekly and monthly distribution and commodity status reports, recipient status reports, loss reports, and inventory reports, submitting these to woreda management who in turn collected information and submitted it to regional management, and then to central management. This is discussed in greater detail in the Record and Reporting section, below. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 22 •Store Keepers •Food Distributors FDP Level •Program Coordinator •Commodity Supervisor Woreda Level • Field Office Manager • Commodity Accountant • Commodity Transport Off •WH Supervisor / Storekeepr Regional Level •Head of Operations •Commodity Accountant •Logistics Mgr / Dep Logistics •Compliance Officer Central Level The review team found all warehouse staff interviewed for both CRS and Save the Children to be knowledgeable, with extensive experience in warehouse and commodity management. CRS had taken effort to connect the DFAP program to a broader environment where it benefited from programming experience and guidance from a technical committee comprised of Food for the Hungry, Save the Children, World Vision, CARE, the Relief Society of Tigray (REST), Hararghe Catholic Secretariat (HCS), and the Meki Catholic Secretariat (MCS). In addition to broader programmatic discussions, commodity related issues were also discussed in the monthly meetings, and to determine how best these safety net commodities could be channeled across the PSNP beneficiaries. Head office management conducted periodic visits to support the committee and meet local and regional staff. Save the Children conducted a workshop of extensive capacity building in Adama in 2015 with participation of store keepers, distributors, commodity supervisors and accountants. Central management and technical staff conducted periodic visits to local and regional warehouses and FDPs to provide active support to staff managing and operating at these levels. Field monitoring reports were provided as evidence to the review team confirming these visits were being conducted. Adequacy of commodity management practices, safeguarding mechanisms and internal control systems When commodities were received from transporters, evaluators found evidence that store keepers counted and tallied all bags received, and completed the waybill in its entirety. Commodities were stacked separately in the warehouses, by commodity type, and BIN cards were updated to reflect information pertaining to commodity type, date received, number of bags received, dates of dispatch, and balance remaining. BIN cards are stock cards, the mechanisms detailing a particular batch of commodities. Across all primary and secondary warehouses reviewed for both CRS and Save the Children, the review team observed that commodities were stored as per USAID regulations. Stacks were one meter from the walls, ceilings, and between stacks; plastic sheets and pallets were used for stacking; stacks were less than 25 levels high; different shipments were stacked separately; the first-in/first-out (FIFO) approach was adopted; damaged commodities were stacked separately; and regular fumigation was performed. With regards to damaged bags, the commodity manual of Save the Children instructs that to minimize losses, damaged commodities should be “re-conditioned” immediately. The review ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 23 team observed that damaged bags were stacked separately in all four warehouses visited by Save the Children. The commodity director explained that the manual does not explicitly detail who should be on the Reconstitution Committee, the verbal instruction is that this should be performed by the store keeper, in presence of the commodity accountant and the commodity supervisor. Although the warehouse staff of Save the Children warehouse in all four warehouses under this review stated that reconstitution is performed as per guideline, the review team could not observe this in action as there was no commodity transfers taking place at the time of the review, and neither was there any recorded evidence of who oversaw any previous reconstitution of commodities. The CRS commodity manual states that when there is any movement of commodities, a daily report must be produced showing all movements of the commodity, and evidence was provided to the review team of these reports for both the regular DFAP and Contingency commodity movement. Spot sample checks by the evaluator confirmed that the daily reports matched with the authorization documents for different commodity movements. The CRS manual further states that damaged commodities must be immediately “reconditioned” in clean packaging. As per manual, the review team observed that reconstituted commodities were stacked separately, with information attached explaining the reason for the reconstitution. However, the manual does not stipulate who should be on the Reconstitution Committee, and the verbal instruction was that the Field Office Manager should form this committee. The review team did not observe any reconstitution activities as there were no commodity movement from warehouses during the field research period. Nor was there evidence of who was on such reconstitution committees. Commodities were sent out to FDPs after receiving authorizations from central management, where third party transporters carried commodities from warehouses to FDPs. In relation to internal control systems, the review team observed that both IPs followed organizational guidelines and bags were counted at the time of receipt at the FDPs. Save the Children, as an additional measure, weighed a sample of bags at time of delivery, or bags which appeared to have been damaged or tampered with. The review team considers that not weighing a sufficient sample of bags at times of receipt, may have increased vulnerability to misappropriation. Transportation by third party vendors is a considerable risk as trucks travel through remote areas from the time of collecting commodities from the Primary or Secondary warehouse, to its delivery point the FDP. During this travel time, bags can be perforated and small quantities taken out, and then resealed again. Catching such misappropriation is difficult with the naked eye, especially when large quantities of commodities are being received at the FDP. Save the Children primary warehouse, Adama ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 24 As discussed earlier, the internal control system of CRS was vulnerable in the Sire woreda where beneficiaries were issued paper demands from the woreda administration on their entitlements, and collected rations by presenting these at the FDP. There was little room for CRS, or its partner the Meki Church Secretariat (MCS), to cross-check the identity of these beneficiaries as they did not have Client Cards and relied only on the paper demands issued by the woreda. Another potential internal control weakness was that neither of the IP woreda offices, nor FDPs, had a copy of a list of “authorization signatures.” This could be a potential threat as instructions from unauthorized people could be sent to the FDPs to act on. When probed, staff at FDPs expressed confidence that they were aware of all authorized signatures. Nonetheless, this remained a threat, especially at times of staff turn-over. FDPs were retained all throughout the program life, and consequently had commodities remaining beyond the distribution date. CRS had dedicated paid staff at each of the FDPs, ensuring that commodities remain protected. Save the Children relied on a community participation approach where the community took the responsibility to care for and secure the FDPs and the commodities stored within. The approach taken by Save the Children may have been vulnerable considering the target communities were from the poorest and most food insecure population, exposing greater risk of potentially small and undetectable thefts to occur, or even larger thefts when beneficiaries faced acute hunger periods. Muddy FDP in Sire ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 25 Storage/warehouse conditions at Primary, Secondary and Tertiary levels Overview of Catholic Relief Services Primary Warehouses Physical structures in all of the four CRS Primary warehouses in Adama, and the rented primary warehouse in Dire Dawa, were found to be adequate with walls, ceilings and floors offering adequate protection; having electricity; being easily accessible by trucks; have wide doorways for offloading commodities; and have sufficient storage space to store commodities as well as space for reconstitution of commodities. The Dire Dawa rubhall was an exception as it was found to be aged and damaged. The floor was covered with plastic tarpaulin, and decaying walls and roof rendered the warehouse unable to protect from leaking water and pests. Evidence about security, based on observations at all warehouses, suggests that all had solid and secure doors, maintained quality locks, and offered 24 guards service where all guards demonstrated to the evaluators their knowledge about how to operate fire extinguishers. Potential weakness still remained for the rented warehouse in Dire Dawa where the landlord was responsible to provide security. CRS had little oversight on security matters. Another potential weakness was that the keys to the warehouse were retained by the store keeper. Yet, the review team found that copies were loosely kept at the regional office and accessible by others. There were two potential risks as a result. First, having copies of keys not formally being assigned to staff may have made security vulnerable as copies of keys could be made with little to knowledge of CRS management. Second, having the store keeper being the only person with keys elevated vulnerability as there were no immediate checks on the store keeper that could prohibit possible misappropriation. CRS did have efficient monthly monitoring visits from food monitors who would conduct physical counts – and shortfalls could be detected at these times, but after possible misconduct had already occurred. Finally, there was no evidence of surprise visits by external CRS staff to check on commodities – an important practice to ensure that security remains tight and commodities are safeguarded. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 26 Overview of Save the Children Primary Warehouse and Secondary Warehouses All three secondary warehouses and the primary warehouse of Save the Children visited by the review team had sturdy physical structures offering protective walls, ceilings and floors; were easily accessible by trucks with wide doorways; and had space for transferring, reconstituting and storing commodities. The Dollo Ado and Yabello secondary warehouses did not have electricity as they were only functional during daylight hours. This may need to be reviewed to consider how Save the Children staff in these warehouses would react to an emergency that required a 24-hour operation, or should a threat occur placing commodities in danger that may require attention beyond daylight hours. All Save the Children warehouses were found to offer solid and secure doors, quality locks, and a 24-hour guard service where all guards demonstrated their knowledge of using fire extinguishers – except for the guards at the Dhas secondary warehouse. Security problems arise as warehouse keys are retained only by the store keeper, and there was no evidence of surprise visits by other Save the Children staff and management to check on security. Availability of warehouse equipment The review team observed that all warehouses reviewed for both IPs had the basic equipment required to operate. Furniture and stationary were all present, together with commodity management tools such as standardized scooping measures, bags for reconstitution, sieves/sorters, and ladders, sampling spears, and twine and needles or bagging machines. Evaluators found that all warehouses had weighing machines, but there was no evidence these were calibrated except for the machine in the Save the Children Primary warehouse in Adama. Communication systems were present and functioning in all warehouses that included mobile phones and land phones. Rat traps were seen in all warehouses, but none of the traps were observed to be set. Fire extinguishers were examined at each warehouse, and were found to be functioning. However, the fire extinguishers in the Save the Children primary warehouse in Adama did not have a gauge, making it difficult for regular checks to determine whether these continue to remain operational. The fire extinguishers in the Save the Children secondary warehouses in Dollo Ado and Yabello were found to have passed their operational dates. Overview of Food Distribution Points For all the FDPs visited of both IPs, the review team observed that the FDPs were accessible by trucks and had sufficient storage space to maintain commodities. At the time of receipt, evaluators checked and found that receipt forms were completed in full by the FDP Store Keeper, all bags were counted at the time of delivery, appropriate BIN cards maintained and attached to the commodity stacks, and monthly distribution reports prepared. A similar check was initiated for 2014/15 commodities, where records were maintained in regional offices of ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 27 CRS and Save the Children, and the review team were provided evidence that the regular DFAP commodities also underwent the same rigor and documents were completed in full. Only two Save the Children FDPs were found to include problems. The first was the FDP in Bowbow under the Dollo Ado Woreda. The evaluators learned that the FDP was previously located elsewhere, but beneficiaries raised objections as this was too remotely located making it difficult for beneficiaries to come to the FDP. Through a consultative process, Save the Children opted to situate the FDP in its current location in Bowbow, where the FDP was constructed by local people under a traditional design. Consequently, the FDP was observed to have a weak structure, with gaps in between the wooded walls and ceiling allowing little protection from weather and pests. The second FDP raising concern was the Obda FDP in Yabello. Many beneficiaries interviewed raised concern that the location of the FDP made it very difficult for them to reach the FDP having to travel down from the mountain area where their village exists. They complained that by the time they received their rations, it was dark and the return journey home became more hazardous due to carrying heavy commodities. This appeared to result in many recipients selling their commodities to the local market, preferring the cash rather than paying for expensive transportation to carry back the commodities to their homes. Adequacy of commodity loss management practices According to Regulation 11, Section 211.8., for commodities unfit for authorized use the cooperating sponsor shall immediately arrange for inspection of the commodity by a Public Health Official, and if the commodity is found unfit for intended use, the cooperating sponsor shall notify USAID of the circumstances pertaining to the loss or damage of commodities. The disposal of unfit commodities valued over $500 in theory requires approval from USAID as well as observation by a USG representative. Evidence indicates that both CRS and Save the Children followed this process for 2014/15 and 2015/16 commodities. For example, Save the Children wrote a letter to the Adama Health Office to check 180 bags of wheat and 16 bags of pulses, which were then confirmed as unfit for human consumption. USAID was duly notified and approval obtained to destroy the wheat and pulses valued at $7,200. The commodities were eventually destroyed in the presence of a USAID representative and local officials. Losses were also recovered from third parties with evidence provided to the review team. This included CRS and Save the Children recovering from third party transportation companies the full value of the commodity market price at the time of loss as per Regulation 11 Section 211.9., which states “the value of the lost commodity shall be determined on the basis of the domestic market price at the time and place of lost commodities.” The review team did not find an unusual volume of inventory adjustments, write-offs or disposals of commodities for either CRS or Save the Children. Record-keeping and Reporting To assess the accuracy and consistency of record-keeping and reporting, the review team reviewed the following documents for both the periods of 2014/15 and 2015/16, as well as checking samples of records retained at the regional warehouses and FDPs, to include: o Annual Results Reports o Monthly Commodity Status Reports ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 28 Commodity BIN Card at the Arisi Negelle FDP o Quarterly Commodity Status Reports o Loss and Adjustment Reports o Inventory Reports o Food Monitoring Reports The record keeping process for both the 2014/15 and 2015/16 commodities adopted the same management, verification and reporting process. Documentation and program records were reviewed at the central level at the Addis Ababa premises of the two IPs. This enabled the team member to consult senior management and commodity specialists in parallel with the documentation review. At the same time, the field evaluators sampled and checked records and documents retained in warehouses and FDPs to check accuracy and consistency of records and reports. The entire team ensured that sample checks of records were taken from both the 2014/15 and 2015/16 periods. The field evaluation teams, when visiting warehouses and regional offices, randomly selected from the 2014/15 period two inventory records that were maintained at the different warehouses and regional offices visited. The information on these inventory sheets were matched with the BIN cards and found to be largely accurate across all the warehouses under review. Similarly, two inventory records for the 2015/16 period were extracted and matched with corresponding BIN cards where no major anomalies were found. Evidence in the form of Physical Count and Monitoring reports were reviewed by the review team at warehouses covering the 2014/15 and 2015/16 periods, which confirmed that monthly monitoring visits did take place, and were appropriately recorded and reported to management. Several monthly and quarterly Commodity Status Reports were pulled from both these years, and the respective teams verified the content in these reports with the inventory ledger and found that the information was consistent. For the 2015/16 commodities, the review team conducted direct assessments at the warehouses and FDPs chosen for review by examining the content and information displayed on BIN cards and by physically counting the number of bags in select stacks where all checks proved the accuracy of the BIN cards. Inventory ledgers were then checked and again found to contain accurate information consistent with the BIN cards. Information contained in waybills and authorization documents were matched for both the 2014/15 and 2015/16 periods and found to be consistent. All authorization notes picked for review were found to be appropriately completed and signed by the authorizing manager. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 29 Waybills were also completed in full at the time of receipt of commodities. All losses were appropriately logged in the Loss and Adjustment report and submitted to management. At a program level, the document/record review examined and confirmed that the content of the Annual Results Reports matched with the information coming from the collection of Quarterly Commodity Status Reports for both implementation years 2014/15 and 2015/16. The few exceptions included one of the CRS primary warehouses in Adama and the Save the Children Dhas secondary warehouse not maintaining separate BIN cards for reconstituted commodities. Further investigation is also recommended for the Dhas secondary warehouse as all the requested Loss and Adjustment reports were not provided to the evaluation team. Regulation 11 Section 211.10.b. states “Cooperating Sponsors shall submit reports to USAID as USAID may reasonably request. The report should contain periodic summary reports cataloguing receipts, distribution and inventory of commodities.” The USAID QWICR report is a fundamental reporting tool of USAID. Both CRS and Save the Children provided documentary evidence that these reports were prepared and submitted in a timely manner over the 2014/15 and 2015/16 periods. Other operational reports prepared by CRS and Save the Children covering these two implementation years of 2014/15 and 2015/16 included the weekly and monthly commodity transaction reports prepared by Store-Keepers and submitted to the Field Office Manager. The Field Office Manager then prepared and submitted monthly Commodity Status Reports, Loss and Adjustment Reports, and Inventory reports to the central management. As explained above, the review team verified the accuracy of information contained and collated from one report to another and were satisfied that both IPs followed due diligence in producing these reports for 2014/15 and 2015/16. The central commodity management team of both CRS and Save the Children then submitted Commodity Status Reports quarterly to USAID, as well as completing the QWICR Reports. CRS and Save the Children had developed effective validation checking processes to ensure accuracy of the reports produced both under the regular DFAP 2014/15 and the Contingency 2015/16. CRS have an established Commodity Risk Monitoring and Compliance Department at both the central and field office levels. At central level, the COP manages this department, and is supported by the Deputy Chief of Party, the Commodity Risk Monitoring and Compliance Manager and compliance officers. At field office level, the department is headed by the Risk Team Leader and risk monitors. Management and technical staff from central and field office levels conduct regular meetings and reviews to ensure the accuracy of the reports produced. Save the Children also maintains a validation check on the accuracy of reports through its compliance and quality assurance staff who conducted monthly field visits at times of distribution, to ensure that distributions were being conducted accurately, to conduct physical count of inventory, and to review the accuracy of records being updated as commodities entered and exited warehouses and FDPs. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 30 4. Conclusions Both IPs were largely found to be compliant with Regulation 11. The military were not involved in either the handling of commodities, or receipt of commodities, and beneficiaries were selected via an equitable targeting process led by the Kebele Task Force and approved by the Woreda Task Force. Beneficiaries fell either under the category of public works or direct support. Lists were prepared by the woreda regarding eligible beneficiaries based on their participation in public works and sent to the FDP where they collected rations. Directly assisted beneficiaries were eligible to receive their rations once registered, and not required to participate in public works. Overall, the time required for beneficiaries to reach their FDPs, and the wait time to receive rations was lengthy in many instances. Often beneficiaries spent one to two days collecting their commodities. This was a non-productive activity with inadequate accommodations for long waiting periods, and used time that could have been otherwise better spent by beneficiaries in activities such as income generation, agriculture production, child care, or conducting household and family chores. Suggestions include reviewing locations of FDPs and ensuring that these be in closer proximity to serve the beneficiary catchment area so that times to reach FDPs can be kept to a minimum. At the same time, remoteness and landscapes affect this process making some FDPs more readily accessible than others. Better planning and management at FDPs would also lead to more efficient delivery of rations. Staggering distributions throughout the day to groups clustered around number of rations a household is entitled to, and starting the distributions as soon as beneficiaries arrive at the FDP, rather than waiting for the full set of beneficiaries to arrive would also improve the distribution process. Universally, all FDPs visited were found to offer no basic provisions such as drinking water, shade from the sun or rain, or latrine facilities. The absence of such basic accommodations coupled with inordinately long wait times at FDPs resulted in beneficiaries being subjected to difficult conditions. These can be averted in the future with relatively small investments. Responses from beneficiaries found that a high number of ration recipients paid fees to receive rations. This was collected by the village leaders as a contribution to cover labor costs and the cost of guarding commodities. However, this practice does not comply with Regulation 11. It Commodity preparation for distribution in CRS Arisi Negelle woreda ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 31 was further found that at the Ibseta FDP in the Sire woreda, operated by CRS, government representatives collected 5 Birr ($0.23) from each beneficiary to contribute to a savings account. The review team found no trace or evidence of any such savings account. Many beneficiaries believed they had received less rations than their entitlement, especially in the case of pulses. This was more evident across the CRS sites, where the evaluators observed that food distributors were not using the standard scooping measures provided for the contingency rations. Save the Children were an active partner in the targeting process led by the Kebele and Woreda Task Force, and were also a member of the Appeals Committee. CRS guidelines indicate that its partner Meki Church Secretariat (MCS) should participate in the targeting process by working with the Kebele and Woreda Task Force, but responses from partner staff in all CRS FDPs reviewed were that they did not participate in the targeting effort and relied on the government to conduct the targeting process. Both IPs received the payroll list from the woreda, against which rations were disbursed to beneficiaries when presenting their Client Cards at the FDP during distribution schedules. Visits to warehouses revealed that both IPs appear to be adhering to USAID’s Regulation 11 and were receiving, storing and dispatching commodities appropriately. All warehouses were located in suitable areas, and offered structures to adequately offer protection and sufficient space to manage commodities. The only exception was the CRS rubhall warehouse in Dire Dawa which was in need of a major renovation. Security across CRS warehouses were found to be relatively sound, with the exception of the privately rented warehouse in Dire Dawa which was managed by the landlord. CRS should consider either managing the security directly, or outsourcing to a reputable third party security firm. In the case of Save the Children, the review team was concerned that security in all FDPs was provided by the communities. Considering the target communities are from the poorest and most food insecure populations of Ethiopia, it may be advisable for Save the Children to review this approach and manage security directly themselves. Finally, record-keeping and reporting for both Implementing Partners covering the periods of FY 2015 and FY 2016 were found compliant with USAID policy when this review ended. Operations were efficient, which included warehouse documentation of retaining and displaying updated BIN cards, commodity status reports, waybills, dispatch authorizations, loss and adjustment reports, and the timely completion of QWICR reports. Sample checks were performed on documents from both the 2014/15 and the 2015/16 periods, and the review team were satisfied that the information reflected in different documents were derived and accurate when collated at different levels. Several notable good practices and lessons learned were found during this review. The review team considered the record keeping of Save the Children to be a particularly good example of the best practices for record keeping and staff training. This was found across the Save the Children commodity supply chain from central level, to primary warehouse, to secondary warehouse, and down to the FDPs. The relationship between Save the Children and the government was another area found to be exemplary. Save the Children actively engaged in the targeting process and retained a copy of ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 32 the Master Beneficiary List, thereby strengthening their internal control system of ensuring ineligible beneficiaries did not manipulate the system by collecting non-entitled commodities. CRS initiated a commendable approach where they required those beneficiaries who were delegating others to collect rations on their behalf, to first obtain approval from the Kebele Task Force. Although beneficiaries were unhappy having to go through this additional step, the review team considers this an approach which enhanced the security of commodities of not being transferred to ineligible recipients. 5. Recommendations 1. General: Covering both Catholic Relief Services and Save the Children a. Payments to receive food: USAID’s cooperating sponsors know that beneficiaries having to pay to receive rations is prohibited by USAID, in any form. Yet, as found in this review, a number of beneficiaries were required to pay 2-3 Birr [the equivalent of $0.09 - 0.13] as a contribution to cover costs such as labor to carry food in the FDP, or to cover the costs of guards in the FDP. Save the Children needs to aggressively communicate to all their field offices to take measures to cease charging beneficiaries, in cash or kind, to receive commodities for the remaining period of Contingency distributions. Considering CRS has concluded their distributions, for future distributions USAID needs to ensure beneficiaries are not required to pay to receive commodities. b. Management of FDPs: A combination of long wait times at most FDPs, as well as the absence of basic accommodations at FDPs has created a difficult environment for beneficiaries and accompanying family members on distribution days. For the remaining Contingency Program distribution period, Save the Children should provide the minimal facilities at FDPs to include safe drinking water, a structure to provide shelter from sunlight or rain, and latrines segregated for females and males. CRS should follow these same suggestions for future programs that may have commodity distributions at FDPs. Better planning could have led to reduced wait times at FDPs. One reason leading to the long wait times observed was the widely-practiced delayed start of actual distribution beginning only after all eligible beneficiaries had arrived. This works against the efficient operation of a FDP. From interviews, the review team found this approach to create enormous frustration in beneficiaries as many of them arrived early, or on time at the FDP, and then were required to wait several hours for other beneficiaries to arrive before the distribution began. When interviewing CRS and Save the Children distribution staff, it was learned that this approach was adopted to enable households to be clustered into different groups based on the number of rations a household was eligible to receive. This allowed the distribution team to then provide bulk rations to different groups for self-distribution. Although the spirit of why this approach was followed is appreciated, it nonetheless delayed the distribution process by hours making it very inefficient. This could easily be addressed by creating dual distribution points at a single FDP and by staggering appointment times over the course of the day. Additionally, special accommodations should be made to fulfill distribution requirements ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 33 for latecomers who may have encountered transportation or other barriers to arriving on time. c. Communication: There are three topics that merit attention to ensure that all beneficiaries are able to recognize and understand: i. the commodities are a gift from the American people; ii. their exact entitlements; and iii. that there is a functioning complaints system. An essential asset to strengthen the communication system is to ensure that the signboard posted at the FDP clearly reflects up-to-date information, along with the USAID logo and other required branding. Beneficiaries should be informed as to what commodities are being disbursed and quantities of each. The name and contact details of whom to contact, should beneficiaries wish to raise questions or concerns, should be clear. There should be additional messages developed and disseminated to ensure beneficiaries that there will be no retribution for complaints. Save the Children has initiated an innovative approach where rather than replacing signboards, which can be costly, they prepare a cloth signboard with the latest contingency rations and other branding information. The cloth is draped over the permanent signboard at FDPs on the day of distribution. This appears to be a good approach to keeping costs down, and should be followed for the remaining duration of the contingency ration distribution. Although references can be made on the signboard of where beneficiaries can lodge complaints, IPs can actively communicate to beneficiaries verbally that they can raise grievances without fear of retribution, to an independent management staff based at the regional or central office, though in most cases this won’t be feasible for beneficiaries. d. Security: As the Store Keeper serves as the only person with keys increases the risk that warehouses can be opened at any time by the Store Keeper, leaving commodities exposed for misappropriation. A minimum of two locks should be attached to all entry points to the warehouse, and keys to one lock be retained by the Store Keeper, and the keys to the second lock retained by another staff. This would provide an added layer of security as the warehouse can then only be opened with the presence of both individuals to open both locks, an added layer of accountability. 2. Catholic Relief Services – Standard Scooping Measures: The majority of beneficiaries interviewed expressed frustration that they were receiving less quantities of pulses than their entitlement. The review team across all sampled CRS FDPs observed that distribution workers were not utilizing standard scooping measures for the Contingency commodities – the 5 kg and 4 kg containers. This almost certainly contributed to incorrect quantities being transferred to beneficiaries. CRS should make it mandatory for future distributions that all FDPs are issued standard scooping containers, and that these are used for the distribution process. 3. Catholic Relief Services – Sire Woreda Client Cards: Across the Sire woreda, for both the sampled FDPs with distributions, and the one FDP with non-distribution, the review team found that none of the beneficiaries of these three FDPs had Client Cards. This exposed CRS to possible misappropriation where ineligible recipients could collect ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 34 commodities as their identities could not be verified via the Client Card. CRS should make it mandatory that all beneficiaries must have Client Cards to collect rations. 4. Catholic Relief Services – FDP Management Committees: The usual composition of these committees is four elected community representatives, with one position reserved for a female. However, across Sire, the review team found that the FDP Management Committee was composed exclusively of different government representatives. Further, for the two distribution Sire FDPs that were observed by the review team during distribution day, neither of the FDP Management Committees were present and overseeing the distribution. The FDP Management Committee is an essential quality assurance mechanism, and it is recommended that in the future, CRS ensures that all such committees are composed of elected community representatives and present at FDPs as tasked. 5. Catholic Relief Services – Sire Woreda Ibseta FDP: The review team, through its interviews with beneficiaries, was made aware that government staff collected 5 Birr at each distribution from all beneficiaries as a contribution to a “savings” fund. The evaluators found no evidence of such a fund, nor any records of where this money is retained or channeled. It is recommended that this be halted immediately and investigated further. 6. Catholic Relief Services – Primary Warehouses, Dire Dawa, and Private Rented Warehouse: The review team considered the security provided by the landlord of the rented warehouse as weak. Security is provided solely by the landlord, with no evidence that CRS has any involvement in this activity or its’ quality control. It is recommended CRS request evidence from the landlord that they are making every effort to provide reliable security. Any weaknesses found must be immediately addressed by the landlord. Alternatively, CRS may renegotiate the rental contract and exclude the security as part of the contract, and directly manage this themselves. Either full-time guards providing 24-hour security in shifts can be retained by CRS, or the security can be outsourced to a third party professional security firm. 7. Catholic Relief Services – Primary Warehouse, Dire Dawa, Rubhall: The CRS owned rubhall (a conventional, large, storage facility) reviewed by the review team was clearly in need of renovation given its old age and decaying structure. This places commodities stored in the rubhall at risk, as the structure offers little protection from rain and pests. It is recommended that CRS overhaul the structure, or find an alternative warehouse. 8. Save the Children – FDP Security: Save the Children’s effort in community engagement, entrusts communities to guard and protect FDPs. Although this can be considered an effective approach to forging better partnerships with communities and also reduce costs, the review team has concluded that in the context of Ethiopia, the risks outweigh the benefits of this approach to security. PSNP beneficiaries comprise the most impoverished and food insecure populations in Ethiopia. Entrusting food to these communities brings unnecessary risk which are prone to spike during lean and hungry seasons when food is not available for hungry families. Food stocks can also be at constant risk as community members can gain easy access, with the potential for small portions to be siphoned off from individual bags without detection. It is recommended that Save the Children budget for appropriate security for all FDPs, and manage this directly rather than rely on communities to provide this service. Alternatively, Save the Children could initiate a ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 35 more rigorous checking process where staff will be required to frequently perform physical counts, and also weigh an statistical sample of bags to ensure commodities remain intact (perhaps using lot quality assurance sampling sample sizes). 9. Save the Children – Yabello Woreda Obda FDP: It is recommended that the locations of all FDPs be carefully selected to be accessible and centrally located to the communities it serves, ideally within acceptable walking distances. Invariably, more remote areas will require beneficiaries having to cover greater distances. The Obda FDP requires immediate attention. When conducting interviews with beneficiaries, the responses to the review team clearly relayed frustration where beneficiaries complained that to reach the FDP from their homes, located in the mountains, was an arduous journey. Upon collecting rations, the return journey was even more hazardous as it inevitably became dark by the time they received rations. Rather than venturing the journey home with heavy rations, many beneficiaries opted to sell their rations locally, far below the existing market price. Save the Children needs to actively review this particular FDP, and work with the communities it serves in this catchment to identify a more suitable FDP location and distribution process. 10. Save the Children – Dollo Ado Woreda Bowbow FDP: It is commendable that Save the Children has energized community participation through the relocation of the Bowbow FDP. This relocation is an excellent example of engaging the local community in decision making. The previous location was very far off and inconvenient with beneficiaries having to cover long distances to reach the FDP. Through this consultative process, the FDP was relocated to its new location in Bowbow. The community actively engaged in establishing the FDP, building a traditional structure as the FDP. The review team observed the structure and determined that it was inadequate to protect commodities from natural elements. It is recommended that Save the Children budget to build a more protective structure suitable to store commodities safely, or relocate to another existing structure in close proximity which can provide more secure storage facilities. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 36 Annex A: Scope of Work from the USAID Request for Task Order Proposals Original USAID DEVELOPMENT FOOD ASSISTANCE PROGRAM (DFAP) COMMODITY MANAGEMENT REVIEW IN SOMALI AND OROMIYA REGIONS I. BACKGROUND The United States Agency for International Development (USAID) plays a lead role in food security programming in Ethiopia and is the largest donor to the Government of Ethiopia’s (GOE) multi-donor funded Productive Safety Net Program (PSNP) as well as humanitarian assistance. USAID provides its assistance through consortium of non-governmental organizations (NGO) implementing partners (IPs) and through the World Food Program (WFP) to address emergency food needs that arise from natural disasters, such as droughts. USAID’s Development Food Assistance Programs (DFAPs), are mechanisms that support the PSNP to address chronic food needs with a goal to make sustained changes to food security and livelihoods of chronically poor communities and reduce their vulnerability to disaster. The DFAPs are implemented by four NGO partners namely, Catholic Relief Services (CRS), Food for the Hungry (FH/E), Relief Society of Tigray (REST) and Save the Children International (SCI). In 2014/15, USAID approved the distribution 83,805 MT of - cereals, pulses and vegetable oil for 1,256,961 PSNP core beneficiaries and 14,123 MT for 251,392 contingency beneficiaries. Similarly, in 2015/2016 (2008 FY), USAID approved the distribution of 126,084 MT of cereal and pulses for about 1,558,142 PSNP base beneficiaries and 19,251 MTs of contingency resources for approximately 227,442 transitory beneficiaries in 44 Woredas of Amhara, Oromia, Tigray, Dire Dawa and Somali regions. Title II food assistance programs are subject to USAID’s 22 Code of Federal Regulations 211, which is commonly known as Regulation 11. This regulation, as well as other USAID directives, outlines commodity management responsibilities, reporting requirements, and claims handling practices that applies to NGO Title II programs. This includes a number of quarterly commodity and loss reports submitted through USAID’s Quarterly Web-Interfaced Commodity Reporting system (QWICR). The focus of this commodity review is to assess the overall food aid commodities management systems, accountability, and control procedures that the DFAPs Title II Implementing Partners (IPs), namely CRS and SCI, apply in Oromiya and Somali regions under the DFAP program. Oversight and guidance for this review work will be provided by the Assets and Livelihoods in Transition (ALT) office of USAID/Ethiopia. A technical team composed of experts within ALT will review and provide feedback on the work plan and draft reports to be submitted by the contractor during the course of the review. II. PURPOSE AND SCOPE OF THE REVIEW The purpose of this review is to assess and determine the Title II commodities approved and received by the two DFAP implementing partners in Oromia and Somali regions, namely Save The Children International, and Catholic Relief Services (CRS) and its sub-grantee Hararghe Catholic Secretariat (HCS) respectively, are managed as per the established standard of USAID and maintains a sufficient internal control systems to manage and utilize Title II food resources efficiently. The review will assess and determine: - Whether the procedures and systems of commodity receipt, management, handling and utilization under the DFAP activities in Oromiya and Somali regions are consistent with USAID regulations and guidelines; ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 37 - Whether DFAP partners follow appropriate standards of best practices in commodity receipt, management, distribution and reporting. The geographical scope of the study will be limited to detail assessment and review of sample Woredas in Oromiya and Somali Regions served by CRS and SCI. The timeframe covers commodity management activities in 2014/15 and 2015/16 (to date). The review will conduct a partner-by-partner analysis through selection of representative sample Woredas, warehouses and systematic review of targeted households. This review will assess and determine key aspects of commodity management and distribution practices, including, but not limited to, the following: A. Distribution of Food Aid - Assess and determine food distributions are conducted per USAID regulations and standards; commodities are distributed to intended beneficiaries; and IPs are using an efficient system for distribution; - Whether distributions are made to entitled beneficiaries per the approved ration size and mix, with no over or under payment of entitlements to beneficiaries; o the communication system related to entitlements practiced at distribution points; The extent of control partners exercise/maintain during food distributions ; Assess the role of local government in the distribution processes, especially at the FDP; Observe distribution process (group based or scooping methods) and comment on the efficiency of distributions systems: o the time beneficiaries spend at distributions and the relative length of one distribution last; o the existence and reliability of predictable transfer schedule (public works versus direct support beneficiaries); B. Food Management Practices and Internal Control System - Map out food management, distribution and reporting systems at all stages - regional, Woreda, and Kebele - and review the technical backing and support provided by regional experts; - Determine whether adequate commodity management practices, safeguarding mechanisms and internal control systems are in place and adhered; - The availability of warehouse staff that have a clear understanding of USAID commodity management rules and regulations, and availability of all the required warehouse equipment (including but not limited to operable fire extinguishers and reconstitution materials) at the warehouse level; - Whether record keeping of all relevant documentation and authorizations is adequate and used at all levels, and commodity movements and transfers are supported by appropriate documentation, including distribution plans, dispatch notes, and any other documentation that authorizes the movement of commodities; - Whether commodity tracking, management, reporting, and handling and disposal of losses follow Regulation 11 requirements and other USAID directives, including reporting through the QWICR system; - The frequency of physical inventories and other commodity monitoring conducted, and the consistency of figures between stocktaking and the records, including answering questions such as: o Is compliance with internal controls reviewed periodically? o Is there an unusual volume of inventory adjustment, write-off, or disposal? o What is the frequency of commodity monitoring visits conducted by the head office? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 38 - The storage/warehouse conditions at primary, secondary and tertiary levels, as applicable, including maintenance of necessary warehouse documentation and records including bin cards, stock control sheets, receiving and issue notes; - The adequacy of commodity loss management practices, measures taken to reduce losses, disposal methods for unfit commodities, and handling and tracking of loss claim payments by third parties or from IPs to USAID; and - Good Lessons drawn from management and internal control system, innovative approaches. C. Reporting Reporting flow, frequency and communications - Type of reporting and frequency; - System for internal data quality validation of various reports (from field level to Addis office); - The adequacy of reporting processes in terms of report/data quality, frequency and timeliness at all levels -- from food distribution points (FDPs) to Woreda and to head offices in Addis Ababa; - Reporting supervision practiced; and - Best Used by Date (BUBD) management – tracking, documentations, reporting and posting. IV. METHODOLOGY A) Methodology It is envisaged that the review will employ a mix of methods and triangulation of data in order to ensure the appropriateness and efficiency of IP’s commodity management and utilization. A.1) Methods The methods should include but are not limited to the following: - Document Reviews, including Reviews of Quarterly Commodity Reports - Key Informant Interviews - Surveys of Beneficiaries/beneficiary interviews - Observations of commodity distributions to beneficiaries - Warehouse visits - Local Market assessment (to analyze utilization vis-a-vi extent of self-monetization) - PSNP beneficiary food collection cards A.2) Data Sources and Collection Methods Primary sources such as beneficiaries, key informants and commodity management personnel, and secondary sources such as IP’s commodity distribution and other periodic reports, are to be used as the sources of data. Qualitative methods will be used for the collection of data among the sampled program beneficiaries and key informants. Sample Selection and Size: CRS DFAP is implemented in 11 Woredas in the Oromia region and SCI implements in four Woredas of Oromia region and three Woredas in Somali Region. The Contractor will prepare a detailed assessment framework including sample size and instruments which will be reviewed and approved by USAID/Ethiopia as part of the Work Plan. . A.3) Data Analysis and Presentation The contractor will apply appropriate data analysis to the qualitative/quantitative data so as to be able to clearly determine the appropriateness and efficiency of IPs’ Title II food management systems, and any corrective measures needed. Before and after completion of the field work, the Contractor will conduct an entry and exit conference with IP staff working in the selected Woredas. Finally, the Contractor is expected to present all the ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 39 findings together with recommendations including: (a) generic issues applicable to all IPs and reviewed Woredas and (b) specific findings (positive and negative) applicable to each reviewed partner. V. RELATIONSHIPS AND RESPONSIBILITIES a) The Contractor The Contractor shall complete the following tasks: - Identify key data sources; - Meet with the Chief of Party of the DFAP IPs, Commodity management and logistics officers, warehouse managers and commodity accountants who will outline commodity management and distribution requirements; - Determine the effectiveness of control systems established by partners for food management and the entire supply chain starting from receipt at the primary warehouse, allocation, dispatch and distribution as well as identification of key commodity management issues; - Prepare and deliver a report as per the required purpose, scope, methodology, tasks and deliverables in this statement of work; - Conduct a presentation on preliminary findings to USAID/Ethiopia; - Submit draft report for review and feedback by the technical team; and - Submit a final report which incorporates feedback from the draft report. b) USAID/Ethiopia The USAID/ Ethiopia’s ALT office will provide overall technical leadership and direction for the review team throughout the assignment and will undertake the following specific roles and responsibilities: - Provide information on the total levels of food resources approved for partners in 2015 and 2016 for DFAP activities in Oromiya and Somali Regions; - Provide reference materials to the Contractor preferably in electronic form: including the Regulation 11 and other relevant guidelines and checklists; - Provide a list of implementing partners in operational Woredas, the number of warehouses, and contacts; - Throughout the assessment work, ensure constant availability of a point of contact who can provide technical leadership and direction for the team’s work; - Introduce the assessment team to implementing partners and other stakeholders; and - Provide timely review (within five working days) of the work plan, draft/final reports and approval of the deliverables. VI. DELIVERABLES AND PRODUCTS The Contractor review is expected to deliver the following outputs: - Provide an initial work plan which includes data collection techniques, a list of sites and warehouses per Woreda to be reviewed, and a site visit schedule. USAID will provide approval of the work plan; - Presentation of preliminary review findings and submission of a draft report and PowerPoint materials to USAID before submitting the final report; and - Submission of a final report incorporating feedback from USAID review of the draft report, as well as submission of raw data collected. VII. LOGISTICS USAID will not provide any logistical support. The Contractor will, therefore, be responsible for arranging the required travel and logistics. USAID will inform NDRMC and implementing partners of the planned commodity review that will be conducted during the assigned period. VIII. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORTS ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 40 a) Skills and Team Composition Team Composition: The assessment will be conducted by a team of international and local experts with experience and skills that are applicable to carrying out this review. The Assessment Team should be consisting of one Team Leader plus 3 – 6 Team Assistants/Commodity management Experts. The Team Leader will be responsible for organizing the review, reviewing all relevant documentation, producing a work plan, assembling and coordinating Team Assistants, planning and oversight of data collection efforts, and ensuring the timeliness and quality of deliverables. The Team Assistants will be responsible for various data collection and analysis responsibilities as directed by the Team Leader. Team Qualifications: It is anticipated that the Task Order will be carried out by a firm with commodity management expertise with the following skills: - Demonstrated qualifications, experience, and skills related to commodity management review, audits and assessments applicable to conducting this review. - Technical competency in related fields such as accounting, economics or logistics management. - Substantial experience with data collection procedures, surveys, and analysis and interpretation of databases. - Good communication skills and the ability to interact with Ethiopians and expatriates, including at least one team member fluent in the regional languages (Oromiya and Somali regions). b) Level of Effort and Timeframe The level of effort (LOE) for the Team Leader is expected to be about 55 days. Below is an illustrative timeframe. Note that these may be modified based on availability of contractors, key stakeholders and time for fieldwork. The bidder is expected to submit a detailed LOE, including staffing needs. ACTIVITY RESPONSIBLE PARTY PERIOD OF PERFORMANCE (Tentative Date) Final award Contract USAID September 2, 2016 Submission of Approval Work plan USAID and Team Leader September 08, 2016 Field work/Data collection Assessment Team September 12-30, 2016 Data Analysis, Develop preliminary finding, draft report and presentation of findings to USAID assessment team Assessment Team October 01-11, 2016 Presentation to USAID Assessment Team Assessment Team October 12, 2016 Submission of Draft Report Assessment Team October 17, 2016 Review of Draft Report USAID October 18-31, 2016 Final Report Submitted to USAID including raw data Team Leader November 8, 2016 IX. REVIEW PROCEDURES a) Team Planning Meeting (TPM): The assessment will commence with a Team Planning Meeting (TPM) to allow USAID/Ethiopia and the Contractor to clarify expectations and the purpose of the assignment. The meeting will discuss roles and responsibilities, timeline, inputs, data collection techniques, and logistical and administrative procedures in relation to the assignment, and to discuss procedures for resolving any differences of opinion. The Contractor will develop a methodology for data collection, including identifying information sources, developing data collection techniques, and establishing a sampling plan frame, which will be included in the work plan for USAID concurrence. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 41 b) Interim Briefings including status reports: The Team Leader will provide a bi-weekly status briefing to USAID-Ethiopia throughout the assignment, either in person or by telephone. c) PowerPoint Presentation (in MS PowerPoint) used during debriefing to USAID/Ethiopia staff on the preliminary findings and recommendations that address set of objectives and associated questions. d) Draft report in English no longer than forty pages, excluding coversheets and appendix. The report shall follow the general format indicated below: (i) Coversheet indicating the title of work performed (ii) Table of Contents (iii) Acknowledgments (iv) Acronyms (v) Executive Summary (vi) Introduction and Background (vii) Scope and Methodology (viii) Findings (ix) Conclusions and Lessons Learned (x) Recommendations (xi) References (xii) Appendix (includes, but not limited to, SOW, data collection instruments, sources identified, and list of people contacted or interviewed, if any). e) Raw Data: The assessment team will provide electronic files of all raw data to USAID/Ethiopia for future use. f) Final Report: The final report bust adequately address the USAID/Ethiopia’s comments on the first draft. The Contractor shall submit the edited and formatted final document in 5 hard copies and electronically in Word format. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 42 Annex B: Team and Timetable The DFAP Review team was comprised of one expat (evaluation expert) and five Ethiopian experts in commodities, food aid and evaluation. - Mr. Faheem Khan was the Team Leader and had overall responsibility of the review, coordinating and supervising the two field teams, and the Commodity Management Expert responsible for secondary documentation review. He accompanied the field teams at the start of the survey, and although planned to continue in the field, was forced to change plans and work from Addis Ababa after completing trips in Adama and Dire Dawa, due to the unrest across the Oromia region where travel for foreigners was severely restricted. Nonetheless, Mr. Khan continued to work effectively providing daily guidance and check-ins with the two field teams, and conducting analysis on a daily schedule. Mr. Khan prepared and delivered the formal presentation to USAID Ethiopia on October 20, 2016, led the overall analysis of the evaluation and was the principal author of this report. - Mr. Temesgen Burka and Mr. Benti Erena served as the Commodity Management Experts and were responsible for various data collection and analytical works as directed by the Team Leader. They took lead roles in planning, managing, and supervising the commodity review, and provided ongoing and regular cataloguing, tracking, synthesis, and analysis of findings. Mr. Burka led a large share of Key Informant Interviews (KII) with commodity managers, warehouse staff, and other stakeholder interviews and oversaw a portion of beneficiary surveys. Mr. Erena led the review of secondary documents, coordinating and sharing findings with the Team Leader. - Dr. Mesfin Beyero served as Evaluation Specialist with considerable participation in meetings, planning, training, travel, field visits, interviews, group discussions and analysis. He played a lead role in facilitating logistics and travel for the review team. Dr. Beyero supported the Team Leader in analyzing the results and findings and produced written material for the report. - Mr. Temesgen Wario and Mr. Solomon Woldeab were the field researchers in food accounting. They both participated broadly in meetings, planning, field visits, interviews, group discussions, and analysis. They were responsible for ongoing and regular cataloguing, tracking, synthesis and analysis of findings. ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 43 DFAP Review Timetable ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 44 ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 45 ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 46 Annex C: Warehouse Questionnaire WAREHOUSE QUESTIONNAIRE Notes: the Respondent for this questionnaire is the Warehouse Supervisor or other relevant NGO staff. All responses need to be verified by formal documentation or by observation and physical checking from the evaluator. Date: Partner (SC/CRS): Evaluator’s Name: Region: Respondent’s Name: Zone: Respondent’s Position: Woreda: Warehouse Name: Warehouse Type (Primary/Seconda ry): Questionnaire 1 – Warehouse Structure and Equipment Item Yes (enter 1) Yes, but needs to improv e (enter 1) No (ente r 1) Comments Is WH easily accessible with ample space for trucks to turn around and for loading and unloading? Is WH on solid ground? Are walls and ceilings protected from leaks? Is floor made of concrete and of a smooth surface? Is there proper drainage for water collection in the yard? Are roads leading up to WH accessible in all weather? Does WH have solid and secure doors? Is entrance wide enough for smooth transfer of commodities? Does WH have electricity? Are these in good working order? Is there a circuit breaker? Does WH have sufficient space for re￾constitution of commodities? Are weighing scales calibrated? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 47 What is the MT capacity of the WH? (Capacity(MT)={(Length-1xNo. Rows) x (Width￾1xNo.Rows)x(Height-1)} / 2) Does WH capacity match requirements of commodity being received? Does WH construction protect from weather, pests and theft? Is there ventilation, and is this protected from rodents / insects? #NAME? Are communication systems in working order (radio, telephone, modem, etc.)? Does WH have following items? Table, chair, filing shelf? Stapler and staple pins? Hole punch and filing clips? Calculator? Office stationary (pens, paper, pencils, eraser, sharpeners, etc)? Registers, ledgers, reporting formats? First aid kit? Pallets / plastic sheets? Platform weighing scales (100kg capacity)? Are weighing scales calibrated? Fire extinguishers? Are fire extinguishers in good working order (i.e. not expired + has sufficient liquid inside)? Are guards and warehouse staff trained on using fire extinguishers? Flash lights and batteries? Bucket with stand and water? Warehouse cleaning materials (brooms, buckets, etc.)? Rat glue, rat traps? Empty bags and containers for reconstitution? Standardized scooping equipment? Sieve / sorter? Ladders? Sampling spears? T-wine and needle/bagging machine? What is the management structure adopted for commodity management? How are management and operational duties segregated? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 48 Questionnaire 2 – Commodity Storage Are ONLY USAID commodities stored in warehouse? If NO, are USAID commodities stored separate to commodities from other donors? Are stacks 1 meter away from walls and ceiling? Are stacks 1 meter away from each other? Are plastic sheets or pallets used for stacking commodities? Are stacks for processed commodities less than 25 levels in height? Are stacks for oil less than 10 levels in height? Is FIFO method adopted? Are commodities rotated if they are retained for long periods? Are stocks inspected daily to see signs for infestation or damage? Is there more than 1 type of commodity in a stack? Are the exterior of commodity packages clean and undamaged? Are oil cartons stacked upright? Is there a layout stack plan before commodities arrive? Are different shipments stacked separately from each other? Are processed food commodity bags bonded or interlaced to construct stack? Are bags lined up with edge off pallet? Are the same number of bags placed in each level? Are separate stacks and cards for each food type maintained for each shipment? Are stack cards attached to the stack? Are separate stack cards maintained for food unfit for human consumption? Are these stack cards attached to the stack unfit for human consumption? Are separate stack cards maintained for food to be repackaged? Are the stack cards attached to the stack for repackaging? Does information on stack cards compare to the inventory ledger? Does the information match? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 49 Questionnaire 3 – Security Does WH have good quality locks? Does WH have 24 hour guard service? Who has access keys to the WH? Are there surprise visits to check on security? Who performs these surprise visits? Questionnaire 4 – Fumigation and cleaning Is regular spraying conducted to eliminate weevils and pests? How often are fumigations performed? Is WH cleaned before arrival of commodities? Is WH cleaned and swept after food receipt and dispatched? How often is the warehouse cleaned on a routine basis? Are there any rodents observed, or signs of rodents such as droppings, tracks, burrows, holes or signs of feeding? Is there any refuse that can attract rodents/insects? Is there any food observed on the floor? Questionnaire 5 – Receipt of Commodities Is there a plan of how commodities are transferred from WH to FDP? Does the plan take into consideration seasonal changes? If YES, are Turn-Around-Times for trucks established? At the time of receipt, are damaged/slack/torn/leaking units separated for reconstitution? Are these damaged commodities stored separately? Is reconstitution performed by a committee formed by management? Who are the members of this reconstitution committee? Are stack cards appropriately displayed and updated as and when transaction occur? Questionnaire 6 – Checks and Balances and Record Keeping Are daily/weekly commodity receipt confirmations maintained? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 50 User ledgers maintained? Monthly commodity status reports? Are BIN cards properly filled containing the correct information? -commodity type -program name -date received -number of units received -units and dates of dispatch/issue -control number -balance Physical count and commodity reconciliation report filed? Daily WH activity report? Loss and adjustment report? Waybills as per commodity, control number and month (separated by incoming and outgoing Waybills)? Tally sheets Handing and taking over file? Furniture and fixture file? Incoming letters file? Dispatch authorization? Monitoring file? Weekly CSR? Transmittal memo file? How often does physical counting of commodities take place? Who performs the physical count? Are Waybills filled out in its entirety by WH Officer, Transporter, and Receivers of commodities? Are sample signatures of WH Officers kept at a central level? Are bags counted when loaded on trucks? WH Inventory Ledgers Are ledgers maintained for different shipments? Is there a shipment number assigned for each shipment for tracking purposes? Are all receipts, issues, losses recorded in a ledger? Are there a Delivery Survey reports? Are there Waybills maintained? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 51 Are there Loss and Adjustments reports? Are ledgers closed at the end of each month? Are Carry Forward and Opening Balances recorded? Are two separate inventory ledgers maintained – one for good and one unfit/damaged goods? Are losses and adjustments recorded in the pending column of ledger? Are losses submitted for approval from management? Who approves these losses? Once approved, are copies routed to the original warehouse? Are negative entries made to remove losses from pending column of ledgers and the amount entered in loss column of ledger? Does the balance in the pending column represent loss and adjustment amounts not approved? Commodity Status Report Are monthly and quarterly CSRs maintained? Reports in KG? Are physical inventories completed at the beginning of the reporting period? Are all receipts added and reported during a period? Are all distributions for a period added and reported? Are balances for a period regularly calculated? At the end of a reporting period, are physical counts reported and documented? Are there differences between physical count and balances – are these reported and explanations provided? To whom are these reports sent to? Recipient Status Report Are RSRs prepared monthly? Commodity Loss Status Report Evidence of Commodity Loss Report? Does this report type of commodity lost? Location of loss? Are recovered commodities reported? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 52 Annex D: Questionnaire Employed at Food Distribution Points Annex 3 – FDP QUESTIONNAIRE Notes: The respondent for this questionnaire is the FDP Supervisor. All responses need to be verified by formal documentation or by observation and physical checking from the evaluator. Date: Partner (SC/CRS): Evaluator’s Name: Region: Respondent’s Name: Zone: Respondent’s Position: Woreda: Name of WH serving FDP: FDP Name: Item Yes (enter 1) Yes, but needs to improve (enter 1) No (enter 1) Comments Is there documentation showing the number of beneficiaries served by the FDP? Is there documentation on entitlement per beneficiary (ration size)? Are the distribution periods set? Has the distribution schedule been shared with beneficiaries? Does the planned distribution date match with the actual distribution date? Is FDP accessible by truck? If NO, and trucks cannot access FDP, how are commodities delivered to FDP Are there adequate facilities to store commodities at FDP? Does the store keeper at FDP count the number of bags / containers received at FDP? Does the store keeper weigh a sample of commodities received to ensure no loss/theft has occurred during transportation? If yes, has weighing machine been calibrated? If yes, what is the sample size? Does the store keeper fully complete the receipt form? Are stack cards and beneficiary lists maintained at FDP? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 53 Are records kept of the quantity of food taken each day during distribution? Does FD prepare monthly distribution reports? Does the report contain the following: -planned number of beneficiaries eligible to receive food? -total amount of food authorized to be distributed? -actual amount of food distributed? -approved individual ration size and the actual ration size distributed? Does FDP maintain an up-to-date display board indicating name of USAID and ration size to be distributed? Is there an effective crowd control mechanism in place? Do recipients acknowledge the receipt with thumb print or signature? Are commodities weighed and/or scooped in front of the beneficiary? Is this being performed properly? Are beneficiaries identified as eligible before handing over rations to them? Does FDP puncture oil tins and open all containers before issuing out? Has FDP issued ration cards? If yes, are these being effectively used? If no, how are beneficiaries identified? Is there any evidence that food aid is being diverted to the market? Is there evidence of people abusing this through fake names, double names? Are there absent beneficiaries? If yes, what percentage? How are beneficiaries marked after they receive rations, to ensure they do NOT take a second ration? How are multiple registrations of beneficiaries detected/eliminated where a HH is not able to take rations from multiple FDPs, or have other HH members collected more than 1 ration? What is the shortest wait time at FDPs for beneficiaries to receive rations? (nearest hr)? What is the longest wait time at FDPs for beneficiaries to receive rations? (nearest hr)? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 54 Do the beneficiaries have any input to determine where to setup the FDP? Which HH member usually comes to the FDP to collect rations? Is there a committee at the FDP overlooking the ration distribution? If YES, who are the members of this committee? What proportion of these committee members are women? Any comments/observations from Committee Members regarding commodity management and distribution? Are there any government counterparts present? Y / N If the government is present - what is their title? What observations/comments does the Government Officer have on commodity management and distribution? Any further observations/comments from the field researcher? ETHIOPIA DFAP COMMODITY ASSISTANCE REVIEW FINAL REPORT AID-603-TO-16-00002 International Business & Technical Consultants, Inc. 55 Annex E: End-Use Beneficiary Questionnaire 1. 2. Particulars of Receipent Direct Participants Household 1.1 Region - Name : ______________________ 2.1 HH Unique ID Code: 1.2 Zone - Name : ______________________ 2.2 Ration Card No. 1.3 Woreda - Name : ______________________ 2.3 Name of Beneficiary : ________________________ Member ID: 1.4 Kebele - Name : ______________________ 2.4 Household Head Name : _______________________________ 1.5 Village - Name :______________________ 1.6 Name of Implementing Partners :_____________________________ 1.7 Date : ______________________ 3. Total members of the household 4. 4.1 How many times/months did you receive the ration? 4.2 According to the Ration Card how many times did ben. receive ration? 4.3 According to the Registration how many times did ben. receive ration? 4.4 Deviation Found: (1=Yes, 2=No) 4.5 If yes explain the reasion:________________________________________________________ 5. 5.1 According to the Ration Recipient, how many days earlier did they receive the last ration? Days 5.2 According to the Ration Card how many days before did recipient receive last ration? Days ; 5.3 Deviation Found:(1=Yes, 2=No) 6. Who received your last ration from the FDP? (Use Code) (1=Self, 2=Nominee, 3=Others________specify) 7. How long did beneficiary need to wait to receive ration this time? hrs (Round Figure) 8. 8.1 Distance of FDP from Beneficiary's House? km 8.2 How long does it take for beneficiary to reach FDP from your HH? hrs (rounded) 9. 9.1 Do you face difficilties collecting rations from FDP? (1=Yes, 2=No) 9.2 If yes explain:_______________________________________________________________ 10. 10.1 How happy is the Beneficiary with the ration distribution process ? (Give Tick√ Mark) 10.2 Why do they feel this way: __________________________________________________________________________________________ 11. 11.1 Did you pay for getting the ration? (Use Code: 1=Yes, 2=No) 11.2 If yes, then what type? (e.g. food share/money/other) 11.3 To Whom: 12. 12.1 Do you know who the Ration provider is? 1=Yes, 2=No 12.2 If yes, then use code (Code: 1= USAID/ From American People, 2=NGO, 3=Others (specify): 13. Give opinion on behaviour of the person providing you the ration (Use Code) (Code: 1=Very Satisfactory, 2= Satisfactory, 3= Fair , 4=Not Satisfactory, 5=Not Applicable) 13.1 NGO Staff Reason of Opinion: __________________________________________________________________________ 13.2 CHV/Volunteers Reason of Opinion: __________________________________________________________________________ 13.3 Other (specify) Reason of Opinion: __________________________________________________________________________ 13.4 Scooper Reason of Opinion: __________________________________________________________________________ 14. 14.1 Do you know how much food you are entitled to? (Use Code: 1=Yes, 2=No) (if No Skip 14.2 &14.3) 14.2 If yes, mention quantity of ration per Month per person (break down by quanity for each commodity): 14.3 Is the beneficiary's response correct? (Use Code: 1=Yes, 2=No) 15. 15.1 Do you use clean Container/Pot to carry Veg. Oil from the FDP? (Use Code: 1=Yes, 2=No) 15.2 Do you use clean Container/Pot to Store Veg. Oil at house? (Use Code: 1=Yes, 2=No) (Answer after Validation) 16. 16.1 Do you have any opinion/complaints related to the FDP or commodities? (Use Code: 1=Yes, 2=No) 16.2 If YES, please explain: 16.3 Do you have any opportunity to raise concerns/complaints? (Use Code: 1=Yes, 2=No) 16.4 If Yes, than what is the option (Use Code) (Code: 1=Mobile/Telephone Number, 2=Help Desk at FDP, 3=Complaint Box at FDP, 4=Others____________Specify 17. Any other observation/comments (evaluator) Name of the Participant:_____________________ Name of Evaluator:______________ General Information