PHOTOS: CARE KORE LAVI HAITI MID-TERM EVALUATION April 2016 This publication was made possible by the generous support of the American people through the United States Agency for International Development (USAID). This evaluation was conducted by Absolute Options (AO) on behalf of CARE and USAID. The opinions expressed herein do not necessarily reflect the views of USAID or the United States Government. Kore Lavi Title II Program Haiti – Midterm Evaluation 2 1. Executive Summary..............................................................................................................................................7 2. Introduction........................................................................................................................................................ 22 2.1 Kore Lavi Background................................................................................................................................. 22 2.2 Kore Lavi Results Framework................................................................................................................... 22 3. Evaluation Purpose............................................................................................................................................ 27 4. Methodology....................................................................................................................................................... 28 4.1 MTE Evaluation Team.................................................................................................................................. 28 4.2 Data Collection Methods and Data Sources.......................................................................................... 28 4.3 Evaluation Fieldwork.................................................................................................................................... 30 4.4 Data Analysis................................................................................................................................................. 32 4.5 Known Limitations to the Evaluation Methodology............................................................................. 32 5. Evaluation Findings............................................................................................................................................ 34 5.1 SO1: National Systems for Vulnerability Targeting Strengthened .................................................... 34 5.1.1 Background .................................................................................................................................................... 34 5.1.2 Timing of Key Milestones........................................................................................................................... 35 5.1.3 Evaluation Approach.................................................................................................................................... 37 5.1.4 Development of Proxy Means Indicators and Data Collection Surveys.......................................... 37 5.1.5 Initial Data Collection and Use Challenges............................................................................................ 37 5.1.6 Skewed Deprivation Reporting ................................................................................................................. 38 5.1.7 Issues with Census Approach to Field Surveys..................................................................................... 38 5.1.8 Convergence on Definitions and Methods............................................................................................. 39 5.1.9 Stakeholder Buy-In....................................................................................................................................... 39 5.1.10 Conservative Definition of Deprivation.................................................................................................. 39 5.1.11 Targeting Efficacy.......................................................................................................................................... 40 5.1.12 Variability in Inclusion and Exclusion Errors.......................................................................................... 40 5.1.13 Lack of Initial Strategic Plan........................................................................................................................ 41 5.1.14 Quality Problems with Kore Fanmi Data................................................................................................ 41 5.1.15 Human Resource Deficiencies at MAST................................................................................................. 42 5.1.16 Inadequate Compensation of MAST Staff............................................................................................... 42 5.1.17 Institutionalization of Information System.............................................................................................. 42 5.2 SO2: Access of extremely vulnerable households to local and nutritious foods increased ....... 43 5.2.1 SO2 Activities and Outreach..................................................................................................................... 43 5.2.2 Voucher Activities........................................................................................................................................ 43 5.2.3 Impact of Voucher Activities ..................................................................................................................... 44 5.2.4 Technical and Other Issues Related to Voucher Activities................................................................ 46 5.2.5 Market Impact............................................................................................................................................... 48 5.2.6 Village Savings and Loan Associations...................................................................................................... 49 TABLE OF CONTENTS ii Kore Lavi Title II Program Haiti – Midterm Evaluation 3 5.3 SO3: Maternal and Child Nutritional Status Improved ....................................................................... 50 5.3.1 Insufficiency of Food Rations..................................................................................................................... 50 5.3.2 Quality of Food Rations.............................................................................................................................. 52 5.3.3 Issues with Beneficiary Registration......................................................................................................... 52 5.3.4 Issues with Ration Distribution................................................................................................................. 52 5.3.5 Cascade Training Model.............................................................................................................................. 53 5.3.6 Anecdotal Evidence of Behavior Change among SO3 Beneficiaries................................................. 54 5.3.7 Helpfulness of CADEPS and CASECs .................................................................................................... 55 5.3.8 Institutionalization of the Care Group Approach................................................................................. 55 5.3.9 Conditions at Health Centers................................................................................................................... 55 5.3.10 Treating Moderate and Acute Malnutrition ........................................................................................... 56 5.3.11 Training of Health Center Staff and Others........................................................................................... 56 5.3.12 Issues with MAM Outreach ....................................................................................................................... 57 5.3.13 Compensation for CHAs and Lead Mothers......................................................................................... 57 5.3.14 Demand for Additional Training ............................................................................................................... 58 5.3.15 SO3 Performance Monitoring ................................................................................................................... 58 5.3.16 Primary Benefits as Perceived by SO3 Beneficiaries............................................................................ 58 5.3.17 SO3 Sustainability Strategy......................................................................................................................... 59 5.4 SO4: Haitian institutions’ capacity to effectively lead and manage safety net programming improved...... 60 5.4.1 Background .................................................................................................................................................... 60 5.4.2 Evaluation Approach.................................................................................................................................... 60 5.4.3 Demand and Supply Gaps for Institutional Support at MAST............................................................ 60 5.4.4 Kore Lavi’s Facilitation of MAST-Led Coordination ............................................................................ 61 5.4.5 Creating Institutional Buy-In at MAST..................................................................................................... 61 5.4.6 Lack of Institutionalization Plan for MAST ............................................................................................. 61 5.4.7 Infrequent and Inadequate Capacity Development Trainings............................................................ 61 5.4.8 MSPP and CHA Involvement in Kore Lavi.............................................................................................. 62 5.4.9 Lack of Budgetary Transparency............................................................................................................... 62 5.4.10 Weak Program Communication and Outreach .................................................................................... 62 5.4.11 Financial Constraints.................................................................................................................................... 62 5.4.12 Effect of Staffing Shortfall............................................................................................................................ 63 5.4.13 Community-Level Engagement.................................................................................................................. 63 5.4.14 Capacity Development at CADEPs and CSOs...................................................................................... 63 5.4.15 Inadequate Training for CADEPs and CSOs in Organizational Strengthening............................... 63 5.4.16 Lack of CADEP Engagement from Remote Locations......................................................................... 63 5.4.17 Role of Social Accountability Manager.................................................................................................... 64 5.4.18 Effect of Grants Culture on CSO Enthusiasm....................................................................................... 64 5.4.19 Asymmetric Engagement of CADEPs and CSOs.................................................................................. 64 iii Kore Lavi Title II Program Haiti – Midterm Evaluation 4 5.4.20 GOH Capacity to Respond to Food Emergencies............................................................................... 65 5.4.21 Actions to Improve SO4 Programming and Initiatives........................................................................ 65 5.5 Gender............................................................................................................................................................ 65 5.5.1 Integration of Gender Considerations in Kore Lavi............................................................................. 65 5.5.2 Integration of Gender Considerations in SO1 ...................................................................................... 65 5.5.3 Integration of Gender Considerations in SO2 ...................................................................................... 66 5.5.4 Integration of Gender Considerations in SO3 ...................................................................................... 68 5.5.5 Integration of Gender Considerations in SO4 ...................................................................................... 69 5.5.6 Integration of Gender Considerations in SSN Programming............................................................. 70 5.5.7 Cascade Approach to Gender Training and Sensitization .................................................................. 71 5.5.8 Collaboration with MCFDF ....................................................................................................................... 71 5.5.9 Human Resource Issues.............................................................................................................................. 72 5.6 Monitoring, Evaluation, Accountability and Learning ........................................................................... 72 5.6.1 Components of the MEAL System........................................................................................................... 72 5.6.2 Complexity of MEAL System..................................................................................................................... 73 5.6.3 Compartmentalization of MEAL System................................................................................................. 73 5.6.4 Strategy ........................................................................................................................................................... 73 5.6.5 Indicator Frameworks................................................................................................................................. 74 5.6.6 Tableau du Bord Kore Lavi ........................................................................................................................ 75 5.6.7 Post-Distribution Monitoring Surveys..................................................................................................... 75 5.6.8 K&L and Information Systems................................................................................................................... 76 5.6.9 SO1 and SO4................................................................................................................................................. 77 5.6.10 SO2.................................................................................................................................................................. 77 5.6.11 SO3.................................................................................................................................................................. 77 6. Conclusions......................................................................................................................................................... 79 6.1 SO1: National Systems for Vulnerability Targeting Strengthened .................................................... 79 6.2 SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased.. 80 6.3 SO3: Maternal and Child Nutritional Status Improved ....................................................................... 82 6.4 SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved. 84 6.5 Gender............................................................................................................................................................ 88 6.6 Monitoring, Evaluation, Accountability and Learning ........................................................................... 90 7. Recommendations............................................................................................................................................. 91 7.1 SO1: National Systems for Vulnerability Targeting Strengthened .................................................... 91 7.2 SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased.. 92 7.3 SO3: Maternal and Child Nutritional Status Improved ....................................................................... 92 7.4 SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved. 95 7.5 Gender............................................................................................................................................................ 97 7.6 Monitoring, Evaluation, Accountability and Learning ........................................................................... 98 iv Kore Lavi Title II Program Haiti – Midterm Evaluation 5 Annex 1: Scope of Work (SOW)................................................................................................................................. 100 Annex 2: Evaluation Team Profile and Biographies.................................................................................................. 112 Annex 3: List of Key Informants Interviewed............................................................................................................ 115 Annex 4: SO4 Institutionalization Dashboard - September 2015......................................................................... 117 Annex 5: Kore Lavi Operations Dashboard - September 2015............................................................................ 119 Annex 6: SO1 Discussion Guides................................................................................................................................. 120 Annex 7: SO2 Discussion Guides................................................................................................................................. 122 Annex 8: SO3 Discussion Guides................................................................................................................................. 124 Annex 9: SO4 Discussion Guides................................................................................................................................. 128 Annex 10: Gender Discussion Guides........................................................................................................................... 131 Annex 11: MEAL Discussion Guides.............................................................................................................................. 149 v Kore Lavi Title II Program Haiti – Midterm Evaluation 6 List of Acronyms and Abbreviations ACF Action Contre la Faim International ASCP Multi-disciplinary Health Agent AO Absolute Options CADEP Participatory Development Support Council (Conseil d’Appui au Développements Participatif) CASEC Collective Territorial Administrative Councils (Conseils d’Administrations des Collectivités Territoriales) CBO Community Based Organization CHA Community Health Agent CNSA National Coordination for Food Security (Coordination Nationale de l SecuriteAlimentaire) COP Chief of Party CSC Community Scorecard CSO Civil Society Organization DCOP Deputy Chief of Party DPC Department of Civil Protection (Direction de la Protection Civile) FAES Social and Economic Assistance Fund (Fonds d’Assistance Economique et Sociale) FCS Food Consumption Score FDHS Food Assisted Development and Humanitarian Support FFP Food for Peace FGD Focus Group Discussion G2G Government-to-Government GBV Gender Based Violence GHI Global Hunger Index GOH Government of Haiti HDDS Household Dietary Diversity Score HDVI Haitian Deprivation and Vulnerability Index HHS Household Hunger Scale HTG Haitian Gourde IHSI Haitian Institute for Statistics and Data Processing (Institut Haitien de Statistiqueet d’Informatique) IR Intermediate Results IYCF2 Infant and Young Children Feeding IPTT Indicator Performance Tracking Table KII Key Informant Interview K&L Knowledge and Learning KLC Knowledge and Learning Coordinator KLWA Kore Lavi Web Application LAYERS LQAS Analysis Information Entry & Retrieval System LQAS Lot Quality Assurance Sampling MAM Moderate and Acute Malnutrition MAST Ministry of Social Affairs and Labor (Ministère des Affaires Socialeset duTravail) MCFDF Ministry for Women’s Affairs and Women’s Rights (Ministère à la Condition Féminine et aux Droits des Femmes) MCHN Maternal & Child Health & Nutrition M&E Monitoring and Evaluation MEAL Monitoring, Evaluation, Accountability and Learning MOU Memorandum of Understanding MSPP Ministry of Public Health and Population (Ministère de la Sante Publique et de la Population) PHAST Participatory Hygiene and Sanitation Transformation PM2A Preventing Malnutrition in Children Under 2 RODEC Resseaux Organization for Development of Cerca Cavajal (Resseaux des Organisations pour le Development de Cerca Cavajal) SBCC Social and Behavioral Change Communications vi Kore Lavi Title II Program Haiti – Midterm Evaluation 7 7 7 1. Executive Summary Evaluation Purpose This reports presents the findings, conclusions and recommendations related to the Kore Lavi mid-term evaluation. The Kore Lavi mid-term evaluation is a “performance evaluation,” which is defined in the USAID Evaluation Policy as an evaluation that focuses “on descriptive and normative questions: what a particular program or program has achieved . . . how it is being implemented; how it is perceived and valued . . . and other questions that are pertinent to program design, management and operational decision making.“ As such, the evaluation did not attempt to assess whether expected outcomes or impacts had occurred nor what was the program’s contribution to such outcomes or impacts. Consistent with this purpose, the evaluation had the following objectives: 1. To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. 2. To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with the Government of Haiti, particularly the Ministries of Social Affairs and Labour (MAST), Health (MSPP) and Gender (MCFDF). 3. To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). 4. To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. 5. To recommend the future orientation (e.g., adjustments to the theory of change, program design, resource allocation, or implementation) and emphasis of the program during its remaining time. Description of Kore Lavi Kore Lavi’s Theory of Change holds that positive and lasting transformation must happen within interrelated domains: (1) where the effective social safety net programming and complementary services reach the most vulnerable populations and protect their access to food while building self-reliance; (2) that achieve breadth and depth in behavior and social change needed to tackle under-nutrition among vulnerable women and children; and (3) that institutionalize accountability, transparency and quality of delivery for mutually reinforcing social protection programs under the leadership of MAST. Towards these ends, Kore Lavi has the following four Strategic Objectives (SOs) supported by 10 Intermediate Results. SO1 National Systems for Vulnerability Targeting Strengthened: SO1 covers three main outputs, which are closely related to each other and which are supposed to be achieved through coordination and joint efforts of Kore Lavi and MAST. First, develop, test and adjust a national Haitian Deprivation and Vulnerability Index (HDVI), which would allow the program to measure vulnerability in a multidimensional way using proxy means calculated through a customized algorithm. Second, define a specific data collection methodology Kore Lavi Title II Program Haiti – Midterm Evaluation 8 8 8 implemented by other organizations on a door-to-door basis, so as to allow Kore Lavi to benefit from already-existing data. Third, develop and implement a database application hosted by MAST, which is a tool to store, analyze and manage collected data. The main function this system is to answer key questions about the structure and distribution of household deprivation or vulnerability so as to allow decision makers to develop an appropriate frame of social assistance for future safety net interventions. SO2 Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased: Under SO2, Kore Lavi had developed voucher transfer modalities prioritizing the use of local foods so as to deliver food vouchers to a target of 18,150 extremely vulnerable households. Kore Lavi uses two types of vouchers: paper vouchers and electronic vouchers allocated for the purchase of local staple foods. The fresh food is purchased in the community markets, and the staple food is purchased in small community shops in both cases from a pool of vendors selected by the program through a rigorous process. Under SO2, Kore Lavi is also promoting and facilitating participation in village savings and loan associations (VSLAs). A VSLA is a group of people – from the same given community – who save together and take small loans from those savings. Kore Lavi is facilitating the creation of VSLAs with a target of 18,000 VSLA members in order to help integrate as many program safety net beneficiaries as possible into VSLA groups and to support the gradual auto-graduation from the safety net of less vulnerable beneficiaries SO3 Maternal and Child Nutritional Status Improved: Under SO3, Kore Lavi implements a social behavior change communications strategy in line with the preventive approach focusing on the window of opportunity in the first 1,000 days of life. The program provides supplementary conditional rations to pregnant and lactating women and children aged 6-23 months. The SBCC strategy is mainly implemented through a Care Group approach, which is a group of is a group of 10 women named “Lead Mothers” (or in limited cases “Lead Fathers”) who are identified by the community and come together for mutual support and learning and who are regularly trained by the local community health agents (CHAs). In order to reinforce the knowledge of the community members regarding proper health and nutrition behavior and practices, Kore Lavi collaborates with the existing network of community health agents belonging to the Ministry of Health, while sensitizing through various community public awareness raising campaigns, key community structures, civil society organizations and traditional health workers. Moreover, to reinforce health and nutrition services provided by the health facilities, the program conducts trainings for health professionals and primary health care workers and supports them to provide services via a network of rural health centers. SO4 Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved: Under SO4, Kore Lavi seeks to: (1) implement a joint USAID/MAST capacity building and resource mobilization plan; (2) work with MAST national and department-level staff on coordinating program safety net activities; (3) develop MAST safety net reference documentation (policies, governance, coordination, implementation); (4) support Participatory Development Support Councils and other civil society organizations to develop capacity building plans and carry out social audit processes; (5) implement gender￾sensitive annual social audits with local stakeholders; and (6) provide training to community structures on social accountability focusing on the application of the Community Score Card, which is a tool to enhance community influence on the quality, effectiveness and accountability of the services offered by Kore Lavi at community level. Kore Lavi Title II Program Haiti – Midterm Evaluation 9 9 9 Evaluation Methodology The Kore Lavi mid-term evaluation was undertaken in November 2015 by a seven-person Absolute Options (AO) evaluation team. The evaluation team worked closely with Kore Lavi staff, and in particular the Knowledge and Learning Coordinator, to plan and implement the evaluation fieldwork and to ensure that field implementation followed the overall evaluation design. Over a two-week period, the evaluation team conducted fieldwork in 11 communes located in the five program departments of Central, Northwest, Southeast, Artibonite and La Gonave in addition to Port-au-Prince. The evaluation employed three data collection methodologies: • Document review: The evaluation team undertook a review of diverse program documents describing different aspects of program design, implementation, monitoring and evaluation, and results in addition to the findings from the second round of field monitoring visits undertaken by teams of Food Assisted Development and Humanitarian Support field monitors during July-October 2015 using the LQAS Analysis Information Entry & Retrieval System. • Key informant interviews: These are in-depth, semi-structured interviews of single individuals or small groups of individuals. The evaluation team conducted over 60 key informant interviews involving 74 persons drawn from the following stakeholder groups: CARE, ACF, WFP, World Vision, MAST, MSPP, MCFDF, CNSA, FAES, World Bank, Digicel, health centers and Lead Fathers. • Focus group discussions: These are in-depth, semi-structured moderated discussions small groups of program beneficiaries or other stakeholders. The evaluation team conducted 48 FGDs with 404 persons involving 260 females and 144 males drawn from the following stakeholder groups: SO2 beneficiaries, VSLA members, SO3 beneficiaries, fresh food and staple food vendors, CHAs, Nurse Supervisors, Lead Mothers, Lead Fathers, CADEPS, CADECs and non-beneficiaries. To analyze the qualitative data, the evaluation team created detailed summaries of all KIIs and FGDs drawing on their detailed interview notes. The evaluation team next conducted a thematic analysis of the summaries organizing them into recurring themes and sub-themes associated with each of the evaluation questions and research hypotheses. Owing to time and logistical constraints, the evaluation team was not able to cover all of the program communes in the five program departments. Thus many of the conclusions drawn during the MTE are not necessarily representative of the program as a whole. Notwithstanding, wherever possible the conclusions are drawn from findings that occurred with sufficient frequency across the different data collection methods to indicate that there is something there that provides useful insights into how the program is operating and what results it is producing. Primary Findings and Conclusions SO1: National systems for vulnerability targeting strengthened • At the time of the MTE, Kore Lavi had completed data collection in 16 communes demonstrating the ability to adapt and innovate to address the initial data quality problems that existed at the beginning of the program resulting from deficiencies in the third party vulnerability data collected under Kore Fanmi. Positive adaptations or innovations implemented by Kore Lavi since the beginning of the program related to vulnerability data collection include the establishment of reference desks, support Kore Lavi Title II Program Haiti – Midterm Evaluation 10 1010 for appeals committees, and the establishment of effective re-survey and integration protocols for those households erroneously excluded from the SO2 beneficiary list. • Quality problems with vulnerability data caused delays in the identification and selection of Kore Lavi SO2 beneficiaries, although Kore Lavi appears to have successfully resolved these data quality problem. • Concerns expressed by senior MAST officials that the delays in SO2 voucher implementation may have hurt the program’s credibility at MAST appear to have been overstated. Implementation of the food vouchers under SO2 was delayed only three or four months suggesting that MAST and other stakeholder expectations may have been too high at the outset of the program. A more appropriate conclusion may be that Kore Lavi could have more effectively communicated the cause and consequence of delays and more effectively managed stakeholder expectations of the program from the beginning. • Pairing up local civil society leaders with survey enumerators may increase access to vulnerable households, buy-in to the survey process, and increase the willingness of households to actively participate in surveys. • All proposed alternatives to the census survey approach – including kiosks, community based pre￾targeting and ‘cluster’ sampling – risk introducing significant inclusion and exclusion errors. Of these alternatives, cluster sampling offers a relatively cost-feasible approach that is less subject to bias and that appear less likely to introduce inclusion errors, although it does so at a possible cost of a higher exclusion errors. • Proxy means testing, combined with the HDVI algorithm and Weighted Sum of Deprivation scoring is effective, systematic, and transparent, and should be expanded by MAST to assess vulnerability in non-Kore Lavi communes. • The HDVI does not appear to have led to the erroneous exclusion of deprived households owing to the fact that target communes have been allocated sufficient numbers of food vouchers to cover a significant number of households in the “mid-deprived” range. • The SSN selection appeals process only captures exclusion errors and not inclusion errors, as it is not likely that better-off households will alert the program that they have been erroneously included in the SSN activities. As a result, SSN targeting may in fact have higher inclusion errors, and thus higher overall targeting error rates, than currently reported. • The pre-allocation of food vouchers per commune prior to vulnerability surveying and scoring produces inconsistent coverage and often does not match level of deprivation in some communes. SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased • In terms of its implementation on the ground, the food voucher scheme is operating well in identifying voucher recipients and enrolling them in the program, distributing food to beneficiaries via paper or electronic vouchers, recruiting and managing the network of collaborating vendors, enforcing policies governing the scheme and overseeing operations. • Food received by beneficiary households from both paper and electronic vouchers is inevitably shared with non-household members, including neighbors and even strangers. Such widespread Kore Lavi Title II Program Haiti – Midterm Evaluation 11 11 11 sharing is a function both of the pervasive poverty and food insecurity among households in program communities, made worse by ongoing drought conditions, and Haitian cultural norms. As a result, the food acquired from the vouchers falls short of meeting 25% of household food consumption needs as was intended and thus is unlikely to have any impact on household hunger, household dietary diversity or food consumption. • The sharing of food acquired from the paper and e-vouchers does not reflect a problem with the voucher mechanism per se, which, as indicated earlier, is working well, nor does it reflect a fundamental flaw in the voucher program design. Rather it is an unavoidable outcome given the circumstances. The sharing of food is inevitable, nor can Kore Lavi do much to actively discourage it. Notwithstanding, given the pervasiveness of the ‘problem,’ and assuming that Kore Lavi retains the objective to improve nutritional outcomes among SO2 beneficiaries, it does require some adaptation to the voucher scheme to address it. • The administration of the voucher scheme has encountered a number of ongoing ‘technical’ problems that include late payments and unreliable access to the TPago mobile network for the electronic vouchers and late or partial payment due to a failure to top off the account at the relevant bank for the paper vouchers. (Kore Lavi has anticipated this technical issue by installing a back-up plan to use paper vouchers for the purchase of staple foods.) Other frequently recurring problems include the presence of inactive vendors on the vendor list, improper and unhygienic food storage by program vendors and the failure by program vendors to wear the hats or aprons signifying their participation in the program. Notwithstanding, the technical problems noted during the MTE are, for the most part, fixable. Where such problems do exist, Kore Lavi’s network of agents (e.g., ASCPs) play a key role in monitoring the implementation of the voucher scheme and resolving issues as they arise, to the extent possible. • Program stakeholders interviewed did not observe a significant impact of the SO2 voucher activities on the local markets for staple and fresh foods, either in terms of food prices or food availability, nor did the team observe that participating vendors experienced any problems sourcing the fresh and staple foods sold through the voucher scheme. • The VSLA scheme has effectively provided a mechanism by which large numbers of vulnerable women and men living in program communities can save and access small loans at reasonable interest rates to invest in their businesses or children’s education or for other purposes. The limited overlap between VSLA members and other SO2 beneficiaries, however, mitigates against using the VSLA as a ‘generalized’ exit strategy for the program’s social safety net (SSN) beneficiaries. Nonetheless, to the extent Kore Lavi can increase VSLA membership among SSN beneficiaries, as is its intention, this does offer a potentially effective generalized exit strategy from ongoing social safety net support. • The MTE team recognizes that there will always be a portion of SSN beneficiaries who are not capable of transitioning into more sustainable livelihood activities and who will thus remain perpetually dependent on government support. Thus the objective is not to move all SSN beneficiaries to the VSLAs or to other forms of sustainable livelihood activities but rather as many as possible given existing program resources. • The VSLAs are popular with the VSLA members, most of who have never participated in any kind of savings and credit scheme before. Members are largely using the savings and loans to invest in their small-scale businesses as well as for other household investments/expenditures, especially children’s school fees. Kore Lavi Title II Program Haiti – Midterm Evaluation 12 1212 • The training received via the VSLA groups is likewise very popular with members and is generally perceived as highly valuable. The demand for additional training on other topics, but particularly those related to micro and small enterprise operation, is high among VSLA members. The VSLA approach provides a mechanism to integrate this training into the program’s SSN activities, should the program choose to exploit it. SO3: Maternal and Child Nutritional Status Improved • SO3 social behavioral change communications interventions are, on the whole, well designed and well implemented. Care Groups, moreover, appear to be an effective methodology for mobilizing women and communicating critical SBCC messages • Community health agents and Lead Mothers play a critical role in SO3 activities. While they are, for the most part, doing a good job and are satisfied with their roles, they also have a number of legitimate concerns related to the lack of monetary compensation and reimbursement for expenses incurred, particularly mobile phone and travel-related expenses, and, in the case of Lead Mothers, the lack of outward demonstrations of appreciation for the work they do. • Another area of concern is the widely disparate number of CHAs and Lead Mothers that, respectively, Nurse Supervisors and CHAs are responsible for supervising as part of Kore Lavi’s cascade training model. In some cases, the supervisory burden is excessive made even more so given the low pay and lack of expense reimbursement. • The program has done a good job identifying and reaching the targeted women and infants. The program has multiple focal points on the ground – including Lead Mothers, CHAs, and CADEPs – that constitute an effective information dissemination and community mobilization team. Nonetheless, there remain gaps in program outreach, particularly with regards to women and infants who live in more remote and/or hard to reach locations, as well as working women who are too busy to participate in the program. • Mothers participating in the Care Groups appear to be learning about better ways to feed and take care of their infants and are appreciative for the knowledge and encouragement they are receiving from the program. They express a high demand for additional learning on diverse topics, particularly family planning and learning marketable skills. • The program has done a good job encouraging women to seek medical consultants for themselves and their infants at program-affiliated health centers, where by most accounts they are receiving good attention. With that said, there are recurring shortage of functioning medical equipment, essential micronutrient supplements, and mother and child health monitoring supplies. Kore Lavi is aware of these problems and is working to address them. • The increased demand for health center services has led to an increased work burden on existing health center staff. While this does not yet appear to have reached a critical point, it may still do so in the future. • The distribution of food rations has encountered a number of ongoing and interrelated problems related to late deliveries, long wait times, shifting distribution sites, unsuitable and crowded distribution sites and long lag times to add beneficiaries to or drop them from the distribution list. These problems are suggestive of more systemic issues related to the SO3 beneficiary list. Kore Lavi Title II Program Haiti – Midterm Evaluation 13 1313 • The most significant problem with the food ration component, however, is that it is almost immediately shared with family members, neighbors and even strangers, such that it lasts no more than a few days. Almost certainly, as a result, the food ration is not serving its purpose to supplement the nutrition of pregnant or lactating mothers or children less than 24 months. Kore Lavi is aware of this problem and is considering approaches to deal with it, particularly the family ration, which was proposed at the beginning of the program but rejected for fear that it would create perverse incentives for women to have more children. • Kore Lavi is pursuing a three-pronged sustainability strategy for SO3 involving (1) replacing food rations with a coupon (similar to SO2), (2) meeting with MSPP to determine which SO3 activities can realistically be transitioned to MSPP at the end of the program and (3) transitioning program-affiliated CHAs back to the ministry at program end. Notwithstanding, MSPP currently holds dim views about the prospect for sustainably integrating SO3 activities within MAST, mostly due to budget shortfalls. SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved • Kore Lavi has actively and effectively coordinated program strategy and activities with MAST since project inception. • Perceptions of Kore Lavi efforts to increase governmental capacity to lead and manage safety net programming would be improved through more regular, strategic communication of MAST-Kore Lavi coordination and implementation of joint initiatives. • Kore Lavi both underestimated the capacity development needs of MAST at project inception and overestimated the commitment and capacity of MAST to make the necessary investment to procure and develop its staff resources as necessary. Contributing to existing gaps in the supply and demand for institutional support at MAST was the failure of a planned G2G institutional support mechanism to launch, which would have provided direct US government support to MAST for capacity development purposes and providing all necessary strategic level support, while Kore Lavi would cover the operational aspects focused on social safety nets. • Targeted capacity assessments at the center and departmental levels are needed to better focus capacity development trainings and to establish concrete training schedules. • The lack of financial support both from the Government of Haiti and from Kore Lavi for staff compensation and reimbursable expenses constrains the quality and intensity of MAST staff engagement in Kore Lavi activities. • Kore Lavi had made significant progress under SO4 since mid-2015. Improved engagement between the Kore Lavi Project Management Team and senior MAST officials, increased formal support from government officials, the creation of a Social Accountability Manager position, and the development of an MAST Institutional Plan suggest that Kore Lavi is taking required action to improve performance related to government capacity and institutionalization. • Initial community mobilization efforts by Kore Lavi were largely effective, as program staff were able to quickly identify existing CADEPs and facilitate the establishment of new organizations to participate in program implementation. • Kore Lavi has been effective in increasing CADEP and other CSO knowledge and understanding of program objectives and technical skills related to community outreach, program promotion, food Kore Lavi Title II Program Haiti – Midterm Evaluation 14 1414 voucher distribution, and social accountability. In particular, CADEPs and other CSOs appear to understand Kore Lavi-promoted social accountability tools well, including the purpose and use of Community Scorecards • More needs to be done to establish and reinforce six-month action plans at CADEPs and CSOs, while more training is needed at the same related to organizational governance, financial management, proposal writing, and long-term sustainability plans. • Kore Lavi is striking the right balance between incentivizing involvement with community service volunteerism. Kore Lavi staff is asking the right questions regarding CBO and CADEP motivation for program participation, namely whether participation is driven by financial motives or by a true commitment to community development and community service. • Many CADEPs believe that they are not respected as true partners owing to the program’s engagement style, which tends to take command and control as opposed to participatory forms in which expectations are clearly spelled out. • The engagement of CNSA to support the execution of vulnerability surveys was an excellent decision, especially given delays in survey completion from other partners. However, the contracting of CNSA staff to conduct surveys may have supplanted the broader objective of increasing the capacity of CNSA and other government agencies to respond to future food emergencies. Gender • Kore Lavi has taken a holistic approach to gender integration from design to implementation and has made a conscious and good faith effort to integrate gender considerations in each of the four program SOs. • Gender considerations were well integrated into the Kore Lavi program from design to implementation. The design and implementation have been well informed by gender needs and priorities of Haiti, particularly, those of the targeted beneficiary communities identified by the baseline survey and more particularly, findings of a gender analysis conducted by CARE as well as recommendations provided by MCFDF. • The integration of gender was achieved through two inter-related strategies: (1) gender equality and GBV training and (2) the integration of gender equality promoting and women’s empowerment activities in almost all components of the program. Both strategies appear to have been very effective. • The VSLAs are another effective mechanism to promote women’s economic and social empowerment. VSLA members uniformly perceived positive benefits from VSLA membership. One particular benefit that stood out was the opportunities it created for women to assume leadership positions. Notwithstanding, women are underrepresented in VSLA leadership, particularly in the higher-level leadership positions of President and Secretary. • Due in part to the program’s efforts, women are participating actively in CADEPs and have ascended to leadership positions on the CADEPs’ Board of Directors. • The program did a good job bringing the MCFDF into the program at an early enough stage to ensure its active involvement and buy-in, as seen by the latter’s involvement and final endorsement of the gender related recommendations, along with gender training resources, which were developed in close collaboration with the Kore Lavi team. Kore Lavi Title II Program Haiti – Midterm Evaluation 15 1515 • Kore Lavi’s engage with the MCFDF, however, has experienced some setbacks owing to erroneous perceptions and expectations among certain MCFDF staff, particularly around the issue of funding. • MCDFD has recently taken steps to reinvigorate the relationship by appointing a new focal person within the Ministry. • The absence of an MCFDC representative on the program Steering Committee has introduced certain inefficiencies in how the Steering Committee and program can anticipate or react to gender issues. Given the program’s obvious commitment to gender issues, the absence of a woman’s advocate on the Steering Committee stands out as an important omission. Monitoring, Evaluation, Accountability and Learning • Kore Lavi has prioritized information management and has demonstrated a clear institutional interest in improving knowledge and learning. Some of the approaches taken by the program in this area, however, appear to rely on compartmentalized and overly technical definitions of information leading to a heavy focus on external reporting and limiting the attention devoted to the internal use of data. • Data accuracy problems persist at an administrative level with the registration of SO3 beneficiaries. A more flexible and less reactive approach could result in a more engaged level of information sharing across the consortium. • The MEAL system is passive, not active, in seeking engagement with the program beyond collecting core IPTT indicators and is overly concerned with SO2 and SO3 suggesting in turn the importance of prioritizing internal communication and relationship-building efforts across the consortium members to explain and secure feedback on existing information and learning systems and processes. • Kore Lavi IPTT indicators are used overwhelmingly for external reporting and are not used for internal decision-making or planning purposes. The Kore Lavi Web Application, together with the resources added in response to data quality issues, offer the program a good opportunity to reallocate monitoring and evaluation (M&E) assets for internal learning purposes. • Other options to promote internal learning is to include internal (non reported) indicators to the IPTT or to conduct rapid assessment exercises, such as those currently planned by the SO3 team to monitor the program’s SO3 activities using either quantitative or qualitative methods – such as mini￾surveys, lot quality assurance surveys (LQAS) and FGDs. • The M&E system was prioritized in the first two years of the program, and this delayed the development of the knowledge and learning (K&L) system meaning in practice that information systems have been developed separately for each SO. Consequences of this ‘silo’ approach include: (1) different information systems and beneficiary lists maintained for each SO, (2) internal learning compartmentalized according to department and SO, (3) increased risk of inaccuracy absent standardized (or standardizable) information systems, and (4) increased difficulty in creating cross￾organizational learning platforms. • Kore Lavi appears to have taken positive steps toward promoting greater cross-organizational learning, including quarterly workshops for senior program, SO coordinators and field managers; monthly calls with field coordinators and field officers among the SO4 team; plans for technical teams to visit each of the program departments. These processes, if used to address measures of operational quality and efficiency, rather than simple output or scale of activity, should help the program to make good progress in the remainder of the current funding cycle. Kore Lavi Title II Program Haiti – Midterm Evaluation 16 1616 Recommendations 1. Engage local civil society leaders in the data collection process. Community leaders such as church members, teachers, members of community-based organizations can accompany external CNSA surveyors to improve responsiveness, facilitate identification of more remote households, and potentially to increase data accuracy. 2. Move forward with data collection pilots currently under consideration to test the efficacy of alternative data collection and household identification approaches. These include testing kiosk￾based self-reporting of households that are subsequently verified by Kore Lavi vulnerability surveys, and “pre-targeting” by community leaders and cluster sampling. The high cost of the census approach makes it likely that one of the above three (or other) approaches will become necessary. 3. Bring together key HDVI stakeholders to reach agreement on basic concepts related to definitions and methods in implementing the HDVI. Preferably, this effort should be led by MAST, with possible facilitation by Kore Lavi. 4. USAID and the Government of Haiti should explore options to expand the Kore Lavi targeting methodology to other regions of the country. 5. USAID should promote the utilization of similar Multidimensional Deprivation Indexes (ranking based on HDVI and Weighted Sum of Deprivation) in future food assistance programs. 6. A more robust assessment of targeting error is needed, including greater frequency of beneficiary verification reports. In particular, the lack of mechanisms to identify inclusion errors needs to be addressed. 7. Food voucher allocation should be flexible to better match need in specific communes. While pre￾allocation may have been necessary for budgeting and call-forward purposes, the lack of flexibility in the pre-allocation approach is creating unnecessary coverage imbalances. 8. Until the MAST staffing issue can be addressed, Kore Lavi should continue and perhaps expand the use of high-energy interns to facilitate the continued development and operations of the Kore Lavi information system. Alternatively, Kore Lavi should increase its advocacy efforts to facilitate increased funding for Kore Lavi-related positions. 9. Secure adequate resources to continue the engagement of the information system management consultant (Thomas Gabrielle) beyond September 2016 to support the full adoption, ownership and operation of the Kore Lavi information system. Funding can be facilitated by adjusting the Kore Lavi budget or by advocating for funding by MAST/GOH. 10. Develop and fund an alternate compensation scale for MAST information management employees, in full collaboration with MAST officials. The higher-level skill set required for information management, and importance of successfully institutionalization information management activities, justify the higher wage scale. Given potential restrictions to the direct funding of civil servants by USAID, Kore Lavi should consider alternative incentive mechanisms, such as facilitating the establishment of a social protection performance fund, that could be managed as a stand-alone public-private partnership utilizing GOH and USAID funding. Kore Lavi Title II Program Haiti – Midterm Evaluation 17 1717 SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased • Evaluate options for improving the effectiveness of the voucher activities: The widespread sharing of food procured through the voucher activities undermines the impact of this activity on beneficiary nutrition. These options may include limiting voucher validity to certain items that would be less likely to be shared, or limiting the use to specific items that could be stored and consumed over a longer period of time, such as beans or grains. • Conduct a technical audit of the voucher activities so as to determine whether technical problems identified are sporadic or systematic and take appropriate actions to fix them. • Increase, and if possible, accelerate, the integration of the VSLA scheme into SSN activities under SO2, particularly by increasing the absolute number and relative share of SO2 SSN beneficiaries in VSLAs. In making this recommendation, the MTE team acknowledges the circumstances that motivated Kore Lavi to begin its VSLA activities prior to launching its other SSN activities under SO2 and that Kore Lavi is aware of this issue and is addressing it. • Consider conducting field research to quantify the extent to which food rations are shared, with whom and how long they last. Although reports of food sharing were common during the MTE, robust quantitative data on this phenomenon remains lacking. A similar recommendation applies to sharing of food rations received by SO3 beneficiaries. SO3: Maternal and Child Nutritional Status Improved • Expand health messaging to include post-natal family planning options and advantages, especially to men. • Take advantage of Care Groups and the program’s existing training infrastructure to provide more training diverse topics, such as family planning, water and sanitation, child growth and development, literacy, math, income generation, business development, agriculture and animal husbandry. Alternatively, link Care Groups to VSLAs for training or to integrate Care Group members into VSLA membership. More generally, Kore Lavi should investigate this and other opportunities for increasing integration between the different program SOs. • Conduct a study of workload of CHAs and SO3 Nurse Supervisors. Reaffirm that the workloads of Nurse Supervisors and CHAs are logical and consistent with criteria such as geographic size, road conditions, and population density and ensure that work responsibilities are consistent with the job descriptions and expected workload. • Consider adding CHAs and Supervisors to reach outlying areas or potential beneficiaries now missed and providing them the means to access these areas. • Consider further compensation for CHAs and Lead Mothers, including, at a minimum, reimbursement for job-related transportation, mobile phone expenses and, for Lead Mothers particularly, outward gestures of gratitude for work they perform. To the extent that this recommendation is acted on, it should be done so in coordination with the MSPP so as to ensure that such efforts are continued after the program. • Follow-up on which areas are giving gifts, what the gifts are, and whether they are making a positive difference in the performance of Lead Mothers, as the success of this program hinges on the performance of the Lead Mothers. Kore Lavi Title II Program Haiti – Midterm Evaluation 18 1818 • Contact health centers to determine whether there is enough supplementary foods, such as Plumpy Sup, for all cases of MAM. It would seem an opportune time to evaluate the other special nutritional needs or antibiotic needs of MAM infants and whether the health centers have sufficient stock of these. • Consider implementing the family ration to supplement the beneficiary ration. It is clear that the current ration is not sufficient and thus will not move the needle in terms of Kore Lavi’s core objective to improve nutritional outcomes of SO3 beneficiaries. This seems to offer an excellent opportunity for a limited, and closely monitored, field test to determine its incremental nutritional impact and whether it creates perverse incentives to have more children. • Consider conducting field research to quantify the extent to which food rations are shared, with whom and how long they last. Although reports of food sharing were common during the MTE, robust quantitative data on this phenomenon remains lacking. A similar recommendation applies to sharing of food procured using paper or e-vouchers. SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved • Conduct detailed capacity gap assessments at both the center- and departmental-levels, and develop concrete capacity development plans in order to increase the frequency and quality of trainings and to appropriately set staff expectations of Kore Lavi capacity development activities. • Consider adding manager-level staff to facilitate institutionalization of other Kore Lavi program components beyond the addition of the SO4 Social Accountability Manager. • Consider creating a second Social Accountability Manager so as to cover the full complement of SO4 initiatives. One manager could be tasked with the capacity development of CADEPs and a second manager could oversee the establishment and operations of reference desks and other feedback mechanisms. • Consider a more open sharing of budget envelopes in order to manage MAST expectations and increase MAST confidence in Kore Lavi statements related to limited financial resources. • Work with USAID and the Government of Haiti to increase financial resources available for staff compensation and to increase the frequency and quality of MAST field visits. These resources need to be included in GOH and MAST budgets. • Assuming MAST retains the objective to transition Kore Lavi’s SSN programming under its own umbrella, it will be necessary to include in the GOH and MAST budget the financial resources to increase staff compensation (especially related to information management) and the frequency of field visits, while simultaneously improving its own operational efficiencies so that it can devote more resources to this task. • Other actions MAST might take to improve the effectiveness of its SSN activities are to provide some form of financial or non-financial incentives to CADEPs, such as adopting a fee-for-service model or to provide small incentive packages for high-performing CADEP members and/or CADEPs, such as note books, umbrellas, rain boots, etc. MAST’s ability to pay for these recommendations likewise depends on whether it can secure the requisite financing and/or improve its operational efficiencies to free up resources. Kore Lavi Title II Program Haiti – Midterm Evaluation 19 1919 • Reinforce the joint-development of six-month action plans for CADEPs and other CSOs with a greater focus on organizational development objectives and actions. • Expanded Community Scorecard implementation to include 360 assessments of all Kore Lavi stakeholders, including the CADEPs and local governments. • Implement frequent planning sessions with CADEPs to develop monthly work plans to increase CADEP ownership of activities and to set expectations appropriately. • Increase program focus and effort related to government capacity to respond to food emergencies. The program should revisit early warning and emergency response activities from the original program proposal. Gender • Increase the recruitment of Lead Fathers either in lieu of or to complement Lead Mothers. Lead Fathers may have a potential to become agents of change. • Consider including a select number of Lead Mothers and Lead Fathers from across the different target departments in the pool of gender champions proposed in the gender strategy, so that they can benefit from the in-depth gender training and serve as champions in their respective communities. • Scaling up the successful gender equality promoting initiatives, such as the theatre initiative developed by CADEPs in the Center Department, will need a participatory approach, starting with brainstorming sessions involving the initiators, KL staff, and CADEPs from other departments. The program should also explore ways of scaling up the use of songs as a tool for promoting gender equality. • Consider creating positions of gender focal points or gender champions on the ground in the different departments to monitor and follow-up on gender sensitization, including CADEP-delivered trainings, and reinforce the program’s context specific gender integration efforts across the different target communities. • Consider strengthening the gender team to mitigate the occasional voids created by the rapid turnover of the Kore Lavi Gender advisor position, by say, moving it from a one-person to a small gender team made up of an advisor and one or two assistants, budget permitting. • Consider including MCFDF on the Kore Lavi Steering Committee so as to facilitate their direct technical advice to the committee and speed up the interaction between the program and its other GOH partners. • Make gender training a priority for all program and local government partner staff not yet trained. • Consider expanding gender and GBV training and sensitization to law enforcement officers, schools and health centers in areas like the Center department where the incidence of rape and other aspects of GBV is reported to be much higher than elsewhere in the country. Monitoring, Evaluation, Accountability and Learning • The existing strategy for M&E/K&L development requires active implementation in order to ensure that processes across Kore Lavi are consistent and that all partners feel engaged in sharing and Kore Lavi Title II Program Haiti – Midterm Evaluation 20 2020 learning. A small cross-partner learning and knowledge group should be formed to (1) ensure the engagement of all partners at regional and national level in learning activities; (2) solicit feedback on M&E and K&L activities, including internal communications; (3) identify a core group of program-wide indicators of impact and operational efficiency in order to assess program efficiency, time/cost savings, and beneficiary service improvement; and (4) develop small-scale pilots of flexible, agile methods of collecting operational data under those indicators. • Acknowledge prime responsibility for beneficiary distribution list accuracy, relying on WFP only to confirm delivery lists within, say, 72 hours of field delivery using the field agents who are currently present at distribution to confirm beneficiary attendance and receipt. • The program-wide extension of the Center department tablet program should be prioritized and used as an opportunity to review M&E and K&L across all SOs using the forums established by the KL program management. • The SO3 beneficiary list is a fundamental data source, and the program should prioritize a complete check of SO3 beneficiary lists for accuracy, including duplications. If the current MIS can be developed to effectively check for duplicates, including name spelling variants and related errors, then this is preferable to a manual review. • Prioritize the transition to web-based MIS where Excel spreadsheets are currently used. Internal training should be given in order to explain the constraints of excel-based data storage and familiarize staff across the program with the strengths of basic MIS systems, particularly record management and basic data quality verification. • Consider creating a complaints system for SO3 activities similar to the one existing for SO2 activities. Such a system would have potentially important operational benefits, specifically in terms of developing opportunities for SO3 beneficiaries to provide information that could improve the operational efficiency of, say, registration and food kit distribution. • Prioritize developing the capacity of the KLWA to perform duplication checks. • Encourage SO teams to develop internal operational quality and efficiency indicators to measure achievement of key SO outcomes or impacts, in addition to those currently recorded in the operational KL operations dashboard. These indicators may be integrated into the IPTT system, or they may be tracked and reported internally, depending on donor requirements. The key consideration in developing such learning indicators, and the methods used to collect them, is their management and operational utility. It is also important, however, to make sure that a system for collecting these indicators is either in place or can be put in place without putting excessive strain on staff or implementing partner resources. • In addition to, or in lieu of, developing new, internal IPTT indicators, Kore Lavi is encouraged to conduct on-going rapid assessment monitoring activities. Such rapid assessments would use a mix of qualitative and quantitative assessment methods, including, for example, FGDs, mini-surveys (short surveys with relatively few questions and administered to a relatively small group of people) or lot quality assurance sampling methods. The MTE team understands that Kore Lavi’s existing resources may not allow it to undertake such rapid assessment activities. Thus, as is the case with adding IPTT indicators, undertaking any new data collection activities should be done only where existing resources allow it. Where they don’t allow it, Kore Lavi might consider the feasibility and desirability of adding or re-allocating resources for this purpose. Kore Lavi Title II Program Haiti – Midterm Evaluation 21 21 21 • Continue with and accelerate efforts to create opportunities for cross organizational learning using, for example, workshops, cross-team field visits, newsletters, brown bag events and so forth. The important point here, however, is that whichever methods are used must be systematized with strong and visible management support. This implies in turn the need to develop a formal knowledge and learning strategy that incorporates all aspects of the program’s MEAL system. Kore Lavi Title II Program Haiti – Midterm Evaluation 22 2222 2. Introduction 2.1 Kore Lavi Background Haiti was ranked 158th out of 179 countries on the United Nations 2011 Human Development Index. The country has very high poverty levels that have dissipated slightly over recent years with stagnation in development due in part to deficiencies across several socio-economic indicators, such as levels of undernourishment, stunting and underweight children—coupled with poor performance in gross domestic product and high levels of overall poverty. The Global Hunger Index (GHI) ranks Haiti 77 out of 79 countries, and “is one of three countries in the world with ‘extremely alarming’ GHI scores” for 2012. Moreover, the high incidence of climatic shocks, environmental hazards and the government’s limited capacity to mitigate their protracted consequences via effective responses further impedes the country’s ability to improve socio-economic development. As a result, a majority of the Haitian population still faces significant challenges in all four pillars food security: availability, access, utilization and stability. A consortium of partners led by CARE, which includes Action Contre la Faim (ACF), the World Food Programme (WFP) and World Vision have been awarded funding to provide developmental food assistance to deprived and food insecure households in Haiti that have been severely affected by the prolonged drought, now into its third year with the Northwest and Southeast departments being among the hardest hit in the country, and to support the National Program of Food Security and Nutrition. Kore Lavi is a four￾year, $80 million USAID Title II Development Food Assistance Program funded through the Office of Food for Peace (FFP), with an implementating time horizon from August 2013 to September 2017. Kore Lavi provides support to the Government of Haiti (GOH), and in particular the Ministry of Social Affairs and Labour (Ministère des Affaires Sociales et du Travail, or MAST) on social protection, hunger and malnutrition prevention through the implementation of safety net programs aimed at building demand for, and improving access to, locally produced nutritious foods by vulnerable households. Over four years, the program aims at providing 18,500 households with food vouchers and 205,000 households with maternal and child health and nutrition interventions with the intent to replicate and scale-up at the national level from an initial five departments. The overall program Goal is to strengthen the national social safety net and improve the food security and nutritional status of deprived populations in Haiti. To achieve this Goal, core programming under Kore Lavi is endeavoring to: (1) address the urgent needs of extremely poor households in the five departments of Northwest, Southeast, Center, Artibonite and West/La Gonave, which are affected by multiple hazards and corresponding higher levels of deprivation and (2) build Haitian institutional capacity to replicate and scale-up, at the national level, the knowledge, behavioral changes and models of social assistance prompted by the program for preventing food insecurity and malnutrition. 2.2 Kore Lavi Results Framework Kore Lavi’s Theory of Change (TOC) holds that positive and lasting transformation must happen within interrelated domains: (1) where the effective social safety net (SSN) programming and complementary services reach the most vulnerable populations and protect their access to food while building self reliance; (2) that achieve breadth and depth in behavior and social change needed to tackle under-nutrition among vulnerable women and children; and (3) that institutionalize accountability, transparency and quality of delivery for mutually reinforcing social protection programs under the leadership of MAST—where a Kore Lavi Title II Program Haiti – Midterm Evaluation 23 2323 targeted effort to improve programming at the national level for those in extreme poverty in Haiti can also help them to escape chronic food and nutritional insecurity and experience a dignified recovery. Towards these ends, Kore Lavi has four Strategic Objectives (SOs) supported by 10 Intermediate Results (IRs): • SO1: National systems for vulnerability targeting strengthened • IR1.1: MAST-led equitable vulnerability targeting methodology developed, tested and implemented • SO2: Access of extremely vulnerable households to local and nutritious foods increased • IR 2.1: MAST-led, gender-responsive food voucher-based safety net model developed and implemented • IR 2.2: Inclusion of local foods in the voucher-based safety net increased • IR 2.3: Access to complementary services for safety net households increased (including particularly access to village savings and loan groups, or VSLAs) • SO3: Maternal and child nutritional status improved • IR3.1: Household practice of appropriate nutrition behaviors to prevent malnutrition increased • IR 3.2: Capacity of community-based entities to promote appropriate nutrition practices to prevent malnutrition improved • IR3.3: Capacity of health facilities to deliver appropriate nutritional services strengthened To achieve the third strategic objective, Kore Lavi implements a Social Behavior Change Communication (SBCC) strategy in line with the preventative approach focusing on the window of opportunity in the first 1,000 days of life. • SO4: Haitian institutions’ capacity to effectively lead and manage safety net programming improved • IR4.1: Institutional capacity of various levels of government to lead, coordinate and implement safety net programs reinforced • IR4.2: Capacity of civil society to monitor and support safety net programs reinforced • IR4.3: Government capacity to respond to food emergencies expanded A short description of each of the four SOs is provided below. SO1 National systems for vulnerability targeting strengthened: SO1 covers three main outputs, which are closely related to each other and which are supposed to be achieved through coordination and joint efforts of Kore Lavi and MAST. First, develop, test and adjust a national Haitian Deprivation and Vulnerability Index (HDVI), which would allow the program to measure vulnerability in a multidimensional way. The index relies on specific indicators (proxy means) attached to each household and is calculated through a customized algorithm. Kore Lavi has worked with various program partners – such as the Kore Lavi Title II Program Haiti – Midterm Evaluation 24 2424 National Coordination for Food Security (Coordination Nationale de la Sécurité Alimentaire, or CNSA), Economic and Social Assistance Fund (Fonds d’Assistance Economique et Sociale, or FAES), United Nations Development Programme (UNDP), World Vision and MAST – to develop a standard vulnerability targeting questionnaire and deprivation algorithm, which can be used more widely than just within the context of Kore Lavi. Second, define a specific data collection methodology (socio-economic survey, community pre-targeting, etc.) implemented by other organizations on a door-to-door basis, so as to allow Kore Lavi to benefit from already-existing data. Third, develop and implement a database application hosted by MAST, which is a tool to store, analyze and manage collected data. The main function of this system is to answer key questions – such as how many beneficiaries (vulnerable households) are in the country or what is the geographic distribution and core characteristics of these households – so as to allow decision makers to develop an appropriate frame of social assistance for future safety net interventions. After developing the questionnaire and corresponding deprivation algorithm at the early stage of the program, Kore Lavi initiated data collection in the program communes with the support of its partner organizations. Once collected, the data is cleaned and the deprivation score is calculated through the algorithm and applied to each household, while the data is stored in a consolidated database at MAST. This data can be used to create a single beneficiary register of vulnerable populations in Haiti to be managed by MAST or other partners, although this is not necessarily an end program objective. SO2 Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased: Under SO2, Kore Lavi had developed voucher transfer modalities prioritizing the use of local foods so as to deliver food vouchers to a target of 18,150 extremely vulnerable households. To promote good nutritional prac￾tices through a balanced diet by fresh food, and at the same time to make a direct contribution to local production, Kore Lavi uses two types of vouchers: paper vouchers valued at 400HTG (9USD), which are allocated for the purchase of local fresh foods (fruits, vegetables, fresh meat, fish, eggs, etc.), and electronic vouchers valued at 700HTG (16USD) allocated for the purchase of local staple foods (local rice, maize, sorghum, beans, flour, and oil) for a total value of 1,100HTG (25USD)1 . The fresh food is purchased in the community markets (which function every week), and the staple food is purchased in small community shops in both cases from vendors selected by the program. To select the participating fresh food and staple vendors, Kore Lavi applies the following selection criteria: (1) capacity to deliver a variety of fresh foods and a commitment to charge stable and fair price, (2) agreement to sell only local products approved by the program, and (3) agreement to be registered on a regular basis in the market in which they do business. Under SO2, Kore Lavi is also promoting and facilitating participation in village savings and loan associations (VSLAs). A VSLA is a group of people who save together and take small loans from those savings. The activities of the group run in cycles of one year, after which the accumulated savings and the loan profits are distributed back to members. Kore Lavi is facilitating the creation of VSLAs with a target of 18,000 VSLA members in order to help integrate as many program safety net beneficiaries as possible into VSLA groups and to support the gradual graduation from the safety net of less vulnerable beneficiaries. 1 At the time of program inception, the prevailing exchange rate of 44 Haitian gourdes for 1 US dollar. Since then, the US dollar value of the Haitian Gourde has varied considerably and currently stands at 59 gourdes per dollar making the current value of the paper voucher equal to 6.78USD and the current value of the electronic voucher equal to 11.86USD for a combined value of 18.64USD. Kore Lavi Title II Program Haiti – Midterm Evaluation 25 2525 target of 18,000 VSLA members in order to help integrate as many program safety net beneficiaries as possible into VSLA groups and to support the gradual graduation from the safety net of less vulnerable beneficiaries. SO3 Maternal and Child Nutritional Status Improved: Under SO3, Kore Lavi implements a social behavior change communications (SBCC) strategy in line with the preventive approach focusing on the window of opportunity in the first 1,000 days of life (Preventing Malnutrition in Children Under 2, or PM2A). The program provides supplementary conditional rations to pregnant and lactating women and children aged 6-23 months with a goal to reach 76,567 women and 96,597 children during the life of the program. The conditions that beneficiaries must meet in order to receive food rations are: (1) take part in primary health care activities (such as antenatal and post-natal consultation, growth monitoring and immunization) and (2) participate in program SBCC interventions. The SBCC strategy is mainly implemented through a Care Group approach. A Care Group is a group of 10 women named “Lead Mothers” (or in limited cases “Lead Fathers”)2 who are identified by the community and come together for mutual support and learning and who are regularly trained by the local community health agents (CHAs)3 . Each Lead Mother has responsibility for approximately 10 households and conducts visits to each household on monthly bases to share and promote key infant and young children feeding (IYCF2) practices and other appropriate health and nutrition information. The recommended ratio is one CHA for no more than two Care Groups, for a maximum of 20 Lead Mothers. In order to reinforce the knowledge of the community members regarding proper health and nutrition behavior and practices, Kore Lavi collaborates with the existing network of CHAs belonging to the Ministry of Health (Ministère de la Santé Publique et de la Population, or MSPP), while sensitizing through various community public awareness raising campaigns – such as media campaigns, nutrition fairs, special days and events, etc. – key community structures, civil society organizations and traditional health workers. CHAs also play a central role in the program’s moderate and acute malnutrition intervention. A network of nurse supervisors is responsible for training and overseeing the work of CHAs. CHAs in turn are responsible for approximately10 to 20 Lead Mothers. Moreover, to reinforce health and nutrition services provided by the health facilities, the program conducts trainings for health professionals and primary health care workers. One of the main activities currently supported at the health facility level is the management of moderate and acute malnutrition (MAM). Specifically, the program supports and facilitates active and passive screening of children through its network of CHAs so as to identify new cases of MAM and refer them to healthcare facilities to receive nutritional treatment. SO4 Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved: Under SO4, Kore Lavi seeks to: (1) implement a joint USAID/MAST capacity building and resource mobilization plan; (2) work with MAST national and department-level staff on coordinating program safety net activities; (3) develop MAST safety net reference documentation (policies, governance, coordination, implementation); (4) support Participatory Development Support Councils (Conseil d’Appui au Développements Participatif, or CADEPs) and other civil society organizations to develop capacity 2 CHAs are also known as Agentes de Santé Communautaire Polyvalent (ASCPs). 3 Given the relatively small number of Lead Fathers currently working for Kore Lavi compared to Lead Mothers, the report uses the generic term ‘Lead Mothers’ from here on, except for those instances in which it refers specifically to Lead Fathers. Kore Lavi Title II Program Haiti – Midterm Evaluation 26 2626 building plans and carry out social audit processes; (5) implement gender-sensitive annual social audits with local stakeholders; and (6) provide training to community structures (local authorities, civil society organizations) on social accountability focusing on the application of the Community Score Card (CSC), which is a tool to enhance community influence on the quality, effectiveness and accountability of the services offered by Kore Lavi at community level. CADEPs are commune-based organizations comprised of local government officials and civil society and private sector representatives for the primary purpose of planning and coordinating community development. This component also involves support to CNSA and the Haitian Institute of Statistics and Information (Institut Haïtien de Statistique et d’Informatique, or IHSI) to deliver timely and standardized survey assessments/early warning reports and supports the Haitian Civil Protection Agency (Direction Protection Civile, or DPC) to enable safety net expansion during emergencies. In this regard, Kore Lavi provides technical support to CNSA to update livelihood profiles (last versions dated back to 2005), and has actively contributed to CNSA’s involvement in the implementation of vulnerability targeting in the program target areas. Gender: Activities for promoting gender equality are integrated into each component of the program to enhance the quality, and not just the quantity, of female participation at different levels, from participation in strategic planning at the national level to more effective participation in decision-making at the household level. In addition, the program has been providing support to the Haitian Ministry of Women (Ministère à la Condition Féminine et aux Droits de la Femme, or MCFDF) to promote and support gender integration in food security and social assistance programming in Haiti. The allocation of responsibility for the above four SOs and 10 IRs among the four Kore Lavi consortium members is shown in Figure 1. Figure 1: Allocation of Responsibility for SOs and IRS among Kore Lavi Consortium Partners Program Component CARE ACF WFP World Vision SO1 N/A N/A Technical lead & implementation N/A SO2 Technical lead: E-vouchers Implementation: Southeast & Center Technical guidance: Fresh market vouchers Implementation: Northwest & Artibonite N/A Implementation: West/La Gonave SO3 Implementation: Southeast & Center Technical lead & implementation: Northwest & Artibonite Commodity management N/A SO4/IR4.1 N/A N/A Technical lead & implementation N/A SO4/IR4.2 Implementation: Southeast & Center Implementation: Northwest & Artibonite Technical lead N/A SO4/IR4.3 N/A N/A Technical lead & implementation N/A Gender Technical lead & implementation: Southeast & Center Implementation: Northwest & Artibonite N/A N/A Kore Lavi Title II Program Haiti – Midterm Evaluation 27 2727 3. Evaluation Purpose This reports presents the findings, conclusions and recommendations related to the Kore Lavi mid-term evaluation (MTE). The Kore Lavi MTE is a “process evaluation.” According to Food for Peace monitoring and evaluation (M&E) and reporting guidelines, a process evaluation focuses principally on “implementation methods and outputs” and on exploring “how well the project is following implementation plans and meeting targets, the acceptability of the methods employed to the beneficiary population, and signs of changes that beneficiaries associate with project interventions.”4 Consistent with this purpose, the Kore Lavi MTE had the following objectives: 1. To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. 2. To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with the GOH, particularly MAST, MSPP and MCFDF. 3. To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). 4. To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. 5. To recommend the future orientation (e.g., adjustments to the TOC, program design, resource allocation, or implementation) and emphasis of the program during its remaining time. 4 “USAID’s Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Assistance Projects: Draft for Public Comment,” September 2015, p. 12. Kore Lavi Title II Program Haiti – Midterm Evaluation 28 2828 4. Methodology 4.1 MTE Evaluation Team The Kore Lavi mid-term evaluation was undertaken in November 2015 by a seven-person Absolute Options (AO) evaluation team consisting of a Team Leader; a Vulnerability Targeting and Governance Specialist; a Commodity Management, VSLA and Market Assessment Specialist; a Maternal and Child Health & Nutrition and Social Safety Net Specialist; a Gender and Social Inclusion Specialist; a Monitoring, Evaluation, Accountability and Learning (MEAL) Specialist, and a remote-based Logistics Manager (see Annex 2: Evaluation Team Profile and Biographies). The evaluation team worked closely with Kore Lavi staff, and in particular the Knowledge and Learning Coordinator, to plan and implement the evaluation fieldwork and to ensure that field implementation followed the overall evaluation design. 4.2 Data Collection Methods and Data Sources The evaluation employed three data collection methodologies: document review, key informant interviews (KIIS), and focus group discussions (FGDs). Document Review: As part of this midterm evaluation, the evaluation team undertook a review of diverse program documents describing different aspects of program design, implementation, monitoring and evaluation, and results. In addition to the various internal program documents, a key source of information for this MTE report was the findings from the second round of field monitoring visits undertaken by teams of Food Assisted Development and Humanitarian Support (FDHS) field monitors during July-October 2015 using the LQAS Analysis Information Entry & Retrieval System (LAYERS). Over a four-month period, field monitors visited 23 communes across all five of the program departments and wrote up their findings in a series of Field Trip Reports and in a Summary of Findings document dated November 23, 2015. The document review served two primary purposes. One purpose was to inform the evaluation design and implementation. Another purpose was to integrate the information from these documents with the information collected during the fieldwork to generate a set of evaluation findings and conclusions. Key Informant Interviews: Key informant interviews are one-on-one or small group in-depth and semi￾structured interviews conducted with key program stakeholders. KIIs were conducted, for the most part, in-person using a combination of pre-prepared interview questions and interviewer follow-up probing questions and took an average of around 60 minutes to complete. Over the course of the midterm evaluation, the evaluation team conducted over 60 key informant interviews involving 74 persons drawn from a diverse set of program stakeholders (see Figure 2) (Annex 3: List of Stakeholders Interviewed and Annexes 6-11: Discussion Guides). Kore Lavi Title II Program Haiti – Midterm Evaluation 29 2929 Figure 2: Number of KIIs by Stakeholder Group Focus Group Discussions: Focus group discussions are facilitated discussions conducted with groups of 6-10 program beneficiaries and implementing partners. FGDs were conducted using a combination of pre-prepared interview questions and facilitator follow-up probing questions and took on average around 60-90 minutes to complete. Over the course of the MTE, the evaluation team conducted 48 FGDs with 404 persons involving 260 females and 144 males (see Figure 3). Stakeholder Group Total Interviewed Port-au￾Prince Northwest Center Artibonite Southeast La Gonave CARE 16 13 0 1 0 2 0 ACF 15 4 2 0 9 0 0 WFP 10 10 0 0 0 0 0 World Vision 6 2 0 0 0 0 4 MAST 10 7 1 0 1 1 0 MSPP 3 1 1 0 0 1 0 MCFDF 2 2 0 0 0 0 0 CNSA 1 1 0 0 0 0 0 CADEC 1 0 1 0 0 0 0 FAES 2 2 0 0 0 0 0 World Bank 1 1 0 0 0 0 0 Digicel 2 2 0 0 0 0 0 Port Internationale 1 1 0 0 0 0 0 Health Center Staff 3 0 1 0 1 1 0 Lead Fathers 1 0 1 0 0 0 0 Total 74 46 7 1 11 5 4 Kore Lavi Title II Program Haiti – Midterm Evaluation 30 3030 Figure 3: Breakdown of Focus Group Discussions Held during Kore Lavi MTE 4.3 Evaluation Fieldwork The evaluation fieldwork commenced with a kick-off meeting at the Kore Lavi headquarters in Port-au￾Prince on November 2, 2015 attended by evaluation team members and members of the Kore Lavi senior management teams. Over the next two days, evaluation team remained in Port-au-Prince to conduct a series of initial interviews and departed for the field on November 5 beginning in the Center department and continuing on in succession to Northwest, Artibonite, Southeast and West/La Gonave. The team spent 1-3 days in each department conducting KIIs and FGDs in a total of 11 communes (Figure 4). While the Evaluation Team was in the field, the Team Leader and MEAL Specialist remained in Port￾au-Prince to conduct interviews with Port-au-Prince based stakeholders. (See Figure 5 for a graphical presentation of the evaluation team’s scheme of maneuver during the MTE fieldwork.) The Kore Lavi Knowledge and Learning Coordinator and MTE Team Leader developed the specific fieldwork schedule on a rolling basis while the evaluation team was in the field so as to incorporate developments in the field into the planning. In the days prior the field team’s entry into a particular department, members of the Kore Lavi M&E team would travel to the department to make the appropriate arrangements with the intended KII and FGD participants. Figure 4: Breakdown of Focus Group Discussions Held during Kore Lavi MTE Stakeholder Group Center Artibonite Northwest Southeast La Gonave Total VSLA members 5 1 3 0 0 9 SO2 beneficiaries 1 1 2 0 1 5 SO3 beneficiaries 3 1 1 2 0 7 Vendors 4 1 2 0 1 8 Lead mothers 2 1 0 0 0 3 Lead fathers 0 0 0 1 0 1 CHAs 1 1 1 3 0 6 CADEPs/CADECs 3 1 1 1 0 6 Non-beneficiaries 1 0 0 1 0 2 VSLA/SO3 0 0 1 0 0 1 VSLA/SO2/SO3 1 0 0 0 0 1 Total 21 7 11 8 2 49 Center Artibonite Northwest Southeast La Gonave Thomassique Labranle Marre-Rouge La Valle Anse-à-Galets Cerca Carvajal Gonaive Bombardopolis Thomonde Anse-Rouge Plaine de L’arbre Kore Lavi Title II Program Haiti – Midterm Evaluation 31 31 31 The communes visited in each department in addition to the number of KIIs and FGDs conducted in each commune and department and with each stakeholder group was a function of the following factors: • The objective to achieve a reasonable representation of program locations and stakeholder groups, to the extent possible. • Logistical issues that emerged in the field, which required revisions in the planned fieldwork, including the substitution of one stakeholder group for another, the cancelation of planned FGDs or the decision to forego certain communes where work had been planned. During the fieldwork, for example, a vehicle transporting the field team broke down and required repair. In other cases, the poor road conditions resulted in much longer travel times than anticipated. In both of these cases, the resulting delays required unplanned changes to the fieldwork schedule. • Logistical difficulties involved in traveling to a representative sample of communes where specific stakeholder groups were located due to poor roads and the long travel time required to reach them. As a practical matter, fieldwork could only be conducted in communes that were reasonably accessible to the field team during the few days the team had scheduled to work in the relevant department. • The availability of key informants to participate on the days the relevant members of evaluation team were in a particular location. • The desire to follow-up on findings from previous FGDs with specific stakeholder groups. Or, alternatively, the field team felt that it had sufficient information from one stakeholder group such that further FGDs with that group were not warranted. • The failure of certain FGD groups to form as planned due to one reason or another. • Illness among the field team members, which truncated the team’s work in the Southeast department. • The absence of SO2 or SO3 activities in the communes that were accessible by road to the field team during the time allotted (see above), or because, in the case of La Gonave, SO3 activities were not operating in the department. The Evaluation Team returned to Port-au-Prince on November 18 to consolidate its interview/discussion notes, begin writing-up their evaluation findings and conclusions and prepare the evaluation out-brief. The out-brief was held at Kore Lavi headquarters on November 19 and was attended by key members of the Kore Lavi and Kore Lavi consortium member management teams, USAID and other Port-au-Prince based program stakeholders. Kore Lavi Title II Program Haiti – Midterm Evaluation 32 3232 Figure 5: Scheme of Maneuver during MTE Fieldwork 4.4 Data Analysis To analyze the qualitative data, the evaluation team created detailed summaries of all KIIs and FGDs drawing on their detailed interview notes. The evaluation team next conducted a thematic analysis of the summaries organizing them into recurring themes and sub-themes associated with each of the evaluation questions and research hypotheses. 4.5 Known Limitations to the Evaluation Methodology The Kore Lavi mid-term evaluation relied almost exclusively on qualitative data collection methods. By its nature, qualitative data lacks representativeness. In this case, and owing to time and logistical constraints, the evaluation team was not able to cover all of the program communes in the five program departments, for reasons explained below. While the evaluation team was conscious of avoiding tarmac bias5 in the selection of study communes, the poor road conditions made it impossible to visit certain communes within the timeframe allotted for the fieldwork under AO’s contract with CARE. In those communes visited by the evaluation team, moreover, it was not possible to meet with all primary stakeholders in 5 Tarmac bias occurs when study areas are selected because they are easily accessible. However, these areas are likely to be systematically different from more inaccessible areas. 20'N 1.9N ctoe De't Dame-Marie Les Irais 28'N - • - • - Frontiere internationale Limite de departement Coors d'eau Route prindpale Route seroodalre 1-] Agglomerations Sommets * Capita le nationale + Chef-lies de departement o Autres vines Projection UTM - WG584 Datum Tiburon Abric viere de La Cabboene • r / • -0 ARO Proll Chardonniere Port-a-Pirnent 30 SO oremie ase MASS/T. DE LA Poe Maeaya „ ---- 2 347 m ,—T lihne Camp-Peni Coteaux Roth-a-Bateau -ss To Port-Salut 24'W 6„slef't Cap Saint-Nicolas Cap a Foux Mer des Caralbes Pointe ouest lies Cayemites ------ Presqu'lle des Baraderes Petit Trou de Mopes Pastel Petite limere Anse-a-Vea de Nip St-Jean du Sod radere NIP de RI Les Cayes lie a Vache _ Golfe de la Gonave - Bale de Grande Henne Pointe Pointe-a-Raquett-ebs,,,,..„ Cana/ ele ia Gonave Rang de 22'W Canal de la Tortoe Pointe Jean-Rabel St. Louis du Nord - - stro, , I a etit Me•e,/ aissa -P toottO 4,, S CENTRE Thomon Montrodis La Chap 0 ahaie uvaliervile True Cal C. bellan Sou e Ch se d'Ha RAN E-ANSI e Gavel le St.- a ouge Te Jame Carreto /Owe mance PORT￾AU￾PRINCE Kensco uanarn ches 1 res Iladere has eB,..eap, re ae Baptiste o. -Saxenette ei'd/ • oCornilion o￾Bale de Is Tortue Pointe de la Grande-Pierre Bait de Grand-Pierre ce, Grande-Salin Pointe Ouest MOgasin _ , t Cotes-de-Per Cap Raymond Pointe Faniasque Be de la Tortue Bale de Port-au-Prince Stier Le Borgne laacierndee Mer des Corathes OCEAN ATLANTIQUE Cap-Haitien Bale de Mancevitie artiorfj _ orino Caracol Phaeton ou-du-Nor Mon Cris per and Pansien t Jimani /4/Tot), ka 5-Verre iNE DE LA S orne Sods-Pin dk.Pro Belle ,A72-23S-ns- - - - - - - 68- - 2 -"o 0 0,7 SUD-EST Bee_ arigot crAnse ot - M 3 .03 • Lag° Enriquiao • Grand-Gosler Anse-a-Pitres 7 2' W Boucan Carre Bassin-Bleu Initial planning with KL managment / KIIs in PAP Base of operations in deparment AO Team conducting research in PAP All SOs of the KL program Includes only SO1, Food Vouchers, SO4 Includes only VSLA, SO3 and SO4 Includes only VSLA, Prevention and SO4 Kore Lavi DFAP Interventions AO End of mission debrief with KL management Action Points Kore Lavi Title II Program Haiti – Midterm Evaluation 33 3333 each case, again owing to the contractually imposed time constraints put on the fieldwork. Thus while the evaluation team worked with Kore Lavi to identify a set of ‘representative’ communes and stakeholders in each department, these do not give a fully representative picture of the program. The tight time schedule also limited the ability of the evaluation team to directly observe certain elements of Kore Lavi activities. In particular, evaluators were not able to observe Lead Mothers interacting with pregnant and lactating women. Given the limitations, certain findings related to program implementation and results, particularly those based on respondent anecdotes, may be true in certain cases, but are not necessarily representative of the program as a whole. Throughout this report, we have endeavored to report findings that have occurred with sufficient frequency across the different data collection methods to indicate that there is something there that provides useful insights into how the program is operating and what results it is producing. In those cases where we are drawing on more limited anecdotes, or where we are less comfortable asserting that something more generalized in occurring, we make this explicit. Kore Lavi Title II Program Haiti – Midterm Evaluation 34 3434 5. Evaluation Findings This section presents the main evaluation findings for each of the four program SOs in addition to the primary findings related to gender and MEAL. 5.1 SO1: National Systems for Vulnerability Targeting Strengthened 5.1.1 Background As of November 2015, 130,937 household socio-economic surveys in 16 communes have been completed and stored in the Kore Lavi database, 129,921 of which have been ranked using two composite indicators: (1) Weighted Sum of Deprivation Scores and (2) the Haitian Deprivation and Vulnerability Index (HDVI)6 . Both are composite measures of multiple dimensions of poverty built on the analysis of proxy means indicators, which are summarized in Figure 6 below7 : Figure 6: Proxy Means Indicators 6 Although the HDVI constitutes only of the two scores calculated by the algorithm, the algorithm itself and the resulting ranking are also called “HDVI score” and HDVI ranking. 7 From Haiti’s targeting instrument for social programs: The Haiti Deprivation and Vulnerability Indicator - HDVI; Executive Summary, March 2014, page 3. The wording of variables is exactly as listed in this document. Dimensions (7) Variables (21) Demographic vulnerability 1. Household demographic composition 2. Presence of children under 5 years of age 3. Elderly at home Health 4. Presence of disabled or permanently injured 5. Chronically ill at home Education 6. Illiteracy 7. Absence of at least basic school 8. School non-attendance 9. School lag Labor conditions 10. Inactivity 11. Unemployed 12. Child labour Food security 13. Hunger 14. Absence of food 15. Restricted consumption of food Resources at home 16. Absence of remittances 17. Deprived material of floors, ceilings, and walls Living conditions 18. Overcrowding 19. Deprived lighting access 20. Deprived access to water 21. Deprived sanitation conditions Kore Lavi Title II Program Haiti – Midterm Evaluation 35 3535 Of the 129,921 households (representing an estimated 594,450 persons), only 1.4% of households were classified as most deprived, 11.8% as mid-deprived, 36.4% as least-deprived, and 50.4% as non-deprived, with significant geographic variability as summarized in Figure 7. Figure 7: Household Classifications into Deprivation Categories The resulting classification of surveyed households into the four deprivation categories and relative ranking of each household drives the selection of beneficiaries for inclusion into the food voucher and savings group components of the Kore Lavi program (SO2). As of November 2015, 15,086 households have been selected for inclusion into the program using this targeting methodology. 5.1.2 Timing of Key Milestones The timeline of key milestones in the implementation of SO1 are summarized below. Timeline of Key Milestones Pre-contract SO1 Technical Working Group established October 2013 Kore Lavi contract signed October 2013 First meeting of SO1 Technical Group October 2013 to May 2014 Program inception period April 2014 HDVI algorithm ready for use April to June 2014 Vulnerability surveys begins with data collection of PMT and HDVI calculation. Lists of vulnerable households ready for four communes Kore Lavi Title II Program Haiti – Midterm Evaluation 36 3636 The methodology is applied in two more communes and the results transferred to SO2 staff First 612 households receive food vouchers Appeal/complaint reference desks established in two communes MAST point person for the development of the vulnerability database/information system appointed Full time Information Management Officer recruited July to September 2014 Targeting methodology implemented in six communes 2,847 extremely vulnerable (most-deprived) households identified in the six communes Six reference desks in two communes produced 1,067 appeals, of which 257 were confirmed as extremely vulnerable households Two Beneficiary Verification Reports completed SO1 Strategic Plan developed and accepted by MAST and Kore Lavi partners Equipment for database (information system) establishment purchased One MAST staff member trained in database management Four Vulnerability Appeals Committees established in four communes with 50% representation by women October to December 2014 Targeting surveys completed in three more communes Assessments of data management systems of two MAST-affiliated institutions initiated: IBESR (Institut de Bien-Etre Social et de Recherche) and CAS (Caisse d’Assistance Sociale). Detailed manual for the HDVI algorithm completed Second MAST staff member trained in database management January to March 2015 Targeting surveys in seven communes completed Over 70,000 vulnerable households in Kore Lavi/MAST database Inclusion/exclusion error calculated to be 5.7% Two additional MAST staff members trained in database management April to June 2015 Quarterly report completed July to September 2015 Annual report (with 4Q FY2015 data) completed Kore Lavi Title II Program Haiti – Midterm Evaluation 37 3737 5.1.3 Evaluation Approach In accordance with mid-term evaluation SOW and preliminary conversations with the SO1 Technical Lead, AO consultants organized its assessment of SO1 into three primary components: 1. Data Collection 2. Targeting Efficacy 3. Information System Management Additionally, given IR1.1: MAST-led equitable vulnerability targeting methodology developed, tested and implemented, findings, conclusions and recommendations related to the institutionalization of the Kore Lavi targeting methodology is also covered in this section. Note that for the purposes of this evaluation, the term “targeting methodology” is used to describe the entire targeting process from proxy means indicator definition, data collection, identification and selection of vulnerable households, and information system (including data base) management. The term “targeting efficacy” is focused on the effectiveness of the HDVI algorithm and Weighted Sum of Deprivation scoring techniques to accurately identify vulnerable households for participation in the program. The findings reported here are based upon the review of Kore Lavi Quarterly and Annual Reports, key informant interviews with Kore Lavi and MAST staff, and key informant interviews with Kore Lavi field staff and FAES staff. 5.1.4 Development of Proxy Means Indicators and Data Collection Surveys Proxy means indicators and data collection surveys were developed in a timely manner despite coordination difficulties with external partners. The Kore Lavi SO1 team was able to define 21 indicators of deprivation across seven dimensions (as listed above in the background section) within the eight-month program inception window, including the design of survey instruments for field data collectors. As described below, the HDVI was also ready for use by April 2014, despite reports from MAST staff and Kore Lavi field staff that the slow development of the HDVI caused subsequent delays in program implementation (especially SO2 activities). As Kore Lavi was tasked with reconciling its data collection (and targeting) methodologies with existing Kore Fanmi/FAES (World Bank) approaches, initial discussions with these external partners regarding a revised set of vulnerability indicators was arduous.8 Despite this, Kore Lavi staff effectively engaged Kore Fanmi/FAES and was able to secure concurrence with its proxy means indicators within the program inception period. 5.1.5 Initial Data Collection and Use Challenges Data collection quality problems and delays caused significant delays in the roll-out of SO2 activities. Kore Lavi staff was directed to utilize existing Kore Fanmi data wherever possible. This reliance on third parties for household vulnerability data resulted in several challenges. Data quality problems and inadequate coverage (less than 100% of community households) required the re-surveying of several areas. Additionally, 8 The Community Social Worker Initiative (Kore FANMI) was a pilot program of the Haitian Government implemented by the Economic and Social Assistance Fund (FAES). It was established with World Bank financing in partnership with a number of United Nations agencies such as UNICEF. As part of a national social protection strategy, Kore Fanmi sought to harmonize and improve the provision of basic services to poor and vulnerable families, such as access to education, vaccines, and latrines, with a view to moving away from a national fragmented and unequal system to one of systematic coverage focusing on the rights and needs of families. Kore Lavi Title II Program Haiti – Midterm Evaluation 38 3838 FAES, which implements Kore Fanmi, was slow to complete Kore Lavi-related vulnerability surveys, which resulted in delayed program implementation and in one case the exclusion of a commune altogether (Cerca La Source). Several stakeholders reported quality problems with Kore Fanmi data, including senior MAST officials, central-level MAST staff, Kore Lavi field managers, and at least one CADEP (Thomonde). Delays in the collection of comprehensive and accurate household vulnerability data caused knock-on delays to the identification of households qualifying for participation in Kore Lavi, and subsequently caused delays in initiation of the food voucher component of the program. More than one MAST official mentioned that the delay in the implementation of the program, especially food vouchers, decreased enthusiasm, credibility and support for the program throughout several levels of MAST (i.e., Director General, Senior Staff, Field Staff) in the first year of the program implementation. Kore Lavi staff has addressed many of the initial data collection challenges by the mid-point of the program. Kore Lavi took timely corrective action to address both data quality issues and incomplete survey coverage by contracting Coordination National de Sécurité Alimentaire staff to conduct/complement household vulnerability surveys. Although Kore Lavi only completed data collection in three communes by the October to December 2014 reporting period, the program has now completed data collection in eight more communes (November 2015) bringing the total of covered communes to sixteen (therefore also including the surveys done by Kore Fanmi). Kore Lavi also demonstrated agility in responding to data quality problems and reports of initial exclusion errors by establishing appeals/complaints reference desks through which households who felt they were incorrectly excluded could apply for re-evaluation. 5.1.6 Skewed Deprivation Reporting Local stakeholders (CADEPs, local government officials) report that the use of surveyors from outside target communities skews deprivation reporting. Although the contracting of CNSA staff to conduct household vulnerability surveys accelerated the pace and quality of data collection, several CADEPs reported that the use of external surveyors (typically from Port Au Prince) skewed survey results. They state that households are reluctant to share information with outsiders, tending to overstate their socio￾economic status. However, the use of local residents (and officials) also has the potential to introduce bias as local surveyors may favor friends, relatives and political allies. This risk was deemed more significant by Kore Lavi staff, which contributed to the decision to contract CNSA for data collection. 5.1.7 Issues with Census Approach to Field Surveys The census approach to field surveys is costly and may not be sustainable as Kore Lavi activities transfer to MAST. Current Kore Lavi data collection methodology mandates that 100% of households in target communes be surveyed for beneficiary selection. At $7 to $8 per household survey, MAST will not likely be able to afford 100% survey coverage in all of its target communes. Despite the cost of the census approach, MAST officials have stated unequivocally that they prefer surveying all households in target areas. They would prefer to focus on fewer new program areas with 100% survey coverage over adding a larger number of areas that are assessed via alternative means. All of the CADEPs interviewed expressed suspicion of pre-targeting by community leaders, citing the potential of favoritism for friends, family and political allies. CADEPs were more open to the idea of kiosk-based self-reporting, but they also expressed concerns about the most vulnerable households having access to information related to the location and hours of operation of kiosks. In addition to a census, pre-targeting or kiosk-based self-reporting system, another option Kore Lavi is considering is a cluster sampling approach in which an iterative and participatory process is used to Kore Lavi Title II Program Haiti – Midterm Evaluation 39 3939 identify geographic clusters with high concentrations of vulnerable/deprived households, while excluding geographic clusters with lower concentrations of vulnerable/deprived households, and then sampling a census of households within those clusters determined to have higher concentrations of deprivation (in consultation with community leaders). This approach has the advantage of being significantly less expensive than a census approach, while avoiding problems of selection bias (and thus relatively high rate of inclusion error), which are likely to emerge using a pre-targeting or kiosk-based approach. It has the disadvantage of increasing the incidence of exclusion error relative to the census approach. 5.1.8 Convergence on Definitions and Methods Development of the HDVI as a common tool to be used not only by the GOH but also other development organizations (e.g., Kore Lavi, World Bank, UNDP, FAES) is hindered by a lack of consensus around common definitions and methods. In terms of definitions, key informants across the above talked alternatively about measuring vulnerability, deprivation or poverty, and there was even no consistency among Kore Lavi staff in defining just what the HDVI is measuring. While it may appear semantic wrangling, poverty, deprivation and vulnerability are distinct concepts that mean different things to different people. That no agreement has been reached on this most basic of concepts portends further problems with harmonization down the line. There is also no agreement on how the HDVI is to be administered. Although there is agreement on the basic structure of the HDVI, different organizations (at least 5) frame questions and code responses differently thus greatly complicating the merging of survey data sets at the level of MAST, which is necessary to use the HDVI as a policymaking tool. Key informants explain this outcome in terms of a lack of leadership by MAST, or some other entity, to bring the different actors together to reach agreement on definitions, methods and the architecture for harmonizing the implementing the HDVI across its users. Key informants explain that this outcome is mostly due to the fragmentation of programs, including a lack of coordination in relation to social protection. This renders agreement on definitions, methods and the architecture for harmonizing the HDVI across stakeholders especially challenging. 5.1.9 Stakeholder Buy-In Kore Lavi targeting approaches have significant buy-in from key social protection stakeholders. Despite initial resistance to the revision, refinement, and improvement of pre-Kore Lavi vulnerability scoring mechanisms (especially from Kore Fanmi), key social protection stakeholders express confidence and satisfaction with Kore Lavi targeting approaches, including MAST staff at center- and departmental-levels, Kore Fanmi, UNICEF, external actors (CRS and ICF), field program managers and field program officers. 5.1.10 Conservative Definition of Deprivation Several stakeholders believe that the thresholds used in the HDVI algorithm results in an overly conservative definition of the “most-deprived” cohort, resulting in the exclusion of eligible vulnerable households. Both members of Kore Lavi senior management and field operatives (social protection agents and CADEPs) report that the HDVI algorithm scores deprivation conservatively meaning fewer households are classified as extremely vulnerable or “most-deprived.” Kore Lavi field counterparts (CADEPs) and lower-level MAST staff report that the conservative (or strict) classification leads to the exclusion of eligible households (see conclusions below). Kore Lavi Title II Program Haiti – Midterm Evaluation 40 4040 5.1.11 Targeting Efficacy Kore Lavi proxy means testing and methodology of ranking, which is the result of the calculation of Weighted Sum of Deprivation and the HDVI, provide a systematic, transparent, and adaptable identification and ranking of vulnerable households. The development of the 21-variable proxy means test, together with the HDVI algorithm represents an important innovation in USG food assistance programming. Based upon evaluator experience, most USG food assistance programs (in particular Development Food Assistance Programs) conduct a one-time targeting exercise based upon aggregate data with little or no systematic ranking of vulnerable households within targeting area. Kore Lavi targeting offers a transparent ranking of vulnerable households for identification and selection. Several stakeholders confirmed this observation, including senior MAST staff at the headquarters and departmental levels, program field managers and program officers. Weighting variables and the elaboration of the HDVI algorithm allows for future adjustment to improve targeting. ACF, for example, is using Kore Lavi targeting approaches not only to improve identification and selection of households for program participation, but also to revise their target beneficiary list on a monthly basis by incorporating post-voucher distribution verification and by processing reference desk claims in a timely manner. The Kore Lavi targeting approach is effectively targeting the most vulnerable households in target communities. Kore Lavi staff report current targeting errors of approximately 4%, well below the program target of 30%.9 This targeting error is calculated by dividing the number of verified appeal claims by the total number of vulnerable households initially identified. Therefore, targeting error calculations are not based on the percentage of “most-deprived” households receiving benefits. The additional food security criteria may also unfairly disqualify some households from inclusion. Indeed, the proxies used to measure the household status in terms of food insecurity are not sufficiently reliable. They more represent a “snapshot” of the current situation of the household rather that measuring the longer-term status of chronic food insecurity. As of FY16Q1, all “most deprived” households have been included as beneficiaries to address this potential source of exclusion error. 5.1.12 Variability in Inclusion and Exclusion Errors The initial allocation of food voucher resources per commune before vulnerability surveying and ranking is driving variability in inclusion/exclusion errors, especially for the “mid-deprived” cohort. The number of food vouchers allocated per commune was determined prior to vulnerability surveys and deprivation rankings. Program staff report, moreover, that this allocation cannot be modified.10 The pre-determined number of vouchers is allocated according to the ordinal ranking produced by the HDVI and Weighted Sum of Deprivation scoring until all vouchers are distributed. Although it appears as though enough vouchers were allocated in all communes to cover the “most-deprived” cohort, the pre-determined voucher allocation does lead to a wide variance in coverage of the mid-deprived and least-deprived cohorts across communes. As summarized in Figure 8 below, less than 2 of 10 deprived (most, mid, and least) households are receiving food vouchers in Center communes, while 8 of 10 deprived (most, mid, and least) are receiving vouchers in La Gonâve communes. 9 The January to March 2015 Quarterly Report reported a similar error rate of 5.7%. 10 Kore Lavi staff report that as of February 2016, a revised allocation of vouchers is being considered to better match allocations with need in target communes. Kore Lavi Title II Program Haiti – Midterm Evaluation 41 41 41 5.1.13 Lack of Initial Strategic Plan The lack of strategic plans for database and information system management at the inception of the project slowed the establishment of a quality information system. Per key informant interviews and review of Kore Lavi Quarterly Reports, the development and adoption of an SO1 Strategic Plan did not happen until the 4th Quarter of FY2014. The lack of a strategic plan early slowed the establishment of the MAST beneficiary database in addition to causing initial database development efforts to be incremental and unfocused. Figure 8: Variability of Household Inclusion Rate in Kore Lavi Communes 5.1.14 Quality Problems with Kore Fanmi Data Quality problems with the Kore Fanmi database content caused delays in implementing the Kore Lavi/MAST beneficiary database. Instead of duplicating previous efforts to establish a social protection beneficiary database, and given the large number of Kore Lavi beneficiaries that were originally identified through Kore Fanmi surveys, Kore Lavi staff correctly started with the existing Kore Fanmi dataset. Kore Lavi staff were under the impression that the Kore Fanmi database was “ready-to-use” and that Kore Lavi dataset needs could easily be bolted-on to the existing database. However, several data quality issues were identified including inconsistent spelling of locations, beneficiary names, etc., that renders cross-sectional reporting nearly impossible. Inconsistent data entry (spelling) required a re-coding of previous entries so that standard and custom reports could efficiently be created from consistent data panels. Additionally, key informant interviews indicate that the quality of data entry (beyond inconsistent variable and/or category naming) also hindered the development and implementation of the MAST beneficiary database. Kore Lavi addressed these database quality issues by re-designing the Kore Fanmi dataset and through a significant re-coding effort. By using open source software for the MAST beneficiary database and reporting modules, Kore Lavi staff created cost efficiencies (avoiding of licensing fees). The eventual adoption of an Information System Strategic Plan addressed database quality issues and helped MAST develop a coherent vision for information system management. Although some MAST staff Kore Lavi Title II Program Haiti – Midterm Evaluation 42 4242 are clearly not familiar with this strategy, senior MAST officials clearly understand and have ownership of the Strategic Plan. 5.1.15 Human Resource Deficiencies at MAST MAST has not appointed sufficient staff for the operations and management of the Kore Lavi/MAST database and information system. The single greatest challenge to the effective and sustainable establishment of the Kore Lavi information system at MAST has been the inability of MAST to appoint sufficient staff to SO1 activities. According to quarterly reports, 15 MAST staff members were to be trained in database management, but only three had been trained by March 2015. Additionally, the program anticipated the appointment of ten staff to SO1 activities, but no additional staff were allocated after the initial appointment of a single point person and the recruitment of one full-time information management officer. Kore Lavi invested heavily in the capacity development of this staff member, who eventually after a little less than one year. The lack of personnel depth in information system management hindered the continuity of MAST information system management, as no other staff had been made available for information system management training. As of November 2015, the Information Management Officer has not been replaced by MAST, while at present three WFP interns and a WFP consultant are implementing information management duties. 5.1.16 Inadequate Compensation of MAST Staff Inadequate compensation of MAST staff involved in information system operations and management is contributing to recruitment and retention difficulties of MAST SO1 staff. Both Kore Lavi staff and senior MAST officials point to inadequate compensation as the primary difficulty in recruiting/appointing MAST staff to SO1/information management positions. The skill set required (even basic skills) command $2,000 per month from NGOs versus $400 per month most MAST staff members make. Additionally, MAST staff see appoint to Kore Lavi-related activities as uncompensated addition to their workload (this is true for all SOs). 5.1.17 Institutionalization of Information System Institutionalization of the Kore Lavi information system has not been effective and MAST is not currently capable of managing the Kore Lavi information system. Despite clear support for the Kore Lavi/MAST information system from the highest-level MAST officials, the inability of MAST to appoint staff members to information management positions has resulted in the weak institutionalization of SO1 activities. Kore Lavi staff correctly point out that without dedicated MAST staff, they are not able to develop institutional capacities related to database and information system management. However, MAST officials also reasonably state that without compensation levels that match the required skill set and information management job responsibilities, they are unable to recruit MAST staff for these positions. This issue must be addressed if MAST is to successfully assume information management responsibilities at the end of the Kore Lavi program. Kore Lavi Title II Program Haiti – Midterm Evaluation 43 4343 5.2 SO2: Access of extremely vulnerable households to local and nutritious foods increased 5.2.1 SO2 Activities and Outreach According to project staff, SO2 activities reach approximately 17,700 beneficiaries in 16 communes across the five program departments (out of the 24 Communes covered by the project). These activities include the implementation of the SSN intervention, which consists of both electronic vouchers (e-vouchers) and paper vouchers, and VSLAs implemented in part, though not exclusively, as an “exit strategy” for SSN participants. A school feeding activity is planned, but is still in the pilot phase and was therefore not evaluated during this MTE. While the target departments and communes for Kore Lavi activities were selected using CNSA data, SO1 (targeting) activities determined actual beneficiary lists, which (presumably) consist of food insecure households in the bottom ten percent of the population in terms of deprivation. 5.2.2 Voucher Activities The voucher activities began in May of 2014, initially in the Center and Southeast Departments. Similar SSN activities have been implemented previously in several areas, including in the Northwest and Center by CARE and Oxfam and in La Gonave by World Vision. Currently, the SO2 voucher activities are the only Kore Lavi activities implemented on La Gonave with beneficiaries being selected from a previous similar project. The food vendors through whom the vouchers can be redeemed for food were chosen by a “census” of households within SO2 communities according to the following criteria: (1) distance to beneficiaries, (2) hygienic conditions and (3) availability of food at their shops. Each vendor is required to have a business license, a mobile phone, and a bank account. Some vendors declined to enroll in the project because they did not want to obtain the business license, as this also serves to register them for the payment of business taxes. Likewise, a smaller number of vendors have resisted opening a bank account. Kore Lavi works with the local mobile phone service provider Digicel to implement the e-vouchers, the values of which are valid for one month. Digicel credits the electronic platform (TPAGO) upon request by CARE. Once the platform is credited, CARE refills the vouchers. After the vouchers have been refilled, a communication is issued to the SO2 Field Officers who pass the communication down to the beneficiaries and vendors. The beneficiaries then go to the vendors to redeem their vouchers. Digicel makes the payment to the vendors account twice a month. Kore Lavi uses CHAs to provide day-to-day technical support, help mitigate problems and enforce guidelines around the voucher activities. As a prerequisite for the job, the CHAs are required to be familiar with the zone in which they work and have the ability to read and write with strong community leadership skills. They receive two days of training before beginning during which they learn how to monitor markets and where to go if they need further support. They also receive monthly “refresher training.” As a condition for voucher use, all foods purchased must be locally produced. In FGDs, both fresh food and staple food vendors in addition to voucher recipients noted that this requirement is strictly enforced. Kore Lavi Title II Program Haiti – Midterm Evaluation 44 4444 “Sometimes it is difficult to purchase oil (with the vouchers), because the vendors only have imported oil available, and we can only buy locally produced items with the vouchers.” (FGD with voucher recipients) Although some vendors note that this requirement could at times be inconvenient, this was more the exception than the rule. “It would be good to allow people to purchase some imported foods, like oil, because sometimes local foods are not available.” (FGD with fresh food and staple food vendors) 5.2.3 Impact of Voucher Activities The baseline study for the Title II program in Haiti employed three measurements of food security; the household hunger scale (HHS)11, the household dietary diversity score (HDDS), and the food consumption score (FCS). While it was beyond the scope of this evaluation to update values for these indicators, a number of observations and comments recorded during FGDs with voucher recipients provide insights into some possible impacts of the voucher activities. The vouchers were intended to provide 25% of household food consumption needs for a household of five people. However, a key finding is that the food purchased with the vouchers is shared with a much wider group of people outside the household than was originally intended to consume the food thereby undermining the intended impact of these activities. This seems to be a result of much wider sharing than was anticipated during project design. During FGDs with voucher recipients, numerous beneficiaries commented that, in addition to family members, the food is shared with many people outside of the immediate family whom the household would not usually feed, including neighbors and in-laws. It may be possible to slightly reduce sharing by urging beneficiaries to refer people to the project through the kiosks. However, this was not explicitly discussed with project staff. “The food is consumed in a few days because when we have food we need to share it with our neighbors.” (FGD with voucher recipients) “We have to share (the food we buy) with in-laws and neighbors.” (FGD with voucher recipients) “The voucher value is so small that no change is possible.” (FGD with fresh food and staple food vendors) The number of people that voucher recipients reported sharing the purchased food with ranged between 6-28 and averaged approximately 10 people, or twice the number for which the vouchers were designed (see Figure 9). The number of people was generally higher in the Artibonite than in other departments. In addition, due to the sharing of food purchased with the vouchers, the food lasts for a period far shorter than an entire month. In FGDs, voucher recipients reported that the food lasted between 2-15 days per month, with the average reported being approximately five days per month. In no case did interviewees report the food lasting longer than 15 days. 11 “Baseline Study of the Title II Development Food Assistance Program in Haiti” ICF International, May 8, 2015 Kore Lavi Title II Program Haiti – Midterm Evaluation 45 4545 Figure 9: Reported Number of People Sharing Food and Days over which Food is Consumed The HHS measures the extent of household food deprivation over a period of 30 days. The survey conducted in areas representative of the beneficiary communes targeted by Kore Lavi for the baseline study indicated that 57.5% of these households suffer from moderate hunger and 13.5% suffer from severe hunger.12 Responses during FGDs suggest that it is unlikely that the voucher activities have significantly reduced household hunger, due to the high number of people with whom beneficiaries share the food purchased with vouchers, which in turns means that it is consumed very quickly. The HDDS provides an indication of food groups consumed in the last 24 hours, and is an indicator of food access. The survey conducted in the representative areas targeted by Kore Lavi for the baseline study indicates an HDDS of 6.2. 13 This suggests that households in the Kore Lavi area typically consume six of 12 basic food groups. From these 12 food groups, the survey suggests that the most accessed and consumed foods were oils/fats (99%), cereals (96%), pulses/legume/nuts (82%) and miscellaneous food items such, as coffee, tea and spices. The least accessed and consumed foods were eggs (13%), milk and milk products (15%) meat, poultry, organ meat (19%) and fish and seafood (24%). Similarly, responses during FGDs suggest that the impact of the voucher activities on household dietary diversity is also likely to be low owing to the high incidence of sharing and the rapid consumption of the food. Based on information provided in the KIIs and FGDs, the prevalence of food sharing observed by the evaluation team appears to be a function of at least two primary factors. The first factor is the pervasive poverty and food insecurity among households located in program departments and communes, which has been further aggravated by the ongoing drought in Haiti. The second factor is cultural norms that exist in Haiti that create a set of strong social expectations that those households who have share with those who do not. Notwithstanding, FGD respondents further noted that the vouchers were, if only to a limited extent, increasing household dietary diversity because the vouchers did allow them to purchase and consume foods that would normally be beyond their budget limitations, particularly meat. Although beef still seems to remain beyond the budget of the many beneficiaries, a number of beneficiaries do use their vouchers to purchase consume goat and to a lesser extent chicken, fish and eggs. “We had not consumed meat or eggs until we started getting the vouchers.” (FGD with voucher recipients) “The most commonly purchased meats are goat and fish, but beef is very expensive, more than people can afford.” (FGD with fresh food and staple food vendors) “They (the voucher recipients) are only able to eat meat because of the vouchers. Before the vouchers they didn’t ever eat these things.” (FGD with fresh food and staple food vendors) 12 Ibid. 13 Ibid Region Number of People Number of Days High (e.g., Artibonite) 9-28 2-10 Low (e.g., Center) 6-16 3-15 Kore Lavi Title II Program Haiti – Midterm Evaluation 46 4646 Overall, it appears that voucher recipients participating in FGDs did use the vouchers to purchase and increase consumption of both dry and fresh foods, although with respect to the latter, price emerged as a significant constraint as to what recipients were able to purchase, in particular beef. FCS is an indicator of dietary quality and frequency of consumption and is calculated by measuring frequency of consumption (number of days) of eight food groups consumed by a household over seven days. The survey conducted in the representative areas targeted by Kore Lavi for the baseline study indicate that 69% of all households have an adequate level of food consumption, while 22% score at the borderline level and 9% score at the poor level.14 Although the vouchers facilitate the increased purchase of food, as with the previous indicators, the widespread sharing of the food purchased with the voucher seems to undermine the potential for changes in the frequency and consumption of food. As stated above, beneficiaries report that the food is consumed within 2-15 days, with the average reported as approximately five days. Voucher recipients and vendors universally stated that the food purchased with the vouchers was insufficient, and that there were many people that required this form of assistance but were not receiving it. “Because of the dry season, there are many people that need this activity.” (FGD with fresh food and staple food vendors) “There are many elderly people that are vulnerable and should be added to the activity.” (FGD with fresh food and staple food vendors) 5.2.4 Technical and Other Issues Related to Voucher Activities “Technical issues” refer to challenges or problems with the implementation of the voucher activities, including the timing of distribution, the ability of beneficiaries to redeem the vouchers on time, and security of the voucher purchasing power. Certain technical issues arose with sufficient frequency across the KIIs, FGDs, and FDHS monitoring activities. However, owing to the limitations of the evaluation methodology described above, it was not possible to determine whether these are systemic issues or whether they are more isolated instances. It is also worth noting, with regards to each of these issues, there were also a number of key informants/ beneficiaries who observed that things were working well and that they were satisfied with the course of program implementation. Nonetheless, these issues are discussed below because they were raised enough times during the different assessment activities to suggest that a problem may well exist, whether it be systemic or limited. Late payments for e-vouchers: Staple food vendors in approximately 13 communes covering all five program departments noted problems with late payments. For example, during a FDHS monitoring visit to Bombardopolis at the end of July 2015, e-voucher vendors complained that they had yet to be paid for purchases made during May and June 2015. “Sometimes (funding for redeeming sales through) the e-vouchers are two months late. However, when there are 14 Ibid Kore Lavi Title II Program Haiti – Midterm Evaluation 47 4747 problems with the vouchers, the project staff are available and try to be helpful.” (FGD with fresh food and staple food vendors) “Sometimes the coupons are not topped up on time. Paper coupons are better because this late topping up does not occur.” (FGD with fresh food and staple food vendors) Late payment for paper vouchers: Fresh food vendors in approximately six communes across four departments (Northwest, Southeast, Center and Artibonite) complained that at times the relevant financial institution did not have the funds on hand to redeem the paper vouchers, which should ideally occur on market days. Partial payments for paper vouchers: Fresh food vendors in approximately four communes across two departments (Southeast and Center) reported to the FFP project monitors that the relevant financial institutions did not always have sufficient cash on hand to redeem the full value of coupons. Poor mobile network access: Staple food vendors in approximately four communes across four departments (Northwest, Southeast, Center and Artibonite) noted that they have had difficulties accessing Digicel’s mobile network to record transactions, either due to a poor network signal or to a temporary disruption of service. Inactive vendors on active vendor list: There were at least four instances observed in Center and Southeast departments in which vendors who had left the program were still on the active vendor list. It must be noted that where issues, such as the above, do arise, the voucher recipients and vendors alike consistently acknowledge that the program representatives (e.g., CHAs) are visible and active in the markets, particularly on market days, and who generally provide good service where it comes to addressing the above and other issues as they arise. For its part, Digicel attributes the payment problems largely to vendors providing incorrect bank information, and it denies that network problems related to the network’s strength and reliability are a problem in the areas where program vendors operate. The recurrence of problems related to payment and network access, however, indicate that these problems occur frequently enough to indicate that, in the first case, go beyond incorrect bank information and, in the second case, network access is not always as reliable as Digicel suggests. In addition to the above technical issues, the MTE and monitoring activities identified a number of other issues that emerged with some consistency commune to commune, which include the following. Failure to wear/display program identifiers: Program vendors in at least seven communes in four departments (Northwest, Southeast, Center and La Gonave) were not wearing the hat or apron that program vendors are asked to wear so as to advertise their participation in the program. A common reason given for not wearing the hat and/or apron was that they did not fit properly. (The FDHS field monitoring findings do not mention whether another reason vendors were not wearing their hat or apron because they found them unfashionable, which was cited by Lead Mothers as a reason for not wearing, or liking to wear, their program shirt/uniform.) Improper food storage: FDHS field monitors observed at least seven instances observed in Northwest and La Gonave in which the program vendors stored their food in un-hygienic conditions and/or did not take measures to protect the food from pests (e.g., storing food off the ground). Kore Lavi Title II Program Haiti – Midterm Evaluation 48 4848 5.2.5 Market Impact Purchase of food aid commodities through vouchers is a form of local procurement. This modality of food aid procurement may have a positive impact on market development in terms of stimulating additional income for producers and vendors through increased demand. In addition, local procurement may provide (1) access to food at a lower cost, (2) access to food that better matches local taste preferences or is more culturally appropriate or (3) food in a more timely manner (important in emergency situations) However, in certain situations where supply is relatively inelastic, increased demand may result in price inflation, which can in turn negatively affect consumers across markets. This is a particular risk in situations where markets are poorly integrated, and hence potential supply is easily overestimated. In addition, in cases where redemption is difficult, vendors may discount the value of vouchers. A full market impact assessment was beyond the scope of this MTE. However, based on observations, as well as comments from voucher recipients and fresh food staple food vendors, the market impact of the voucher activities does not appear to have been, on the whole, significant, although there was anecdotal evidence that it possibly did create some small benefits. The vast majority of voucher recipients and food vendors interviewed during FGDs stated that neither previous in-kind food aid activities nor the voucher activities have had an impact on prices due to the small quantities of food purchased and the significant supply available to them. Previous food aid activities in the region distributed wheat flour. Currently, rice and beans are distributed in schools by WFP, but school feeding has not negatively impacted local business, as the students that receive it would not normally purchase it. Also, people prefer local beans, so they continue to purchase these even when beans from overseas are distributed as food aid. However, a few project staff, as well as vendors in Center Department, disputed this and stated that they had increased their incomes due to the voucher activities and also knew of producers that had increased their production in response to the increased demand resulting from the these activities. “Everybody is a beneficiary in this type of activity. Farmers can produce and sell more, and we (merchants) can sell more.” (FGD with fresh food and staple food vendors) “The market development aspect has been “successful, because rice producers claim to have increased sales due to Kore Lavi.” (KII with ACF Assistant Field Manager) Likewise, the requirement that only local foods were purchased with the vouchers resulted in some beneficiaries switching to local rice, which is the preferred variety according to local taste. “Many people buy local rice for the first time (as a result of the voucher activities).” (FGD with fresh food and staple food vendors) The voucher purchases were not cited as a cause of any unusual price fluctuation. In areas where there were price fluctuations, these were most likely attributable to usual price seasonality. “The voucher scheme does not impact prices because the quantities are too small.” (FGD with fresh food and staple food vendors) Kore Lavi Title II Program Haiti – Midterm Evaluation 49 4949 “The activity has had a very small impact on sales, but not much because the people consume the food within two days.” (FGD with fresh food and staple food vendors) In general, local markets in the Kore Lavi program areas are well integrated with merchants purchasing their inventories in regional markets, especially in Artibonite. In some cases, beneficiaries have had to switch vendors, because their old vendors did not participate in the voucher scheme, but this does not appear to have been a problem, as program vendors are typically located in the same market nearby. Interestingly, prices for food are not higher in La Gonave than on the mainland, suggesting sufficient market integration despite the geographic isolation of the island. Beneficiaries interviewed on La Gonave agreed that voucher activities have had no impact on prices due to the relatively insignificant quantities purchased. “Most of the staple foods come from markets in the Artibonite (rice, beans, maize, wheat). Sometimes maize meal and beans are locally available.” (FGD with fresh food and staple food vendors) Likewise, interviews suggest that there are no problems associated with shortages due to the voucher purchases. In FGDs with voucher recipients and vendors, interviewees stated that the foods purchased with the vouchers are always available. The only cases in which certain foods were unavailable could be attributed to normal supply seasonality, a factor especially true for beans. “There have been some items that are not available due to seasonality (all items are seasonal to a degree), but the quantities bought through the vouchers are too small to cause shortages.” (FGD with fresh food and staple food vendors) “(Prices for) black beans are also seasonal. The low price is HTG315 during the production season, but this increases to HTG400 following the harvest (current price).” (FGD with fresh food and staple food vendors) Likewise, discounting of the vouchers was not widely reported by the beneficiaries. The reasons cited for the lack of discounting were most commonly (1) the customers know the prices, and (2) if the price is higher, then the customer will simply go and purchase at another market stall/shop. 5.2.6 Village Savings and Loan Associations The launch of the VSLA activity occurred in February 2014, approximately three months in advance of the launch of the other SSN activities. Over the life of the program, Kore Lavi aims to establish 1,200 VSLA groups with a total of approximately 36,000 members. The VSLA and SSN were originally conceived as 100% overlapping with the VSLA activity comprising an “exit” strategy” for SSN beneficiaries. Notwithstanding, the early start has resulted in a low level of participation by SSN beneficiaries in the VSLAs. Program staff interviewed gave different estimates as to the share of VSLA members drawn from SSN beneficiaries, with a consensus that the total share lies somewhere between 19-27%. CARE has implemented VSLA activities previously in Haiti, but not in Kore Lavi target areas. The VSLA activity functions like a mutuelle de solidarité, which is another form of savings and loans association widely known in Haiti. VSLA is referred to locally as tipa tipa. Each VSLA has 25 to 30 members (optimally 30). Each member pays in HTG25 (US$0.44) each week, which is referred to as a “step.” Most loans are used for expanding existing businesses and to a lesser extent to pay for school fees. For nearly all of FGD Kore Lavi Title II Program Haiti – Midterm Evaluation 50 5050 respondents, the VSLA was the first time that the respondent had participated in a savings and lending group or had access to any type of “formal” credit. FGD respondents expressed some initial reticence to joining VSLA groups due to the perception that the VSLA activity is not affordable and/or concerns about the security of the money saved. Once launched, however, the VSLA has proven to be universally and extremely well received by beneficiaries. “When we saw the box, we were reassured.” (FGD with VSLA members) FGD respondents universally praised the VSLA activity, which they credited with allowing them to save and borrow money at reasonable rates of interest. In addition, the beneficiaries expressed a strong interest in receiving additional training in a wide variety of topics, which suggests that the VSLA activity is encouraging learning. Additional topics requested included; food security, health, math, and numerous topics related to small business such as cattle rearing, basic food processing, how to grow certain specialized crops, and “how to make shampoo from aloe.” Notably the most common response was “we want training in everything,” although looking at the specific list of topics mentioned their interests appear to lean more heavily in the direction of more training on small business development-related topics. In the case of business finance, however, interviewees universally cited inventory credit for expanding existing business, as opposed to start-up funds, which may mean that those without existing microenterprises may not perceive similar benefits from VSLA participation. Common business uses for VSLA loans cited during FGDs included inventory credit for retail of charcoal, small animals (especially goats), cloth for tailoring, flour for baking, sweets, cigarettes, staple foods, fresh foods, and spices. Although MFIs typically support many of these activities, the high cost of interest makes VSLA a more viable source of credit for many of the VSLA members. In FGDs with VSLA members, the majority of people interviewed stated that they had either trained people on VSLA methodology or had assisted other people to form new VSLA groups. “The most difficult thing in forming a new group is making people understand the purpose of tipa tipa.” (FGD with VSLA members) However, SSN beneficiaries participating in mixed FGDs with VSLA members consistently noted that the VSLA members did not train or encourage others to join their own VSLA group due to the potential liability that this might incur, as expressed in the following quote. “I don’t invite people to join tipa tipa because then I will be responsible for their debt.” (FGD with SSN and VSLA members) 5.3 SO3: Maternal and Child Nutritional Status Improved 5.3.1 Insufficiency of Food Rations According to FGD respondents, food rations are the least effective part of the program for bringing about appropriate nutrition behaviors. Interviews and FGDs revealed that SO3 beneficiaries are very food insecure, while beneficiaries in some areas, such as the Northwest, appear to be worse off than others. While most FGD respondents did not give the appearance that they were starving, most claimed that they did not have enough food and neither did their neighbors. Kore Lavi Title II Program Haiti – Midterm Evaluation 51 51 51 Because of widespread hunger in the program communities, SO3 beneficiaries are under pressure to share their food ration with other household members and even with neighbors and strangers, with the latter including non-beneficiaries begging food at the food distribution point. The occurrence of sharing food rations was noted in every SO3 FGD conducted whether with mothers, Lead Mothers, or CHAs, as reflected in the following two quotes: “They are starving too, just like us. We can’t refuse them.” (SO3 beneficiary FGD) “We have to feed our other children too, not just the baby and not just me.” (SO3 beneficiary FGD) The end result of this widespread food sharing is that the SO3 food ration is not used as the program intended. Respondent mothers said they make no attempt to parcel out the food for the month; it is simply too little and too many other people in the household are hungry. The mothers understood well that the ration was intended as a food supplement to prevent babies from being born undernourished, to increase breast milk and to prevent malnutrition in infants, but this did not affect their behavior. Thus it was not an issue of not understanding the purpose of the food ration, which indeed the mothers appeared to understand reasonably well. Rather, it was simply a matter of the necessity to share it with other hungry people. When asked what would happen if they did not share the food ration, respondents generally indicated that something “bad” might happen to them. One group of SO3 beneficiaries in the Northwest said that non￾beneficiaries in their area might actually literally kill them if they did not share their food, although this is clearly an extreme case, and there is nothing in the research to suggest that this necessarily a commonly perceived risk. Certain FGD respondents noted that then the WFP food distribution truck arrives, non￾beneficiaries crowd around the distribution site, and if not given food, have gone as far as to block the road so that neither the truck or Nurse Supervisor could pass, as recounted by one FGD respondent: “We wait here for the distribution truck to arrive. We are here early, waiting. The non-beneficiaries also come and wait with us. They have with them plastic bags ready to be filled. After the distribution, they beg from us and from the Supervisor. Sometimes they block the road and won’t let the Supervisor pass on her motorcycle until she gives them some food.” (SO3 beneficiary FGD) In such cases, SO3 beneficiaries give away food to non-beneficiaries to ease the tension of the situation, and sometimes even the Nurse Supervisor gives non-beneficiaries money from her own pocket to get them to leave. The Nurse Supervisor in the Northwest observed that non-beneficiaries were the “greatest challenge” in her job. While these are admittedly again more extreme examples, they are illustrative of the pressure SO3 beneficiaries are under to share their food rations. Issues related to the sufficiency of SO3 food rations should be considered in the context of the decision made early in the program to issue a single food ration of 4.5 kilograms as opposed to the proposal to issue two food rations, one a “family ration” of 11.7 kilograms and the other a “beneficiary ration” of 4.5 kilograms. Anticipating that SO3 beneficiaries would share be required to share, the idea was to provide the two rations in separate bags, so that beneficiaries could share the family ration with family members and others, while keeping the beneficiary ration for the intended beneficiaries (lactating or pregnant women and children under two years). As explained to the evaluation team, the decision to forego the family ration, and instead issue the single beneficiary ration, was motivated, at least in part, by the fear that issuing larger rations would create perverse incentives for women to have more children so as to increase the amount of the “free” food received. The option of providing a family ration in addition to the beneficiary Kore Lavi Title II Program Haiti – Midterm Evaluation 52 5252 ration is now apparently back on the table given the widespread evidence that the beneficiary ration is not being used as intended due to sharing. 5.3.2 Quality of Food Rations With few exceptions, FGD and KII respondents did not report many problems with the quality of the food rations. While there were a few cases in which respondents noted that at the time of pick up, or within a couple of days, there were insects in the food or that the food smelled bad or was otherwise spoiled, these were the exceptions rather than the rule. To the extent there are more systemic problems with the food rations, these appear to be more problems of quantity not of quality. 5.3.3 Issues with Beneficiary Registration SO3 beneficiaries, Lead Mothers and CHAs say that women have to wait two months after registration to get their first food rations. They also reported in some sites that women were cut off from the registry after their babies turned four months old, and then added again when their babies turned six months old. Others reported being cut off permanently after 12 months. No one knew why these problems occurred; they were simply not on the list and then sometimes their names appeared on the list once again. It was not possible from the interviews and FGDs to get a sense of how systemic this problem is. For those who are affected by it, however, it is a significant inconvenience and is suggestive of underlying issues with the registration process (see MEAL Section). 5.3.4 Issues with Ration Distribution The KIIs and FGDs further uncovered a variety of problems at distribution points that occurred frequently enough to merit mention. These included the following: • Hundreds of beneficiaries assembled at distribution points, which are often over-crowded and unable to accommodate all beneficiaries in reasonable comfort. • Late deliveries and long waits to receive food rations. • Delivery schedule changes that come too late to inform beneficiaries ahead of time and/or which require a change in the distribution point, often to one less convenient or less able to accommodate a large group of people. • Presence of non-beneficiaries at distribution points pressuring beneficiaries to share their food. • Long walks to the distribution points for certain beneficiaries. Several of the above problems, moreover, are interrelated such that if one occurs, it affects the others as well. For example, a last minute change to the WFP’s delivery schedule causes a scheduling conflict with one of its distribution points creating the need to find an alternative site, which cannot accommodate all of the beneficiaries in reasonable comfort, and also resulting in lost time for beneficiaries both due to longer wait periods and the need to return at a later time to the new delivery point. It is notable that, notwithstanding the many issues related to the SO3 food rations discussed above, beneficiaries participating in FGDs uniformly expressed their appreciation for the food rations and wanted them to continue. Their situation is such that, from the beneficiaries’ perspective, even a little extra bit of Kore Lavi Title II Program Haiti – Midterm Evaluation 53 5353 food at the margin is perceived to be worth the transaction costs incurred to receive it, although clearly they would prefer greater benefits relative to the transaction costs. 5.3.5 Cascade Training Model The cascade training implemented under SO3 – with Nurse Supervisors at the top training and supervising CHAs, CHAs training and supervising Care Groups of Lead Mothers, and Lead Mothers training and supporting the Care Groups in their communities – appears by all accounts to be working well. FGD and KII respondents uniformly said that the training materials and messages were well formulated and well understood. Respondent mothers reported that their husbands or fathers of their children were learning also, as well as grandparents and other children in the household. As a result, a number of FGD respondents indicated that they are now more confident that if they leave their children with other family members that they will be fed and cared for properly. Limited anecdotal evidence from the FGDs indicates that in some cases, attitudes among fathers also appear to be changing. Some are now feeding babies, bathing babies and taking them to health centers more so than in the past. Respondents at all levels—Supervisors, CHAs, Lead Mothers and SO3 mothers—felt that men need to be targeted more than currently being done with good family life messages and practices. For example, some women expressed an interest in family planning education for men, as many husbands or partners remain opposed for cultural and other reasons. The evaluation team found wide disparities in the number of individuals or households Nurse Supervisors, CHAs and Lead Mothers were responsible for training. Nurse Supervisors interviewed trained anywhere from 5-11 CHAs (Figure 10), one of whom additionally supervised 99 Lead Mothers due to what she explained was a lack of a sufficient number of CHAs for the number of Lead Mothers for SO3 beneficiaries. In addition, among the 26 CHAs interviewed, the number of Lead Mothers they supervised ranged 7 to 55. This could occur even in one area. In Gonaives, for example, the eight CHAs had between 7 and 45 Lead Mothers each. This was explained, in part, by differences in population density and the geography of the areas covered by each CHA, but it is a range that asks for further follow up for clarification because it varies so greatly from the ideal model. While the Nurse Supervisors and CHAs interviewed during the MTE may not be representative of all Nurse Supervisors and Lead Mothers in the program, their experience suggests that there may be significant variation in how the cascade training model is implemented. Kore Lavi Title II Program Haiti – Midterm Evaluation 54 5454 Figure 10: Number of CHAs and Lead Mothers Supervised in Gonaives Some Nurse Supervisors appear to have a heavy workload primarily because of the long distances they must cover to visit the areas they are responsible for and to be present at each and every food distribution. Roseanna in Anse Rouge (Artibonite), for example, said she had to be present at 11 distribution points per month (in addition to supervising six CHAs and another 99 Lead Mothers). Nurse Supervisors are given motorcycles to ride to the distribution sites and to visit the CHA catchment areas and must at times travel over extremely bad roads. All Nurse Supervisors interviewed said that the number of CHAs in the program is not enough. This view was echoed by the program’s focal point as MSPP who noted that around 10,000 CHAs were needed nationwide, while as of now, only a few hundred were currently operating. Lead Mothers reported training and supporting 10-12 households on average. They did not feel that their workloads were too heavy. They said they visited households in their group two times per month, and they attended one 2-hour meeting per month with their respective CHAs for training and to turn in reports. They report on the status of mothers and babies in the households, deaths, instances of diarrhea, and members who have moved away. As an aside, Lead Mothers do not like wearing the yellow T-shirts given them by the program as they are too big and unattractive, and people make negative remarks to them, saying they must be highly paid since they are wearing a program T-shirt. While this may appear a relatively minor issue, for those Lead Mothers raising it, it was clearly an important issue to them. 5.3.6 Anecdotal Evidence of Behavior Change among SO3 Beneficiaries The FGDs with SO3 beneficiaries uncovered a number of anecdotes suggesting that the program’s SBCC interventions under SO3 are producing positive benefits at least among some beneficiaries, both in terms of knowledge acquisition and child feeding practices. FGD respondents were by and large able to repeat the key messages being promoted under program SO3 SBCC interventions, including the importance of breast-feeding for infants and the importance of feeding children more frequently and with a greater Nurse Supervisor Number CHAs Supervised Junia 5 Elizabeth 6 Roseanna 6 (plus 99 Lead Mothers) Marie 11 CHAs Number Lead Mothers Supervised 1 7 2 8 3 13 4 15 5 25 6 37 7 42 8 45 Kore Lavi Title II Program Haiti – Midterm Evaluation 55 5555 variety of fresh foods. The same respondents, moreover, claimed that, as a result of the SBCC interventions, they have changed their child feeding practices. Women who said they had bottle-fed earlier children say that they are now breastfeeding their infant children. Previously, they claim not to have known about the importance of exclusive breastfeeding for the first six months, but now they know and are practicing it. In other cases, women reported providing supplementary feeding to their children on a more frequent basis. 5.3.7 Helpfulness of CADEPS and CASECs According to the SO3 beneficiaries, Lead Mothers and CHAs interviewed during the MTE, CADEPs and CASECs have been both helpful and influential in how the program has unfolded in the field, a role that has been abetted by the training in infant nutrition and health they have received through the program. Among their important roles has been mobilizing community members to participate in the program. Every three months they notify the families in their area that they can register for the Kore Lavi program if they are pregnant, breastfeeding or have a child less than two years of age. They do not necessarily go house to house, but they do put out the word regarding the program and provide information on how to register for the program. It was the near consensus opinion of those interviewed that this system of program mobilization was working well. 5.3.8 Institutionalization of the Care Group Approach When asked how the Care Group approach might be institutionalized or carried on after Kore Lavi ends, health center nurses, Nurse Supervisors and CHAs all responded that it could be carried on if there were sufficient money to pay the salaries of the Nurse Supervisors and CHAs. Aside from the issue of payment and other forms of monetary or non-monetary compensation (see below), those health center nurses, Nurse Supervisors and CHAs interviewed during the MTE expressed near unanimous support for the program and their role in it and believed that it was generating important benefits for local families. While some of those interviewed would like to stay on indefinitely, others would prefer to rotate out, but most all agreed would be possible to maintain the system over time as long as there were Nurse Supervisors and CHAs to keep it running. In three of the four health centers visited, the Nurse Supervisors and CHAs appear to have been successfully integrated with the program and were operating as expected. Nurse Supervisors and CHAs are attached to health centers, while Kore Lavi employs some, but not all, of the CHAs. In some health centers, half work for Kore Lavi and receive their salaries from the program, and others work for MSPP. Kore Lavi CHAs appear to be an “add-on” work force for the duration of the project. Health Center supervisory staff, primarily nurses and auxiliary nurses, are supposed to work closely with program Nurse Supervisors and CHAs to identify MAM infants and to reinforce the same set of messages to mothers and fathers who bring their infants for immunizations and other treatments. In three health centers visited, this coordination was taking place and in one, the status of Kore Lavi integration seemed unclear. The health centers are generally not linked in with the Care Group activities other than through reporting back from CHAs. Health center staff generally does not interact directly with the Care Groups, as this is the job of the CHAs. 5.3.9 Conditions at Health Centers The evaluation team visited a total of four health centers. Health center nurses reported that their workload has increased by about six hours per week as a result of referrals from the Kore Lavi program. Kore Lavi Title II Program Haiti – Midterm Evaluation 56 5656 One nurse suggested that their pay should be increased accordingly. Nonetheless, none of the nurses felt that their services were being over-stretched by the influx of Kore Lavi patients. Two of the visited health centers were very new and in good shape and appeared to be well stocked with supplies, while the other two were in bad condition with few supplies and broken equipment. This finding is consistent with that from the FDHS monitoring activities, which similarly uncovered mixed findings related to health center conditions. While some of the health centers visited by the field monitoring team were in good shape, others lacked basic equipment and supplies. Among the health centers visited by the FDHS field monitors in Northwest, Southeast, Center and Artibonite departments, 10 centers were missing basic micronutrients (e.g. vitamin A, iodine, folic acid, iron) and/or oral rehydration salts, seven centers lacked an adequate supply of mother and child growth monitoring cards, and six centers had missing or malfunctioning equipment (e.g., stethoscopes, weight monitoring scales, sphygmomanometers). Kore Lavi is aware of the conditions at its assisted health centers described above. With regards to the missing growth monitoring materials, the program has replenished and distributed the growth monitoring cards to its health centers during November and December 2015, and it is currently in the process of purchasing and replacing the missing weight monitoring scales. With regards to the missing micronutrients, this is evidently not a problem of availability but of distribution logistics. Sufficient supplies of the missing micronutrients are available at the MSPP at both national and department levels, but the MSPP has not developed a workable distribution system to get the micronutrients to the health centers on a routine basis. Kore Lavi plans to deal with the problem in the short-term by piggybacking the distribution of the micronutrients with its upcoming supervision visits to the health centers along with departmental MSPP staff. While Kore Lavi has taken responsibility for now to ensure that adequate supplies of growth monitoring materials and micronutrients are distributed to the health centers, this is at best a short-term fix. More generally, the distribution of essential materials to health centers by national health ministry has historically been a major problem, and remains an ongoing problem with no long-term solution currently in the offing. 5.3.10 Treating Moderate and Acute Malnutrition Interviews suggested that health center staff takes MAM seriously. The protocol requires advising parents on good infant feeding practices and giving them Plumpy Sup for 30 days, when it is available. The health centers receive a supply of Plumpy Sup based on a target (projected) number of cases. Respondents mentioned that the target number of cases has been reduced recently resulting in not having sufficient Plumpy Sup for the number of MAM infants being brought in. It thus appears that some MAM infants are not being helped with Plumpy Sup supplements. Nurses mentioned that some MAM parents walk up to three hours to get to the health center and then go away empty-handed or with the 30 packets for the month, which seems so little to them. Some who should return for a second month of Plumpy Sup do not because they feel they fell it is so little. They seem not to understand that the Plumpy Sup prescription of 30 packets per month will restore the health of their infants. 5.3.11 Training of Health Center Staff and Others The evaluation team interviewed four health center staff in three of the health centers. Kore Lavi has provided training to nurses and auxiliary nurses in the health centers, and they in turn have trained their assistants. All interviewees said they had had received this training before, but it was good to have a Kore Lavi Title II Program Haiti – Midterm Evaluation 57 5757 refresher. Within the short time allotted to visit each health center (usually one hour), the evaluation team was unable to determine whether patients were receiving adequate attention and treatment. Overall, Kore Lavi has provided training to approximately 200 health center staff on a wide variety of topics (e.g., birth preparedness and maternity services, immunization, growth monitoring and promotion) for health center staff under SO3 as part of its “essential nutrition actions,” in addition to training on acute malnutrition management, in both cases following training protocols developed by the MSPP. Kore Lavi had initially planned to conduct an assessment of the quality of services provided by program￾assisted health centers using CADEPs during Year 2 of the program with the results used to inform capacity building plans at each health center. However, the program has subsequently decided not to undertake this assessment but instead to focus its efforts on providing and improving MAM services at the health centers. Kore Lavi also provides training to traditional and non-formal health service providers, such as traditional birth attendants, as an integral part of its SO3 strategy. During 2016, the program intends to increase the number of commune-level health trainings it does to include other traditional service providers, such as in this case religious leaders. 5.3.12 Issues with MAM Outreach According to key informants and FGD respondents, MAM rates remain high in the program communities. Interviewees generally agreed that MAM infants tend to come from the very poorest households, many of which are located at long distances from health centers and thus outside the reach of the CHAs, particularly given their already heavy case load. According to one nurse, “They exist where the CHAs don’t go.” In addition, some MAM infants require other medical prescriptions that parents cannot afford and therefore do not buy. Health centers refer these infants to hospitals for in-patient care; however, parents often do not take their infants to the hospital because they cannot afford to pay. Most interviewees also said that MAM is caused largely by parental neglect of the children, leaving the children for long periods of time with other children and/or sub-standard care while the parents are in the fields or at work. 5.3.13 Compensation for CHAs and Lead Mothers A consistent finding emerging from both the MTE fieldwork and the FDHS field monitoring visits is a demand for increased monetary compensation among CHAs, including increased salaries and reimbursement for travel and mobile phone expenses. The lack of compensation also emerged as an issue among Lead Mothers who also wish to receive some form of monetary or in-kind compensation for the work they do. According to the CHAs interviewed during the MTE, hiring motorcycles to travel around their work areas eats into their salaries. While they do receive a transport allowance to attend management meetings or to pick up vaccination supplies from health centers, this is but a small part of what they spend on travel to travel on bad roads over large geographical areas to meet with Lead Mothers. Their jobs also require them to use their cell phones frequently so as to check up on Lead Mothers and to call them to meetings. Lead Mothers also work hard but receive no compensation for their work. Not surprisingly, Lead Mothers interviewed expressed a strong demand for some form of compensation, or at the very least reimbursement for expenses incurred to travel from meeting site to meeting site, attend program meetings or functions Kore Lavi Title II Program Haiti – Midterm Evaluation 58 5858 and make mobile calls, or perhaps even a small allowance to purchase refreshments during long meetings. In lieu of a salary, the Evaluation Team found that at least in some areas, Lead Mothers have received gifts from the program, such as “kitchen kits,” a tablecloth, a curtain or a bed cover. It was difficult, however, to ascertain who received what and when and what the effect has been. Two Nurse Supervisors said the Lead Mothers get one gift a year. Another Nurse Supervisor said that giving Lead Mothers a complete kitchen kit (two serving bowls, four eating bowls, four cups, a frying pan, eating utensils, cooking utensils and a thermos) has made a huge difference in incentivizing Lead Mothers. However, another said they only give out part of a kitchen kit, tablecloth or thermos or other small gifts, and that this has only caused arguments among Lead Mothers who get things they do not want or need when what they really want is cash compensation. 5.3.14 Demand for Additional Training All CHAs, Lead Mothers and beneficiaries interviewed said they wanted more training on any subject, although they suggested in particular more training on health, finance, running a business, better farming practices, and how to raise healthier animals. Mothers would also welcome training for their husbands and live-in partners on family life and relationships. Interviewees further expressed a desire for training on water, sanitation and hygiene (WASH), as WASH is closely related to nutritional outcomes for infants and children. Care Groups offer a good opportunity for MAST or MSPP to expand their messaging activities and build upon the foundations laid by Kore Lavi. 5.3.15 SO3 Performance Monitoring The SO3 team is moving forward with plans to conduct ongoing field monitoring of its SO3 activities. In December 2015, Kore Lavi provided training to its SO3 team in addition to CHAs and Lead Mothers in the use of qualitative field performance monitoring tools, with a specific focus on monitoring the implementation and outcomes of Care Group meetings, home visits by Lead Mothers and the training of CHAs by Nurse Supervisors. Further, during 2016, Kore Lavi plans to introduce the use of FGDs to conduct case studies for the purpose of generating lessons learned about what works and what does not work and to assess changes in knowledge, attitudes and practices (KAP) of SO3 beneficiaries. The current intention is to task the departmental health and technical officers with the responsibility for the ongoing field monitoring using largely qualitative, small-scale and rapid assessment methods. Kore Lavi’s initial plans also called for providing training to CADEPs to conduct monitoring of the program assisted health centers, but these plans have not come to fruition, the SO3 management does not anticipating launching this training activity in the future. 5.3.16 Primary Benefits as Perceived by SO3 Beneficiaries Overall, aspects of the SO3 interventions that beneficiaries found must valuable included training on infant feeding, encouragement to go to health centers and the provision of food rations. With regards to infant feeding, a consistent finding was how much mothers appreciated the training and education they received in this area. In fact, in each FGD with SO3 mothers, Lead Mothers and CHAs, respondents consistently mentioned how much they appreciated the training they received on infant feeding through the program. Mothers said that before the program, they did not know the importance of exclusive breastfeeding for six months, nor did they know how to feed a baby at different ages or about the importance of feeding infants from the three food groups (energy foods, body-building foods and body-regulating foods -vitamin and minerals). A second thing FGD respondents consistently appreciated was being encouraged to go to the health center Kore Lavi Title II Program Haiti – Midterm Evaluation 59 5959 for pregnancy testing, pre-natal care and baby care. Respondent mothers mentioned that they now take their babies to health centers when they are sick in addition to taking them for growth monitoring where they have found the health center staff to be, on the whole very helpful. The primary obstacle to visiting the health center is the distance many must travel to arrive there; it was common to hear respondents mention walking 2-3 hours. FGD respondents also appreciated the growth monitoring carried out by CHAs in their areas with some mothers walking as much as an hour to attend these meetings. The third benefit to program participation most frequently mentioned by FGD participants was the food ration provided to pregnant and lactating mothers and children aged 0-23 months. While greatly appreciated, however, respondents also consistently observed that the ration was too small to make much of a difference in their lives lasting on average only from 2 to 8 days. Nonetheless, they insisted that they still wanted to receive these rations because they do add food for a few days, which is better than nothing and which represents food they no longer need to buy. In their positions, every little bit helps, as all family members are hungry, and any food ration helps to reduce their hunger. Thus respondents insisted that they wanted to continue receiving food rations, even if they were too small to have much of an impact. 5.3.17 SO3 Sustainability Strategy Kore Lavi has developed a three-pronged sustainability strategy for SO3. The first prong relates to the food rations and involves replacing the existing food ration with a coupon, thereby also integrating SO2 and SO3 activities more closely as well. Its current plan is to conduct field tests of the SO3 voucher in two program communities and then to scale it up in line with the field test results. At present, Kore Lavi is developing the research protocol with the intention to launch the field tests in June of 2016. The second prong of the SO3 exit strategy is to begin around the third quarter of 2016 to meet with MSPP (including a workshop with the Kore Lavi SO3 team and MSPP departmental and national staff) to determine which SO3 activities can realistically be transitioned to the ministry and how. The third prong involves the integration of MSPP staff into the program SO3 activities. The CHAs working with Kore Lavi are MSPP staff members who are financially supported by CARE/ACF and who are hired based on an MoU with MSPP using the pay scale that has been adopted nation-wide by the ministry for the same type of work. In this way, the recruitment and training of program-affiliated CHAs has been done with an eye toward expanding the MSPP network in such a way that that the ministry can absorb it at the end of the program without major issues. Notwithstanding, it should be noted that it is not certain whether the MSPP can absorb the program￾affiliated CHAs and the attendant costs at program conclusion. As noted by Kore Lavi’s MSPP focal point, “there is no currently plan to transition the health agents to the ministry.” Nationwide, MSPP estimates that it needs about 10,000 health agents, while currently there are only a few hundred operating, while health centers suffer from systemic staff shortages. Moreover, “the strategy for sustainability doesn’t necessarily indicate who would be in charge of it . . . The problem is that the current approach is not sustainable because these health agents are paid by NGOs and are not part of the national budget, so that when financial support ends, what will happen then? This happens all the time. The project ends, and we will lose all the beneficiaries and results that we had.” Kore Lavi Title II Program Haiti – Midterm Evaluation 60 6060 5.4 SO4: Haitian institutions’ capacity to effectively lead and manage safety net programming improved 5.4.1 Background Increasing institutional capacity to lead and manage donor-initiated development programming is always challenging, especially for safety net programming and development food assistance programs. This is especially true in Haiti, where political instability, poor fiscal management, and chronic poverty have hindered the development of human capital, effective public institutions, and empowered civil society organizations for the past several decades. As such, metrics of success and stakeholder expectations of Kore Lavi institutional capacity development efforts should be realistic and strategically managed by both implementing partners and USAID. 5.4.2 Evaluation Approach Initial discussions with Kore Lavi program management team members and senior officials at MAST intimated that the common understanding of success of what was broadly referred to as “institutionalization” was the eventual full operationalization and ownership of Kore Lavi activities. The evaluation team chose not to use this metric to frame its assessment, as the actual strategic objective speaks to increased capacity of Haitian institutions, but not necessarily the full assumption of Kore Lavi activities. The expectation that a four-year donor initiative can establish a well-functioning national safety net program is not realistic,15 nor is this outcome a stated objective of the Kore Lavi program. Therefore, the evaluation team assessed Kore Lavi SO4 activities and initiatives through the lens of increasing institutional capacity per se, including a particular focus on how institutional capacity efforts can be improved in the second half of the program. 5.4.3 Demand and Supply Gaps for Institutional Support at MAST Kore Lavi both underestimated the capacity development needs of MAST at project inception and overestimated the commitment and capacity of MAST to make the necessary investment to procure and develop its staff resources as necessary. Contributing to existing gaps in the supply and demand for institutional support at MAST was the failure of a planned government-to-government (G2G) institutional support mechanism to launch. During the initial program stages, there was a plan for USAID to provide direct funding support to MAST for internal capacity development through a G2G mechanism that Kore Lavi was expected to complement. MAST would have been the first government ministry to receive this type of US government support. USAID performed a pre-assessment of MAST to identify areas for capacity development to be followed by a detailed assessment. The G2G mechanism was going to handle all the strategic level support, while Kore Lavi would cover the operational aspects focused on social safety nets. Unfortunately, the G2G mechanism was put on hold and has not been revived. 15 Brazil’s Bolsa Familia social safety net initiative, for example, has been receiving significant financial ($572 million from 2004 to 2009) and technical assistance support for the past twelve years. Source: http://www.worldbank.org/projects/P087713/br-bolsa-familia-1st-apl?lang=en&tab=overview Kore Lavi Title II Program Haiti – Midterm Evaluation 61 61 61 5.4.4 Kore Lavi’s Facilitation of MAST-Led Coordination Kore Lavi successfully facilitated MAST leadership of technical working groups and coordination meetings. Kore Lavi’s facilitation of the early establishment of technical working groups led by MAST effectively positioned the agency to lead Kore Lavi coordination efforts and to a lesser extent, program implementation. MAST led 35 coordination meetings in the first year of the program, which far exceed the target of 10 meetings in the program’s first year. Although other challenges to increasing government capacity to implement safety net programs exist, Kore Lavi has clearly been successful in the facilitation of MAST-led coordination. 5.4.5 Creating Institutional Buy-In at MAST Kore Lavi has worked effectively with key senior MAST officials, creating significant buy-in and support for the program. It is clear from key informant interviews with senior MAST officials that the Kore Lavi Project Management Team has done an excellent job of engaging MAST officials in key programmatic decisions, and development of key program strategies. MAST officials are extremely appreciative of the program and clearly demonstrate a high-level of ownership and support of Kore Lavi activities. 5.4.6 Lack of Institutionalization Plan for MAST The lack of an institutionalization plan for MAST has decreased the coherence and effectiveness of governmental capacity development efforts. Key informant interviews with MAST staff and Kore Lavi Project Management Team members indicate that governmental capacity development efforts were somewhat ad hoc in the first 18 months of the program, although project staff did conduct workshops in the summer of 2105 in four departments to discuss the meaning of institutionalization and ideas on how to measure it According to one senior program manager, “The concept of transferring the project to the government was kind of murky. Prior to now, there was not even a clear definition of who the relevant government counterparts were…. Institutionalization was put as a goal without defining it or how far the project would get, so there is no real consensus about what is meant by institutionalization…. All of this should have been done at the beginning.” (KII with Senior Manager) Although the signing of the MAST-Kore Lavi Memorandum of Understanding (Fall 2015) was a good first step, the lack of integration of Kore Lavi into a national framework (national policy or legislation) threatens sustainability of the project. The Kore Lavi IPTT measured progress towards institutionalization primarily by the number of MAST-led coordination meetings, which did not reflect the full complexity of institutionalization The lack of clarity as to what constitutes successful institutionalization slowed the development of a common vision and likely contributed to non-alignment of institutionalization goals and activities. It should be noted, however, that the project has implemented a Performance Scorecard since September 2015, which measures the number of MAST counterparts involved in the project, the number of join field visits, hours of joint work and the quality of MAST relationships with other government institutions involved in Kore Lavi. 5.4.7 Infrequent and Inadequate Capacity Development Trainings Departmental-level MAST staff report infrequent and inadequate capacity development trainings. Departmental-level MAST staff stated that Kore Lavi trainings have focused on program implementation but not direct capacity development. Specific trainings requested included budget management, project management, and technical skills related to vulnerability targeting and community engagement. Kore Lavi Title II Program Haiti – Midterm Evaluation 62 6262 5.4.8 MSPP and CHA Involvement in Kore Lavi MSPP departmental staff and community health agents are actively and effectively involved in Kore Lavi. MSPP departmental staff was very positive about Kore Lavi engagement with CHAs. In particular, Kore Lavi trainings on the promotion of improved feeding practices such as exclusive breastfeeding and CHA-engagement with Lead Mothers were mentioned as particularly useful. Although one department manager reporting a lack of coordination at the beginning of the program, with Kore Lavi staff engaging health centers directly without notification of MSPP, he also stated that the establishment of monthly coordination meetings earlier in 2015 has effectively facilitated the participation, input, and confidence of various stakeholders, including MSPP. 5.4.9 Lack of Budgetary Transparency The lack of budget transparency decreases trust among MAST staff at various levels. Both senior MAST officials and lower-level staff report that they have never been privy to Kore Lavi budget allocations related to MAST cooperation, coordination and capacity development. In particular, MAST officials expressed concern related to the original elaboration of the budget, and expressed frustration that their program recommendations are often dismissed due to budget constraints. Although MAST officials are clearly supportive of the program, they did report that increased budget transparency would help galvanize support at various levels from the Minister to the Director General to lower-level staff members. Sharing of information regarding high-level budget line items, such as amounts available for various activities involving MAST, will help set expectations and increase the level of trust of MAST officials. 5.4.10 Weak Program Communication and Outreach Weak Kore Lavi communications and outreach contributes to unrealistic expectations and misunderstanding among junior-level MAST staff. Despite evidence of frequent and meaningful interaction between Kore Lavi staff and senior MAST staff, interviews with lower level MAST staff working on Kore Lavi activities demonstrated unrealistic expectations of Kore Lavi capacity development efforts and an apparent lack of awareness of the high-level of Kore Lavi-MAST coordination and engagement and specific activities related especially to vulnerability targeting and the MAST beneficiary information system. The lack of effective communication from Kore Lavi and between senior and junior-level MAST staff regarding program objectives, activities, and coordination efforts may be responsible for junior MAST staff misconceptions about Kore Lavi. Kore Lavi should facilitate regular coordination meetings between senior and junior MAST staff to ensure better flow down of Kore Lavi information/communication. 5.4.11 Financial Constraints Financial resource constraints for per diems, logistics support and staff compensation hinders MAST recruitment and retention efforts and constrains MAST field monitoring of activities. Although Kore Lavi and MAST have agreed to standardize per diem rates for Kore Lavi-sponsored staff in line with GOH guidelines, MAST staff at various levels report that Kore Lavi budget allocations for field visit expenses limits the frequency of monitoring visits center-level staff are able to conduct. Additionally, MAST cited the lack of financial incentives as a key factor in Kore Lavi-related staff recruitment and retention efforts. MAST staff members reportedly are reluctant to accept Kore Lavi-related appointments, as they perceive these appointments as adding significantly to their workload without a commensurate increase in compensation. Lastly, for positions related to the Kore Lavi/MAST information system, standard MAST salary rates are insufficient to attract the candidates with the required skill set for database and information management. Kore Lavi Title II Program Haiti – Midterm Evaluation 63 6363 A senior MAST official pointed out that SO1-related skills command $2,000 to $3,000 per month, whereas MAST salaries for the same position average approximately $400 per month. Financial constraints for future takeover of the program in part or in full by MAST are also important for sustainability. This entails that MAST will live up to the obligations stipulated in the MOU signed with the consortium by accessing adequate financial resources (either in its allocated budget or by requesting a specific budgetary support from the Ministry of Finance). 5.4.12 Effect of Staffing Shortfall The inability of MAST to appoint staff to Kore Lavi activities (especially for SO1) is reducing the likelihood MAST will be able to assume Kore Lavi activities at the end of the program and is dramatically slowing progress towards increased capacity of MAST to lead and manage safety net programming. For example, by the end of Year 1, only two of 15 positions related to information management had been filled, and one of the two staff members (the Information Manager) left in early 2015. Kore Lavi is tracking MAST staff time on Kore Lavi initiatives and activities, but participation levels are still well below program targets. Kore Lavi is not able to develop staff capacity if no staff is made available to train. 5.4.13 Community-Level Engagement Kore Lavi has effectively identified and engaged community-level organizations such as CADEPs, CBO Reseaus (networks), and community-based organizations. As of November 2015, Kore Lavi was working with 32 civil society organizations (CSOs) including CADEPs, RODECs (Reseaus), and CBOs. Focus group discussions with CADEPs and other CSOs demonstrated the active engagement of these organizations by Kore Lavi, especially related to the promotion of the program and assistance with food voucher distribution. 5.4.14 Capacity Development at CADEPs and CSOs Kore Lavi has increased the capacity of CADEPs and other CSOs to implement Kore Lavi activities and to lead social accountability efforts. FGDs with CADEPs and other CSOs support Kore Lavi reporting (Quarterly Reports and KIIs with SO4 Staff) of significant training in key areas including social protection, program outreach and community sensitization, food voucher distribution assistance, and the understanding and use of Community Scorecards. 5.4.15 Inadequate Training for CADEPs and CSOs in Organizational Strengthening CADEPs and other CSOs have not received adequate training in organizational strengthening. FGDs with CADEPs and other CSOs indicated a lack of familiarity and ownership of action plans. CADEPS report that Kore Lavi capacity development efforts were mostly focused on Kore Lavi project implementation. Several organizations interviewed further report that they have asked for training in financial management, proposal writing, and overall organizational strengthening. FGDs report that these requests have not been addressed. 5.4.16 Lack of CADEP Engagement from Remote Locations CADEPs are not sufficiently engaged in Kore Lavi activities in more remote locations. Although this was not witnessed directly by evaluators, USAID field visits indicate that CADEPs in more remote areas are not sufficiently engaged in Kore Lavi activities. Such a lack of engagement would render efforts to empower civil society to monitor and support safety net activities post-Kore Lavi difficult. One potential reason why Kore Lavi Title II Program Haiti – Midterm Evaluation 64 6464 evaluators may not have witnessed this directly is due to the fact that most evaluation field visits took place in departmental and commune centers due to logistical and time constraints. 5.4.17 Role of Social Accountability Manager The addition of a Kore Lavi Social Accountability Manager is increasing the efficacy of CADEP and CSO capacity building. The original Kore Lavi program design did not adequately staff SO4 activities, assuming that staff working in other program components would be able to facilitate SO4 objectives. The only staff member formally tasked with responsibilities related to SO4 was the Deputy Chief of Party position. As this position (not specific personnel) is often responsible for the administrative and financial management of development programs, tasking this position with the sole oversight of such a critical program component was shortsighted. However, the addition in mid-2015 of a Social Accountability Manager is paying dividends. Evaluators were impressed with the technical competence and clear understanding of Kore Lavi objectives by the Social Accountability Manager. The addition of this position has reinforced the development and adoption of CADEP action plans, and significantly improved the frequency and quality of trainings on social accountability tools such as the Community Scorecard. An additional outcome of adding this position was the effective reinforcement and expansion of community complaint mechanisms such as reference and appeals desks. Key informants report that although the addition of this position was a positive step, one additional position along is not enough to administer the full complement of SO4 initiatives. Specifically, one manager should be tasked with the capacity development of CADEPs and a second manager should oversee the establishment and operations of reference desks and other feedback mechanisms. 5.4.18 Effect of Grants Culture on CSO Enthusiasm A culture and history of grants prevalent among CSOs dampens CSO enthusiasm for the program. In general, Haitian civil society organizations have a history of receiving financial support from international NGOs to participate in development programs. Specifically, a previous project, PRODEP, which established many of the CADEPs participating in Kore Lavi, provided budget support to CADEPs in the form of rent and remuneration for activity implementation. This context has created a challenge for Kore Lavi efforts to engage CADEPs and to increase their organizational and technical capacity. Despite this, Kore Lavi staff has done a commendable job mobilizing CADEP members to assist with program activities. FGDs indicated that CADEP members are largely enthusiastic about their participation in the program. However, CADEP members frequently mentioned the need for budget support and payment of incentives related to program implementation. 5.4.19 Asymmetric Engagement of CADEPs and CSOs Asymmetric engagement of CADEPS and CSOs is diminishing enthusiasm for the program. CADEPs and other CSOs report that they feel they do not have an equal partnership with Kore Lavi, and that they are basically unpaid staff members who are charged with activity implementation without sufficient input and coordination. CADEPs further stated in FGDs that they are given little notice by Kore Lavi when tasked with activity implementation, community events, or arrangement of meetings on Kore Lavi’s behalf. CADEPs requested more frequent communication and increased respect for their time and position in communities. Kore Lavi Title II Program Haiti – Midterm Evaluation 65 6565 5.4.20 GOH Capacity to Respond to Food Emergencies Kore Lavi efforts to increase government capacity to respond to food emergencies are inadequate. Kore Lavi facilitated the signing of an MOU between MAST and MARNDR/CNSA to engage CNSA in the execution of vulnerability surveys. Although the first Kore Lavi Annual Report stated that Kore Lavi provided financial resources for CNSA to conduct an assessment of the Spring Campaign and to update livelihoods profiles in rural areas, it is not evident that Kore Lavi engaged CNSA or other government institutions in adequate emergency response and planning activities for possible future food emergencies. 5.4.21 Actions to Improve SO4 Programming and Initiatives Kore Lavi is taking effective action to improve SO4 programming and initiatives. Despite previous capacity challenges, the Kore Lavi Project Management Team is taking important measures to accelerate increased capacity of MAST at both center- and departmental-levels. One key example of these measures is the current development of a MAST Institutional Development Plan, which the program expects to finalized in the Second Quarter of FY2016. Further, the addition of a Social Accountability Manager is resulting in a renewed commitment to CADEP action plans and follow-through that will significantly contribute to CADEP capacity development. Lastly, the signing of the MAST-Kore Lavi in the Fall of 2015 was important to clearly define the obligations of each party in terms of program institutionalization. 5.5 Gender To promote gains for men and women, narrow gender gaps and inequalities and empower women, development programs must make gender considerations an integral part of the design and implementation process. This section assesses the role of gender in shaping Kore Lavi interventions and the extent to which they have influenced progress to date. In order to assess the extent to which gender issues influenced the progress of Kore Lavi, the MTE looked at the extent to which gender considerations were integrated in the design and implementation across SOs 1-4. In each SO, the evaluation looked at the key gender-related strengths and weaknesses, gender equality and women’s empowerment promoting activities and the extent to which SO interventions are promoting (and/or failing to promote) program benefits to men and women, starting from beneficiary selection to program implementation and progress achieved two and one-half years into implementation. 5.5.1 Integration of Gender Considerations in Kore Lavi Gender considerations were well integrated into the Kore Lavi program from design to implementation. The design and implementation have been largely informed by gender needs and priorities in Haiti, particularly those of the targeted beneficiary communities, as identified by the findings and recommendations of a gender analysis conducted by CARE in close collaboration with the MCFDF. The integration of gender was achieved through two inter-related actions: (1) gender equality training, which has also included training on gender based violence (GBV) and (2) the integration of gender equality and women’s empowerment promoting activities in almost all components of the program, as described below. 5.5.2 Integration of Gender Considerations in SO1 Kore Lavi’s SO1 interventions do not incorporate any gender specific targeting variables. But to make sure Kore Lavi Title II Program Haiti – Midterm Evaluation 66 6666 that the process is gender sensitive, the composition of the targeting verification committee has a 50/50 gender balance. While there is no reason to suspect that the HDVI or HDVI algorithm generate gender biased results, there are opportunities for gender biases to creep in, particularly as a result of the appeals process, which the program has instituted. Insisting on a gender balance on the verification committee thus represents a good faith effort by the program to ensure more gender equitable targeting outcomes, although the evaluation team was unable to determine extent to which this has contributed to more equitable gender targeting outcomes. 5.5.3 Integration of Gender Considerations in SO2 Under SO2, gender sensitivity and women’s empowerment are deeply embedded in both the food voucher safety net intervention and the VSLA approach. To begin with, 88% of all food vendors engaged in the paper and electronic food voucher program are women. Food trading is a domain in which Haitian women have traditionally dominated. Recognizing this gender trend, Kore Lavi effectively capitalized on this opportunity to build on the women’s comparative advantage. Voucher recipients are both women and men and include the female or male parent in single parent households and the household member. Discussions with SO2 beneficiaries (women and men) suggested that there were no specific gender inequality implications with beneficiary selection. According to respondents, vouchers were delivered to both women and men in a timely manner. To make sure that the male or female voucher recipient is actually the person who makes household nutrition decisions, voucher recipients are observed during household visits by the ASCPs. If there is a discrepancy with regards to which person actually leads the household in nutrition matters, a review of the household head is requested. However, Kore Lavi has accepted the MCFDF’s recommendation to distribute the vouchers to women household members to the extent possible, including putting their pictures on the electronic voucher card. The thinking behind this policy change is to increase the woman’s control over household food resources. Kore Lavi intends to begin implementing this policy during 2016 and then moving forward. Concurrent with the implementation of this new policy, Kore Lavi further plans to conduct field research using rapid assessment methods (e.g., mini-surveys and FGDs) to determine whether a link can be established between the woman’s control over food resources and the ability of the household to cope during lean seasons. Kore Lavi’s hypothesis is that when female control over household food resources improves the households coping outcomes during lean periods. FGDs with SO2 beneficiaries suggested that food vouchers affected at time intra-household gender dynamics in a largely positive direction in that they appear to have enhanced cooperation between household members with regards to food and nutrition matters. SO2 beneficiaries indicated that irrespective of whose name is on the voucher, his/her spouse or partner is able also to collect the voucher and/or use the voucher to buy the accepted food items. Apart from a few instances where it was insinuated that some men use the food voucher to attract new sex partners or to give vouchers to their concubines instead of a female household member, beneficiaries mostly reported that, in the cases where men were the voucher recipients, the man generally gave the voucher to a female household member (wives/female/girlfriends) who is responsible to buy and prepare household food. It should be noted, however, that the evaluation team was not able to get a clear idea on the gender distribution of SO2 voucher recipients; this data was requested but was not available. Kore Lavi Title II Program Haiti – Midterm Evaluation 67 6767 The CARE VSLA model integrates women’s empowerment beginning at its design in which a VSLA membership standard stipulates that at least 70% of all members must be women. This standard was meant to empower women as they are much more involved in business activities (particularly food marketing) than men, and would make good use of the type of capital generated through VSLA. It also responded to the findings of the program’s gender analysis, which indicated that the need for capital was a primary limiting factor to Haitian female microenterprise owners/operators. In addition to building women’s economic power through the growth of their businesses, anecdotal evidence suggests that VSLAs are also giving women more respect within their households as they can contribute more to household financial needs (school fees, food, etc.) and thus have a greater say in household financial decisions. Women are also gaining voice, respect and leadership in their communities through the VSLAs. For instance, in Artibonite, which has 122 VSLA groups (Figure 11), women hold 53.9% of all leadership positions in the VSLAs and are candidates of choice for positions of treasurers and accountant.16 Similar trends were noted in the other communities visited. Figure 11: Women Leadership in VSLAs in Artibonite The large majority of VSLAs in the Kore Lavi targeted communities have a mixed female/male membership consistent with the program’s objective to “facilitate more solidarity between women and men and within the community.” Another gender benefit reported by FGD participants was that, women now have a voice as they have grown their business, increased incomes and can participate in money related meetings at par with men. Kore Lavi has also introduced the idea of community gender dialogue within VSLAs in which the VSLAs are encouraged to have a conversation on gender issues at least once a month. FGD respondents reported that many VSLAs now have gender dialogue on their meeting agenda at least once a month, which involves a 15-30 minute time slot to discuss gender relations in order to encourage group members to embrace gender equality. From the information made available to the evaluation team, such gender discussions are done mainly in an ad hoc manner, while and the content of discussions is context specific and chosen by group members, as opposed to using a formal or tailored curriculum. Together with the field research to determine whether a link exists between the woman’s control over food resources and household coping outcomes, Kore Lavi further plans to conduct similar rapid assessment activities so as determine whether VSLA membership likewise improves the household’s ability to cope during lean seasons 16 This shows that women are more trusted with handling money than men. Position # Positions # Women % Women President 122 35 28.6% Secretary 122 50 40.9% Treasurer 122 74 60.6% Counters 244 170 69.6% Total 610 329 53.9 % Kore Lavi Title II Program Haiti – Midterm Evaluation 68 6868 5.5.4 Integration of Gender Considerations in SO3 The gender equality promoting aspects of SO3 are integrated in the Care Group training and sensitization system. Trainees in the cascade gender equality and GBV training offered by the Kore Lavi gender team include all frontline staff, including the CHAs and Lead Mothers and Fathers. Lead Mothers and Fathers deliver gender equality training and sensitization messages alongside their health, nutrition and breastfeeding messages during monthly household visits. The very fact that Lead Mothers and Fathers work alongside each other, training and sensitizing women and men on gender sensitive issues such as preparing a balanced meal, breastfeeding, childcare and hygiene and sharing of household chores (including childcare and nutrition), once again demonstrates a strong program commitment to promoting gender equality among program beneficiaries of both sexes. Kore Lavi’s practice of targeting is SBCC messages to men as well as women, and its (albeit limited) use of Lead Fathers to deliver such messages is a break from traditional practice, which appears to hold substantial potential, as the following conversation with a Lead Father demonstrates. Why did you decide to become a Lead Father? “I observed the Lead Mothers at work and felt that a man could also do the work they did in the community. I wanted to be a voice for the men.” So how do you feel talking to men and women about breastfeeding and family planning? “I now feel normal. The beginning was a bit awkward.” Do you feel more comfortable talking to your households about nutrition or about gender equality? “I feel equally comfortable speaking to them about the importance of men assisting in household chores, including childcare and food preparation, as I do talking to them about nutrition and breast feeding. I have given both nutrition and gender equality sensitization to my own wife.” Both Lead Mothers and SO3 beneficiaries appreciate the involvement of Lead Fathers. To a question on whether the involvement of men in the care group system was a good idea, the women responded with a rousing “Yes,” because “They can talk to our husbands.” In summary, the presence of Lead Fathers, which in most program areas is still a nascent phenomenon, seems welcomed by all suggesting that Lead Fathers have the potential to become agents of change. 17 Consistent with these findings, Kore Lavi plans to increase its recruitment and use of Lead Fathers to promote behavioral change messages around nutrition. The program’s objective is to involve Lead Fathers in 30% or more of the program’s engagement on nutrition promotion activities with households and communities, including at least two Lead Fathers in each programme commune, both in lieu of and, in particular, alongside the Lead Mothers. The program’s short-term objective is to have 40 Lead Fathers in place and operating by the end of Fiscal Year 2016. FGD participants reported a number of anecdotes about how the SO3 SBCC interventions are producing gradual changes in gender perceptions and practices in their communities. In some cases, men are now 17 The exception here is the Southeast Department where the MTE found that men’s involvement dates back to the start of the program implementation in 2014. It was however, not clear that the men who joined as Lead Fathers (Manman Lide) at the start of the program understood all that the position involved. Kore Lavi Title II Program Haiti – Midterm Evaluation 69 6969 more involved in household MCHN chores, such as childcare, baby feeding, preparing household food, and taking babies for health visits, which are culturally perceived as women’s tasks. A man who used to be called ‘man-woman’ and mocked because he assisted his wife in household chores is now appreciated by community members for his work as a Lead Father. In Plaine de l’arbre, a woman reported that she can now afford to be too tired in the bedroom and not receive a beating for not fulfilling her conjugal duty. Yet other women indicated that they can now ‘go and come’ without much trouble from jealous husbands and partners, because the men now understand that women are also human beings, who should also go out and do what they want to do. While such anecdotes do not add up to evidence of systematic behavioral change, they do indicate that, at least in limited situations, the SBCC interventions are achieving their intended results. 5.5.5 Integration of Gender Considerations in SO4 Under this component, Kore Lavi has provided gender equality and GBV training to Kore Lavi local partners, including CADEPs and CBOs who are in turn expected, under Kore Lavi’s cascade training approach, to train in turn their organizational members and other community members. The outcome of these trainings has varied across the program departments. In departments such as Artibonite and Southeast, the trained CADEP members simply shared the training with other members and delegates of member organizations and left it at that, citing cost constraints. At present, there does not appear to be a process in place to manage or oversee the delivery of trainings by the CADEPs. Possibilities for this role include the gender focal point or identified gender champions. In Center department, however, the gender training and sensitization have been successfully passed on to the community members. This is the case of RODEC18 in Cerca Cavajal, which used the cascade training approach to pass the gender training and sensitization messages to the rest of the community using a combination of theatre, songs, and youth sports. It also piggybacked on local events, such as the International Women’s Day to carry out public sensitization events. FGD participants indicated that the gender focused theatre pieces carrying gender equality themes (messages on gender stereotypes, perceptions on women and men’s roles, relations and capabilities) have attracted a lot of interest and seem to have made a big difference in their community. The diversity of experience and outcomes noted by the evaluation team with regards to gender equality training raises two questions. First, what explains the differences in responses and approaches to gender equality and GBV training/sensitization in the different departments? Second, how can the successful strategies be replicated and scaled-up within and across departments? Answers to both questions would require further exploration into the gender and other socio-economic realities of the different departments that make some more receptive of the gender equality messages and others not so receptive. For instance, would it be that gender inequalities and/or GBV are not as topical and/or prevalent in some departments as in others? What does appear consistent across the departments visited is the fact that women are gradually gaining access into leadership positions within local institutions. In the three FGDs held with CADEPs/CASECs (Center, Artibonite and Southeast), respondents noted that efforts are being made to encourage women to get involved in leadership positions of these organizations and that these are paying off. Among the CADEPs participating in the evaluation, there was not one without at least one female on the management board. Moreover, women occupy 16% of CADEP leadership positions in Jacmel, 30% of RODEC leadership 18 Resseaux des Organisations pour le Development de Cerca Cavajal (RODEC) is similar to a CADEP in the type of activities and services rendered to their communities Kore Lavi Title II Program Haiti – Midterm Evaluation 70 7070 positions in Cerca Cavajal and 42.3% of CADEP leadership positions in Artibonite. According to the female president of the CADEP in Gonaive, “A woman now looks at men in the face while speaking at meetings. This was not possible in the past when women dared not speak in the presence of men, and when those who dared to speak had to look down, lest they make eye contact with a man.” Figure 12 shows female leadership details for the six CADEPs in Artibonite that participated in the FGDs. According to the respondents, these figures represent a substantial increase in women’s participation in CADEP leadership in Artibonite, thanks at least in part to Kore Lavi.19 Figure 12: Female Leadership in Six CADEPS, Artibonite Gender training for GOH staff involved in Kore Lavi and other safety net programs is still pending. The Kore Lavi Gender Advisor presented the findings of the gender analysis study to relevant MAST staff, which was well received. The program has subsequently done an evaluation of the training needs of MAST staff involved in the program so as to tailor the training to the needs identified. Preparation for the training is now under way. The need for such training, at least in some cases, was evident during the field visits with MAST departmental staff. For, as stated by one MAST focal point, “My work is not focused on gender.” 5.5.6 Integration of Gender Considerations in SSN Programming In close collaboration with Kore Lavi, the MCFDF made two important recommendations regarding the integration of gender sensitivity into SSN programs in Haiti: 1. After selection of beneficiaries for social transfers (such as voucher programs), give priority, where possible, to the woman as the representative of the household. 2. Provide training and sensitization on gender to both staff and beneficiaries of social safety net activities from the onset of all social safety net interventions. Program staff members consider the gender recommendations coming from MCFDF to be a significant achievement, particularly as the recommendations were the outcome of close collaboration between the program and MCFDF and further represent a significant contribution by Kore Lavi to Haiti’s gender 19 Secretary General CADEP # Board Members # Women on Board Women’s Positions % Women 1 7 2 SG19, Advisor 28.6% 2 11 4 SG, Advisor, Member 36.4% 3 7 3 SG, Treasurer, Advisor 42.9% 4 9 5 SG, Treasurer, Advisor, Member 55.6% 5 9 4 SG, Treasurer, Advisor, Member 44.4% 6 9 4 President, SG, Advisor, Delegate 44.4% Total 52 22 N/A 42.3% Kore Lavi Title II Program Haiti – Midterm Evaluation 71 71 71 strategy. The validation by MCFDF is also considered a notable achievement that prepares the way for sustainability of this contribution. Kore Lavi has taken on the implementation of recommendations #1 and #2 above. One way the program has responded to the first recommendation is to begin implementing the policy (described above) to place the woman’s picture on the SO2 electronic voucher cards. With regards to the second recommendation, Kore Lavi has made gender training and sensitization an integral part of program activities. The program’s Gender Advisor delivers the gender training working closely with the MCFDF as part of a productive collaboration in which there is mutual learning and sharing of ideas, lessons and training tools. The development of the Kore Lavi gender training materials was informed by the MCFDF gender policy and gender action plan, as well as by gender trainings developed by other international organizations, such as USAID (on GBV) and CARE. Training materials from other organizations have been adapted to reflect gender issues that are relevant in the context of Kore Lavi. Training tools used include role play, pamphlets, photos, drawings and sketches that depict real life roles of men and women and the impact these have on food security, nutrition and poverty in general. The program also organizes workshops and working sessions that focus on the social perceptions of men and women, while also capitalizing on special events, such as the International Women’s Day, to sensitize the public on the implications of gender inequalities on socio-economic development and on food security and nutrition in Haiti. 5.5.7 Cascade Approach to Gender Training and Sensitization The cascade approach used by Kore Lavi for gender training and sensitization – in which the trained train and sensitize others – is, for the most part, effectively reaching intended beneficiaries and their communities. The Gender Advisor trains program staff, shares the findings of the gender analysis with them and works with them on effective ways of integrating the findings into program activities. She also collaborates with the MCFDF and MSPP to train GOH staff involved in the program, as well as with partner CHAs, CADEPs and community-based organizations (CBOs) who spearhead the cascade training and sensitization in beneficiary communities. 5.5.8 Collaboration with MCFDF Kore Lavi’s collaboration with MCFDF has been plagued with erroneous perceptions and expectations on the part of MCFDF, particularly its offices in the program departments. In some departments, MCFDF staff expected financial support from the program and others seemed to view the program as a funding mechanism for their activities. This has set the program back, because when these expectations could not be met, the MCFDF staff in several departments, with the exception of the Northwest, lost interest in working with the program meaning that collaboration between the program and the MCFDF stalled for a period of months during 2014 and 2015. To resolve the stalled relationship, MCFDF recently appointed a focal point within the Ministry who now serves as liaison between MCFDF and Kore Lavi and handles all program related communications. This is a recent change since about August 2015, and expectations are that the relation with MCFDF will stabilize going forward. Collaboration with the MCFDF, however, has been relatively seamless in the Northwest department where gender training was carried out in close collaboration with the department’s MFCDF focal point, thanks in large part to the latter’s easy going personality. Kore Lavi Title II Program Haiti – Midterm Evaluation 72 7272 5.5.9 Human Resource Issues Notwithstanding that the integration of gender issues into Kore Lavi appears to have proceeded apace, human resource issues occurring among the program’s GOH partners and internally have either slowed down progress or threaten the continuity of the program’s gender efforts. These include turnover of Kore Lavi gender advisor position in Year 2 and turnover among MCFDF staff and the absence of MCFDF representation on the program Steering Committee. In the first case, the program is now looking for its third Gender Advisor in just two and half years given that the transition from an expat position to a national position at the end of Year 1 was a planned transition. The second Gender Advisor left on the first day of the mid-term evaluation. With regards to the Steering Committee, the absence of an MCFDF representative (or more generally any gender representative) slows down the speed with which the Steering Committee can anticipate or react to gender issues, as MAST must wait to receive advice from MCFDF before taking action. 5.6 Monitoring, Evaluation, Accountability and Learning 5.6.1 Components of the MEAL System This section addresses the mid-term state of the Kore Lavi MEAL system. The scope of the evaluation here is crosscutting, and treats the MEAL system as having two primary and interrelated component parts: M&E and the knowledge and learnings systems (K&L). While the whole network is evaluated in overview, the importance of developing practical recommendations for the remainder of the program’s current funding cycle means that the analysis is focused on key systems, successes, and challenges. The underlying basis for the MEAL evaluation is that complex and diverse multiple-stakeholder programs perform best when they share collective expertise and learning. In order to do this, they need simple, clear, beneficiary-centered information systems. Simplicity and clarity allow partners to share and access valuable information as quickly and easily as possible; the fewer obstacles to the movement of information, the more collective value a diverse consortium can gain by identifying and using learnings quickly and efficiently. The underlying basis for the MEAL evaluation is that complex and diverse multiple-stakeholder programs perform best when they share collective expertise and learning. In order to do this, they need simple, clear, beneficiary-centered information systems. Simplicity and clarity allow partners to share and access valuable information as quickly and easily as possible; the fewer obstacles to the movement of information, the more collective value a diverse consortium can gain by identifying and using learnings quickly and efficiently. It is particularly important that program partners can: • Access, integrate, and use operational data and performance indicators. • Assess and improve program quality and performance. • Share and improve the consortium learning strategies. In line with these objectives, the MEAL evaluation considers the following three questions: 1. Given the program needs, is the system reasonably clear and simple? 2. Does the system provide relevant operational and performance data? 3. Does the system generate information that easily accessible to key internal and external partners? Kore Lavi Title II Program Haiti – Midterm Evaluation 73 7373 The basis for findings and conclusions related to Kore Lavi’s M&E and K&L systems included program documentation (including internal analysis of data system needs), in-country interviews with consortium partners at headquarters and regional offices, and a review of the ‘Kore Lavi web application’ (KLWA). 5.6.2 Complexity of MEAL System The complexity of Kore Lavi consortium activities presents a challenge to any central MEAL system. The program has clearly prioritized information management, has overcome early challenges, and has adopted new approaches when existing systems did not work effectively. In interviews, program staff were transparent about the challenges faced at operational level, many of which were acknowledged internally well in advance of the evaluation. There is clear institutional interest in improving knowledge and learning. In addition, individual expertise is available across partners to support this effort. Despite the diversity of consortium members, it is clear that there is a broad and sincere commitment to implementing a consortium-wide MEAL system within the program. 5.6.3 Compartmentalization of MEAL System Some of the approaches taken by the program in this area, however, appear to rely on compartmentalized and overly technical definitions of types of information collection (see Figure 13). This has led to a heavy focus of resources on external reporting requirements, which inevitably has limited the attention devoted to the internal use of data. Worryingly, data accuracy problems persist at an administrative level with the registration of SO3 beneficiaries. There is also a widespread perception among interviewees that the program has not been proactive in its information and learning dissemination. A more flexible and less reactive approach could result in a more engaged level of information sharing across the consortium. These are the areas that seem best suited for prioritization as described in the recommendations section below.20 Figure 13: Key Tracking Systems by Strategic Objective 5.6.4 Strategy Several interviewees stated that the M&E department is perceived as passive, rather than active, in seeking engagement with the program beyond collecting core IPTT indicators. A related perception is that M&E is concerned only with SO2 and SO3. It is clear, however, that this is not the intention; the program 20 In transition to ‘Kore Lavi SO3 MIS’ SO Process SO1: HDVI-specific; the reader is referred to the separate SO1 report section SO2: Beneficiary registration Paper → Excel IPTT List originates from SO1 → Exported to Digicel MIS KLWA / IPTT SO3: Beneficiary registration Paper → Excel20 IPTT Paper → Excel → KLWA / IPTT (drawn from a variety of sources) SO4: Modular competency-based systems used to measure progress towards handover Kore Lavi Title II Program Haiti – Midterm Evaluation 74 7474 has provided documentation of the strategic plans for knowledge and learning and has emphasized that knowledge should be shared across SOs and departments. In addition, SO1 and SO4 are clearly areas rich for sharing of insights and measurement systems. Notwithstanding, such perceptions are noteworthy and suggest the importance of prioritizing internal communication and relationship-building efforts across the consortium members to explain and secure feedback on existing information and learning systems and processes. 5.6.5 Indicator Frameworks While any M&E system will have reporting requirements as a primary focus, it is also important to use reporting indicators internally and have an established process for identifying and collecting additional information that can be used to measure operational efficiency, effectiveness and impact. The latter practice is its early stages, but is a clear focus of future strategic plans. This explains why some interviewees stated that IPTT indicators were used internally, and that additional indicators were collected and used for this purpose, although no interviewees could cite specific examples of indicators used for this purpose or operational decisions made as a result of their use. When asked, for example, what role M&E has played in informing program operations and decision-making, one senior manager replied as follows: “No role at all. M&E is mostly just receiving indicators to fill in the indicator performance tracking table. The information it generates is solid information related to whether the program has achieved the targets in the PM and whether it’s meeting these objectives. But it gives no information on quality or impact. To me, it is important to get information on quality and impact. For the social safety net intervention, information on impact is very important.”21 (KII with Senior Kore Lavi Manager) When asked the same question, another senior program manage responded similarly: “The monitoring and evaluation framework is detached from what is important…. It doesn’t tell me anything about how far we’ve gone. Monitoring and evaluation is a tick box; it does not ask about quality. It is not used as a management asset. There is no feedback, no regular player or presentation in management meetings from monitoring and evaluation…. There is no feedback loop.” (KII with Senior Kore Lavi Manager)22 Here, as in other areas of observation, the strategy described by Kore Lavi headquarters has differed from the observed implementation at regional level. The overall lack of clarity is of some concern, as the M&E function at Kore Lavi (or, as perceived by some partners, simply IPTT reporting) involves a high level of resource investment that includes two full-time staff at program headquarters, an M&E Officer in each department and numerous departmental field agents who are responsible for collecting data on beneficiaries. Now that the KLWA is operational at a basic level, it offers the program a prime opportunity to use time saved by that application to reallocate M&E resources to put more focus on internal learning, as opposed to external reporting through the IPTT. Further, although some of the additional M&E resources have been added in response to data quality issues, they could presumably also be used to engage partners in 21 The evaluation team understands the term ‘impact’ here to refer generally to the effect of SSN activities on outcomes such as deprivation, food security and nutrition. 22 It bears noting that the evaluation team is well aware that Kore Lavi (similar to other donor-funded programs) has important and binding external reporting requirements and that compliance with these requirements consumes a significant amount of time and resources, which may or may not leave much of a surplus of either to do M&E for internal learning purposes. However, it is precisely for this reason, and others, that developing an integrated M&E system to serve both external reporting and internal learning purposes, which takes explicit account of time and resources available, is so important. Kore Lavi Title II Program Haiti – Midterm Evaluation 75 7575 additional learning activities as well as to create a more active K&L environment across the program. There is clear interest in indicators beyond those contained in the fixed IPTT set reported to USAID.23 Numerous interviewees expressed the desire to identify a set of additional indicators that measured key program outcomes or impacts or the quality of program outcomes. The SO4 team has, for example already taken steps in this direction by developing an internal system to gauge its progress in institutionalizing SSN programming within MAST consisting of a dashboard to track who is working with whom and how, a 5-point scale to measure counterpart capacity development and transfer of capacity plans, which team members are required to complete and update. (Annex 4: SO4 Institutionalization Dashboard) Another example is the rapid field monitoring activities planned to begin during 2016 by the SO3 team. 5.6.6 Tableau du Bord Kore Lavi The managerial response to the perspectives articulated above includes identifying learning questions in the current K&L strategy document and developing the Kore Lavi operations dashboard, which does incorporate some additional indicators. (Annex 5: Operations Dashboard). Both steps are welcome. The K&L strategy, when implemented, will develop program capacity and insight. However, the dashboard does reveal the different levels of progress towards identifying internal measures of progress across specifics SOs. The Kore Lavi operations dashboard contains clear quantitative targets and focuses clearly on the quality of activity rather than the volume of activity: each component is broken down into several performance measures. The indicators used the dashboard cover a number of operational target progress, such as percentage of processes standardized and progress towards targets in infrastructure, logistics and database management. However, it is concerning that the majority of indicators are simple ‘count’ indicators (number of households in the database, number of beneficiaries, number of vendors, number of groups, etc.). This makes it a more useful tool for counting program outputs than for measuring important program outcomes. There thus remains ample scope for further developing the operations dashboard so that it reflects a clearer focus on operational improvements in quality, rather than simple monitoring of the scale of program activities. 5.6.7 Post-Distribution Monitoring Surveys In September 2014, Kore Lavi implemented a post-distribution monitoring (PDM) survey to assess the operational efficiency and impact of the program in the prevention of malnutrition. A similar follow-up survey was conducted in 2015. While the evaluation team has not seen the survey tool, it has reviewed the resulting report (‘Rapport PDM’). This report demonstrates a sophisticated critical inquiry into the impact of program operations in this area, considering beneficiary conditions and experiences in detail while giving thoughtful consideration to both environmental influences and, most importantly, the consequences of the findings for program objectives. Useful examples of this include the lifespan of food kits (10 days for 74% of beneficiaries), the incidence of kit sharing (72.8%), and awareness of the complaints process. Interviewees did not mention this very often, which may be a function of it having been prepared shortly before the November 2015 visit by the MTE team. While the evaluators cannot comment on the meaning of the findings for the SO-specific sections of this report, the approach, or process, indicates an increasingly operations-focused use of information system, and in that respect alone is very welcome. 23 The fixed nature of the IPTT indicators was reported in interviews but has not been confirmed at donor level. Interviewees also raised concern over the level of funding required to collect additional indicator data at scale. Kore Lavi Title II Program Haiti – Midterm Evaluation 76 7676 5.6.8 K&L and Information Systems As acknowledged by the Knowledge and Learning Coordinator (KLC) and other interviewees, the M&E system was prioritized in the first two years of the program, and this delayed the development of the K&L system. As noted by one senior manager, “During the early implementation stage, the processes to share lessons learned were kind of forgotten…. The project has followed a course over its first years of operation. In the first phase, it was learning how to work together. In the second phase, it was actually implementing the program. And in the third phase, which is beginning now, focus should increasingly be on improving quality and impact and institutionalizing processes to achieve and measure this.” (KII with Senior Manager) The delay in developing an internal K&L system presents a challenge, as the use of M&E data often relies on a clear knowledge management framework, and it seems to have meant that information systems have been developed separately. It is concerning, for example, that different information systems, often multiple different systems, are used for each SO, as confirmed by the January 2015 CARE Database Assessment, which noted the complications caused by separate non-centralized information systems. Similarly, several interviewees at Kore Lavi headquarters stated their opinion that internal learning is heavily compartmentalized according to department and SO. However, it is clear that there are active and well-supported efforts to ensure that there is ‘one Kore Lavi,’ and partner efforts to continue to harmonize operations at the departmental level is clearly considered a priority. Given the clear collective efforts to harmonize practices, the most concerning effect of the separate or ‘silo’ approach is the development of separate information consisting of flow systems for each SO, and, surprisingly, separate departmental beneficiary lists, usually held in excel spreadsheets. Several interviewees expressed surprise that information was still kept separately on Excel spreadsheets, a format, which has very limited information management capability, and was, throughout the evaluation process, was erroneously described as a ‘database.’ In some areas there seems to be limited awareness of the high risk of inaccuracy this format carries for complex data systems, particularly those where members join and leave on a regular basis, a more important problem in SO3, rather than the relatively stable membership of SO2. Moreover, the current SO3 information system, until very recently, had no duplicate-checking facility. These challenges, while important, are not uncommon in development-based MEAL systems. Practical and skill / capacity constraints do require common systems, and Excel is a functional and familiar stopgap system. Unfortunately, its flaws often emerge over time (e.g., as beneficiaries graduate from a program but cannot be automatically removed from a registration list), and so the software can create a false sense of accuracy and consistency. In this case, the persistence of Excel use seems to have been a feature of the early emphasis on M&E and consequential drift of the K&L system. An earlier development of the K&L system would have provided a forum and impetus to assess the processes adopted by the M&E program. It follows that this was an area where Kore Lavi headquarters told interviewees that problems were resolved when regional observation of processes suggested otherwise. The more general silo approach to developing information systems has contributed to a sense among interviewees that cross-organizational learning was not taking place to the extent that it might. The problem appears to stem principally from the absence of a system to promote cross-organizational learning. While Kore Lavi has invested substantial time and resources in developing an external reporting system (e.g., IPTT), it does not appear to have invested to this point corresponding time and resources in developing an internal learning system, including a mechanism for disseminating, processing and using Kore Lavi Title II Program Haiti – Midterm Evaluation 77 7777 information generated through its IPTT, to the extent managers and staff find this information useful for planning or decision-making purposes. The evaluation team notes, however, that this approach to K&L is common to other development programs, which tend to emphasize the external reporting (M&E) aspects of MEAL systems over internal learning (K&L) aspects. The evaluation team also acknowledges that Kore Lavi faces a unique challenge to integrate K&L systems across four disparate organizations. With that said, the program does appear to have taken some positive steps toward promoting greater cross-organizational learning. Several interviewees, for example, mentioned that Kore Lavi had recently begun holding quarterly workshops that involved senior program management along with the SO coordinators and field managers. To cite another example, the SO4 team now plans to hold monthly calls with field coordinators and field officers to discuss what going on in field, review the institutionalization dashboard and status of capacity development plans, and to share the learning derived therefrom via email or other means. As a final example, other interviewees mentioned plans for technical teams to visit each of the departments to observe what is happening on the ground and to provide an opportunity for discussion and mutual learning. 5.6.9 SO1 and SO4 These areas have freestanding SO-specific information and knowledge components, which are addressed elsewhere in this report. However, one area of SO4 warrants mention in this section. In response to the challenges of developing a handover framework (discussed elsewhere in this report), the SO4 partner group has established a module system that breaks down handover into stages of capacity. This is part of a larger effort to formalize the handover of SO4 activities, which also requires a quarterly implementation or transfer of capacity plans. Interviewees described this response as an attempt to build a stronger organizational culture, but also to address the perceived lack of engagement of M&E activities in core (or ‘important’, in the words of one interviewee) operations. The focus on substantial, clear objectives created here by the module system is a powerful tool in addressing shortfalls in measurement and program progress. With appropriate donor support, this type of practical approach to MEAL challenges could benefit the project as a whole. 5.6.10 SO2 The SO2 information systems appear to operate effectively. Vendors, food markets, and basket value, are regularly assessed. The ‘complaints’ system also appears well structured and effective. This was confirmed by department interviews. It merits noting, however, that the SO2 complaints system is limited to SO2, whereas the same or similar system has significant operational potential for SO3 as well, specifically in terms of developing opportunities for SO3 beneficiaries to provide information that could improve the operational efficiency of, say, registration and food kit distribution. 5.6.11 SO3 Several interviewees expressed concern about ongoing problems with the quality of the food kit distribution lists. Current errors include beneficiaries not appearing on the list, non-beneficiaries appearing on the list, duplications, and other errors. One key informant estimated an error rate of 25% in the early stages of the program, but current error rates appear to have fallen to around 10%. While these rates are estimates, the absence of any formal data quality assessment leaves them as the only indicator of the scale of this problem. Further, this area remains a serious concern because, as a measure of the total reach of this SO, the figure is a fundamental indicator of program outreach. Despite significant time spent examining this area, it was difficult to identify specific or consistent reasons Kore Lavi Title II Program Haiti – Midterm Evaluation 78 7878 for errors at the departmental level. While distribution is performed by WFP, interviews at department level provided no clear explanation of why department officers, who are the primary and only recipients of all SO3 registration information from CHAs, could not compile an accurate list. While the distributing party, WFP, could clearly assist in identifying beneficiaries who appeared to receive kits, it was noted that department SO3 officials, often CHAs, were also present at distribution, with their own ‘supervisor’ lists. Several times, delays in receiving distributor lists were given as explanations for delays in providing subsequent beneficiary lists (for the next month’s kits) to WFP. While distributor verification of beneficiaries who appeared to receive kits would certainly be useful to department officers, there seems no reason why departments could not create an accurate list using their own resources. It is also noted that these lists were, until very recently, being used in paper form. During the in-country interviews, the KLWA was introduced to expedite list communication, but, as noted elsewhere, this system was not yet capable of duplication checks. Discussions of this area often gave cause for concern. One interviewee mentioned the importance of withholding distribution until beneficiaries provided full information, otherwise that information would never be received. This challenge was described as operational, rather than information-based, and the suggestion of a complaints system seemed to be considered a non-transferable ‘SO2 activity.’ The identification of the beneficiaries is a primary measure of the efficacy of the information systems; if the program cannot identify whom it is supporting, it cannot identify how many beneficiaries it is supporting. If, as department interviews suggested, SO3 beneficiaries are indeed not receiving food kits until two months after registration, this appears to be a significant information system weakness. In summary, while the question of responsibility for list management is clearly unresolved among the partners, as a simple matter of distribution, there seems no reason why WFP should be responsible for list management beyond checking the receipt of food kits. Program staff or its agents are present at distributions and carry separate lists. Kore Lavi has administrative control over the registration process. Regardless of historic arrangements, WFP currently act only as distributors. As, in practice, a delivery service, it seems perfectly normal that WFP should report on the number of kits received. However, requiring WFP to somehow audit the registration of beneficiaries, when it has no access to, nor control over, the information system used by Kore Lavi, seems inappropriate. It seems similar to requiring a delivery service to report to the sender on why the delivery instructions were incomplete, or, for that matter, duplicated. Moreover, given that CARE/Kore Lavi have agents present at all distributions, and are responsible for giving notice of deliveries to beneficiaries, there seems no reason why Kore Lavi cannot manage its own beneficiary list and use WFP’s post-distribution list only to confirm the reports of program agents. Despite extensive interviews on this subject, no clear reason for current practices emerged. WFP seems to have a clear responsibility to manage and record deliveries, and in so doing verify that deliveries to the list provided were completed It would be natural to return the verified list to Kore Lavi SO3 to assist in the monitoring of registration (e.g., which beneficiaries arrived, or were notified, of distribution). However, Kore Lavi SO3, with the support of Kore Lavi information systems, should still have independent control of registration rolls independently of post-distribution verification. The practice of waiting for WFP to return post-distribution lists prior to the release of any list by ACF does seem to be a function of the limits of the existing SO3 information systems rather than any ambiguity in partner roles. Kore Lavi Title II Program Haiti – Midterm Evaluation 79 7979 6. Conclusions This section presents the main conclusions drawn from the findings reported in above related to each of the four SOs, gender and MEAL. 6.1 SO1: National Systems for Vulnerability Targeting Strengthened The reduced number of variables in the Kore Lavi proxy means testing approach (from over 40 variables under Kore Fanmi and 21 under Kore Lavi) created data collection efficiency without over-compromising accuracy and robustness. However, a few variables appear to be redundant or at least difficult to differentiate: • Education: • Absence of a least basic school • School attendance • School lag • Food Security • Hunger • Absence of food • Restricted consumption of food Both data quality problems with Kore Fanmi data and delays in funding caused in turn delays in the identification and selection of beneficiaries for Kore Lavi program components, especially food vouchers. The data collection problems that caused the delays, however, appear to be resolved. The comment from Senior MAST officials that delays in SO1 activities hurt the program’s credibility with and dampened the enthusiasm of MAST staff may be overstated. Kore Lavi staff report that the implementation of food vouchers was delayed only three or four months suggesting that MAST and other stakeholder expectations may have been too high at the outset of the program. A more appropriate conclusion may be that Kore Lavi could have more effectively communicated the cause and consequence of delays (including an initial delay in funding), and more effectively managed stakeholder expectations of the program from the beginning. Kore Lavi has demonstrated the ability to adapt and innovate to address initial challenges, including the establishment of reference desks, support for appeals committees, and the establishment of effective re￾survey and integration protocols for those households erroneously excluded initially. Reports that outside surveyors lead to overly positive survey results cannot be confirmed. However, the accompaniment of local civil society leaders (from church, teachers, etc.) with external surveyors may increase access to vulnerable households, buy-in to the survey process, and increase the willingness of households to actively participate in surveys. Although other survey approaches may create cost efficiencies over the census approach, other approaches such as kiosk-based data collection and community based pre-targeting are subject to significant exclusion and inclusion errors especially as deprived households are also the most isolated and less informed from the community. Cluster sampling offers another alternative, which offers a more cost-feasible approach Kore Lavi Title II Program Haiti – Midterm Evaluation 80 8080 relative to the census approach and, probably, less inclusion error relative to pre-targeting and kiosk-based sampling, although it does so at the cost of a higher exclusion rate. Proxy means testing, combined with the HDVI algorithm and Weighted Sum of Deprivation scoring is effective, systematic, and transparent, and should be expanded to assess vulnerability in non-Kore Lavi communes. Despite the HDVI resulting in conservative deprivation classifications, the belief that this has led to the erroneous exclusion of deprived households is not accurate. This is due to the fact that target communes have been allocated sufficient numbers of food vouchers to cover a significant number of households in the “mid-deprived” range, thus providing benefits to households that may score just outside of the “most￾deprived” classification. While the calculation of error rates based upon the number of verified appeals is reasonable, it only captures exclusion error and not inclusion error. It is not likely that better-off households who have been incorrectly deemed eligible will report their status to reference desks. As such, Kore Lavi targeting may in fact have higher inclusion errors, and thus higher overall targeting error rates, than is currently being reported. The pre-allocation of food vouchers per commune prior to vulnerability surveying and scoring indeed results in inconsistent coverage and often does not match level of deprivation in some communes. 6.2 SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased Food received by beneficiary households from both paper and electronic vouchers is inevitably shared with non-household members, including neighbors or even strangers. Such widespread sharing is a function both of the pervasive poverty and food insecurity among households in program communities, made worse by ongoing drought conditions, and Haitian cultural norms. While measuring the impact of the food vouchers on voucher recipients is outside the scope of this MTE, the evidence gathered during the MTE allows for the drawing of reasonable conclusions about their likely impact, which is that the food vouchers are unlikely to have much if any meaningful impact on either household hunger, household dietary diversity or food consumption. As a result of the widespread food sharing observed by the MTE team, it is reasonable to conclude that the food acquired by SO2 beneficiaries using either paper or electronic vouchers falls short of meeting 25% of household food consumption needs as was intended. As a result, the food lasts on average only a few days and the small portion allotted to any individual mitigates the nutritional benefit derived. The sharing of food acquired from the paper and e-vouchers does not reflect a problem with the voucher mechanism per se, which, as indicated above, is working well, nor does it reflect a fundamental flaw in the voucher program design. Rather it is an unavoidable outcome given the circumstances. Notwithstanding, given the pervasiveness of the ‘problem,’ and assuming that Kore Lavi retains the objective to improve nutritional outcomes among SO2 beneficiaries, it does require some adaptation to the voucher scheme to address it. The administration of the voucher scheme has encountered a number of ongoing problems that, while perhaps not systemic across the entire voucher scheme, occur with sufficient frequency to warrant Kore Lavi Title II Program Haiti – Midterm Evaluation 81 81 81 attention. For e-vouchers, notable problems include late payments and unreliable access to the TPago mobile network due either to weak signals or mobile phone service interruptions. Although, as noted above, Kore Lavi has a back-up plan in place to replace e-vouchers with paper vouchers if this were to become a pervasive problem. For paper vouchers, recurring problems include late or partial payment due to a failure to top off the account at the relevant bank. It should be noted, however, that Kore Lavi has anticipated this technical issue by putting in place in back up plan, which of consists of using paper vouchers for the purchase of staple foods. In fact, the program disposes a contingency stock to eventually address that issue if or when it arises. Other issues that recurred with frequency during the MTE and/or FDHS field monitoring activities were the presence of inactive vendors on the vendor list, improper and unhygienic food storage by program vendors and the failure by program vendors to wear the hats or aprons signifying their participation in the program. Despite these recurring problems, the implementation of the voucher scheme appears, on the whole, to be operating reasonably well in terms of identifying voucher recipients and enrolling them in the program, distributing food to beneficiaries via paper or electronic vouchers, recruiting and managing the network of collaborating vendors, enforcing policies governing the program and overseeing program operations. For as many people who complained about one or another aspect of the activity, there were as many other people who expressed strong satisfaction with it. The problems that are recurring are (or should be), moreover, fixable. In addition, where issues do arise, Kore Lavi has established a network of agents (e.g., ASCPs) in the relevant markets who play a key role in monitoring the implementation of the voucher scheme and dealing with issues as they arise. These agents appear to have done a reasonably good job to this point monitoring and managing the voucher implementation process, although as evidenced by the number of recurring issues, this has not prevented some problems from recurring. Judging by the evidence available to the evaluation team, the SO2 voucher activities have had no discernible impact on the local markets for staple and fresh foods, either in terms of food prices or food availability, nor did participating vendors experience any problems sourcing the fresh and staple foods sold through the voucher scheme. The volume of goods sold through the voucher scheme is too small to have any discernible effects on market supply or demand. To the extent that fluctuations in prices and food availability do occur, they are caused by the normal fluctuations that occur within the local markets. The evidence as to whether participation in the voucher scheme has affected vendors’ income is mixed. A number of vendors interviewed during the MTE said that participation had increased their income from food sales, while vendors contacted during the FDHS monitoring activities largely denied that participation in the voucher scheme had any impact on their income. There is also anecdotal evidence that participation in the voucher scheme has allowed some households to switch from imported to local rice, which is the preferred variety according to local taste, and which may benefit domestic producers, though again in a minimal manner due to the very low quantities involved. The VSLA scheme implemented in Kore Lavi communities serves a couple of important aims. One aim is to provide a mechanism by which large numbers of vulnerable women and men living in program communities can save and access small loans at reasonable interest rates to invest in their businesses or children’s education or for other purposes. The other aim is to serve as a generalized exit strategy for SSN beneficiaries enabling many (although not necessarily all) of them to transition from SSN assistance programs to more sustainable livelihood activities. To this point of the program, the VSLA schemes appears to have achieved, or is on its way to achieving, the first of these aims. Kore Lavi Title II Program Haiti – Midterm Evaluation 82 8282 The VSLAs have proven to be extremely popular with the VSLA members, most of who have never participated in any kind of savings and credit scheme before. Members are largely using the savings and loans generated through VSLA activity to invest in their small-scale businesses as well as for other household investments/expenditures, especially children’s school fees. The training received via the VSLA groups is also very popular with members and is widely viewed as highly valuable. In fact, members are hungry for more training on a variety of topics, particularly related to small business development, while the VSLA approach provides a mechanism to integrate this training into the program’s SSN activities, should the program choose to exploit it. Progress toward the second of these aims, however, lags significantly behind. Although originally conceived as 100% overlapping, the share of SSN beneficiaries remains today at only between 19%-27% of VSLA membership. The program has plans to increase the share of SSN beneficiaries to 50% by the end of the program, but there appears now to be little prospect that the share of SSN beneficiaries will ever reach the level that would qualify as a generalized exit strategy for SSN beneficiaries. This stands out as a significant project failure to date, particularly given that the VSLA was initially conceived (as the evaluation team understands it) as a complementary initiative within, and fully integrated into, the program’s broader SSN strategy, as opposed to a stand-alone initiative. The evaluation team understands that the data on vulnerability was not available in all communes at the beginning of the program, due in large part of problems with the Kore Fanmi data (see above), thus Kore Lavi made the strategic decision to expand the VSLAs beyond SO2 beneficiaries to the entire commune. Nonetheless, it appears that since then, VSLA formation has taken on a life of its own divorced from Kore Lavi’s broader programming strategy, and it is only relatively recently that the program is attempting bring operations more in line with strategy by taking steps to increase SSN membership within the VSLAs. Given that the other SSN activities being implemented by the program will end at the program’s conclusion, providing a means by which more SSN beneficiaries have opportunities to transition into more sustainable livelihood activities, by way of a generalized exit strategy, is of paramount importance. Note that in saying this, the MTE team recognizes that there will always be a portion of SSN beneficiaries (one key informant estimated from 7%-10% of the population) who are not capable of transitioning into more sustainable livelihood activities and who will thus remain perpetually dependent on government support. Thus the objective is not (or should not be) to move all SSN beneficiaries to the VSLAs or to other forms of sustainable livelihood activities but rather as many as possible given existing program resources. 6.3 SO3: Maternal and Child Nutritional Status Improved The SO3 SBCC interventions are, on the whole, well designed and well implemented. The Care Groups appear to be an effective methodology for mobilizing women and communicating critical SBCC messages. Messages are being passed down effectively, with a large share of pregnant and lactating women in the program locations being reached. Anecdotal evidence among KII and FGD respondents suggests that in certain cases key behaviors within this group of women, and in some cases even among their husbands or partners, are changing, although it was not possible within the context of the MTE to determine how widespread these changes are. The success of the SO3 interventions depends heavily on the CHAs and Lead Mothers. While they are, for the most part, satisfied with their roles in the program and appear by all accounts to be doing a good job, they also have a number of legitimate concerns related to compensation. Among the CHAs, primary Kore Lavi Title II Program Haiti – Midterm Evaluation 83 8383 concerns have to do with the low pay and lack of reimbursement for costs incurred, particularly travel and mobile phone costs. Among the Lead Mothers, primary concerns include the lack of any monetary compensation, including both salary and reimbursement for travel and mobile phone costs, or at a more basic level the lack of tangible demonstrations of appreciation from the program for the services they provide, such as gifts or in-kind compensation. Less seriously, but also of note, a number of Lead Mothers complained about the uniforms they are asked to wear. Not only are these uniforms unfashionable but they are also a source of derisive comments from community members, who associate the uniforms with salaried employment. One further area of concern is the widely disparate number of CHAs and Lead Mothers that, respectively, Nurse Supervisors and CHAs are responsible for supervising as part of Kore Lavi’s cascade training model. In some cases, the supervisory burden is excessive made even more so given the low pay and lack of expense reimbursement. The program has done a good job identifying and reaching the targeted women and infants. The program has multiple focal points on the ground – including Lead Mothers, CHAs, and CADEPs – that constitute an effective information dissemination and community mobilization team. In the context of the program’s SO3 activities, the approach of working with CADEPs to disseminate information and mobilize community members appears to be well-founded approach that is working reasonably well as far as good be observed. With that said, there are indications there remain gaps in program outreach, particularly with regards to women and infants who live in more remote and/or hard to reach locations, as well as working women who are too busy to participate in the program. Mothers participating in the Care Groups appear to be learning about better ways to feed and take care of their infants and are very appreciative for the knowledge and encouragement they are receiving from the program. Moreover, these mothers enjoy learning about such things and say that they feel much more confident than before. They would like to continue learning about anything to improve their lives, but are particularly interested in family planning and learning marketable skills. Overall, feedback from participating mothers was extremely positive. The program appears to have done effective job encouraging women to seek medical consultants for themselves and their infants at program-affiliated health centers, where by most accounts they are receiving good attention. Some areas of concern, however, were observed at a number of health clinics. These include a recurring shortage of functioning medical equipment, essential micronutrient supplements, and mother and child health monitoring cards. For MAM cases, the evaluation team also observed some issues related to a shortage of supplementary feed (e.g., Plumpy Sup), which in turn imposes significant transaction costs on parents who travel long distances to acquire it, and limited outreach to women and children living in more remote and/or hard to reach communities. Finally, the influx of demand for health center services has led to an increased work burden on existing health center staff. While the staff insists that this is not a problem at the moment, it nonetheless merits further monitoring. The food ration component is the weakest link in the program’s SO3 interventions. The distribution process itself has encountered a number of ongoing and interrelated problems related to late deliveries, long wait times, shifting distribution sites, and unsuitable and crowded distribution sites. The distribution list itself has also experienced problems related to two-month lag time required to add names to the list, which is a significant length of time relative to the timeframe of the benefits, and there are reported Kore Lavi Title II Program Haiti – Midterm Evaluation 84 8484 instances in which women are dropped from the list for no (to them at least) apparent reason. These problems are suggestive of more systemic issues related to the SO3 beneficiary list. Finally, there are reported instances of spoiled or insect-infested food, although this problem appears to be more sporadic than systemic. Perhaps the most significant problem with the food ration component, however, is that the food ration is not being used as intended to supplement the nutrition of pregnant and lactating women or children less than two years of age. Rather, the food ration is almost immediately shared with other family members, neighbors and even strangers, due to fear of physical violence, such that it lasts no more than a few days. The food ration’s nutritional benefit is thus diluted across multiple people, while its effect on the individual nutrition of targeted mothers and infants is most likely negligible. Apparently, the pervasiveness of hunger in the program communities serves to create a corresponding set of familial and social obligations to share food, and particularly food people perceive to have been received as a gift or type of windfall. As a result, the distribution of a family ration in addition to the beneficiary ration is now being reconsidered to so as to help ensure that the food rations under SO3 are reaching the target beneficiaries in the appropriate amounts. Kore Lavi is pursuing a three-pronged sustainability strategy for SO3 involving (1) replacing food rations with a coupon (similar to SO2), (2) meeting with MSPP to determine which SO3 activities can realistically be transitioned to MSPP at the end of the program and (3) transitioning program-affiliated CHAs back to the ministry at program end. Notwithstanding, MSPP currently holds dim views about the prospect for sustainably integrating SO3 activities within MAST, mostly due to budget shortfalls. 6.4 SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved MAST staff reports (in two interviews) that MAST was not involved in initial planning sessions is not accurate. From meeting minutes and quarterly reports, in addition to interviews with senior MAST officials, it is clear that Kore Lavi has actively and effectively coordinated program strategy and activities with MAST since project inception. Perceptions of Kore Lavi efforts to increase governmental capacity to lead and manage safety net programming would be improved through more regular, strategic communication of MAST-Kore Lavi coordination and implementation of joint initiatives. Kore Lavi both underestimated the capacity development needs of MAST at project inception and overestimated the commitment and capacity of MAST to make the necessary investment to procure and develop its staff resources as necessary. Contributing to existing gaps in the supply and demand for institutional support at MAST was the failure of a planned G2G institutional support mechanism to launch, which would have provided direct US government support to MAST for capacity development purposes and providing all necessary strategic level support, while Kore Lavi would cover the operational aspects focused on social safety nets. Targeted capacity assessments at the center and departmental levels are needed to better focus capacity development trainings and to establish concrete training schedules. Exceptions to this include the database management assessments of IBERS and CAS conducted by SO1 staff. The lack of financial support both from the Government of Haiti and from Kore Lavi for staff compensation and reimbursable expenses does indeed constrain the quality and intensity of MAST staff engagement in Kore Lavi Title II Program Haiti – Midterm Evaluation 85 8585 Kore Lavi activities. Significant progress has been made towards Strategic Objective 4 since mid-2015. Improved engagement between the Kore Lavi Project Management Team and senior MAST officials, increased formal support (through signing of agreements, MOUs) from government officials, the creation of a Social Accountability Manager Position, and the development of an MAST Institutional Plan suggest that Kore Lavi is taking required action to improve performance related to government capacity and institutionalization. Initial community mobilization efforts by Kore Lavi were largely effective, as program staff were able to quickly identify existing CADEPs and facilitate the establishment of new organizations (Reseaus) to participate in program implementation. Kore Lavi has been effective in increasing CADEP and other CSO knowledge and understanding of program objectives and technical skills related to community outreach, program promotion, food voucher distribution, and social accountability. In particular, CADEPs and other CSOs appear to understand Kore Lavi-promoted social accountability tools well, including the purpose and use of Community Scorecards Currently, Community Scorecards are only being utilized to evaluate Kore Lavi-sponsored activities. Community Scorecards are most effective when used to conduct 360-degree assessments of service provision and performance, including assessments of CADEPs and CSOs themselves, and the scoring of local government officials and services. While Kore Lavi has done an excellent job training CADEPs and CSOs in program-related capacities, more needs to be done to establish and reinforce six-month action plans and more training is needed related to organizational governance, financial management, proposal writing, and long-term sustainability plans. While on the one hand, evaluation team was not able cannot confirm reports of CADEPs not being sufficiently engaged in Kore Lavi activities in remote locations, on the other hand, the addition of a Social Accountability Manager is improving the quality and frequency of CADEP and other CSO capacity development efforts. Despite pressures from CADEPs and CSOs for Kore Lavi to provide budget support, Kore Lavi is striking the right balance between incentivizing involvement with community service volunteerism. Kore Lavi staff is asking the right questions regarding CBO (and to a lesser extent CADEP) motivation for program participation, namely whether participation is driven by financial motives or by a true commitment to community development and community service. The belief by many CADEPs that they are not respected as true partners is driven primarily by engagement style. Kore Lavi field staff need to be careful not to “order” CADEP members to accomplish tasks, but instead should meaningfully engage CADEP members in the development of monthly and potentially weekly work plans in order to set expectations and increase CADEP sense of partnership. Kore Lavi has correctly focused on the accurate targeting of vulnerable households in target communes and on the implementation of core safety net activities such as food voucher distribution, establishment and support of VSLAs, and improvement of mother-child health and nutrition. The engagement of CNSA to support the execution of vulnerability surveys was an excellent decision, especially given delays in survey completion from other partners. However, the contracting of CNSA staff to conduct surveys may have supplanted the broader objective of increasing the capacity of CNSA and other government agencies to respond to future food emergencies. Although CNSA was supported in other areas, such as an assessment Kore Lavi Title II Program Haiti – Midterm Evaluation 86 8686 of the 2015 Spring Campaign and to update livelihood profiles, more needs to be done to strengthen multi￾stakeholder early warning systems and contingency planning by key agencies. Progress Towards SO4 and Sustainability At the time of evaluation interviews in November 2015, Kore Lavi had only begun to gain traction with its sustainability efforts. Through the First Quarter of 2015, most of the attention of Kore Lavi staff had been focused on launching SO1, SO2, and SO3 activities. However, with the addition of Felix Veronneau, who is overseeing institutionalization (SO4) in addition to his DCOP activities, Kore Lavi institutionalization activities accelerated and began showing tangible progress. In particular, the Kore Lavi team worked with MAST to develop a common understanding of what institutionalization means in the Kore Lavi context, and began developing an Institutional Development plan for MAST. The addition of an SO4 program officer in June 2015 was also an important step to accelerate institutionalization efforts. The SO4 program officer was able to focus on community organization development needs, especially for CADEPs, and initiated the joint development of organizational development plans for partner CADEPs. Next, the Memorandum of Understanding signed in the Fourth Quarter of 2015 formalized MASTs commitment to accept increasing responsibility for Kore Lavi activities, including clearly defined roles and responsibilities. Since the signing of the MOU, and increased intensity of engagement with MAST officials, MAST has largely complied with its staffing commitments for Kore Lavi activities, something that had significantly lagged until the Fourth Quarter 2015. As of March 2016, MAST had allocated staff for all positions except for 3.6 FTE SO1 support staff and an M&E officer in the Southeast. For several positions, MAST has allocated more staff than requested, and as of March 2016, MAST have allocated a total of 35.8 FTE staff out of 40.4 FTEs agreed to with Kore Lavi. Additionally, the number of hours dedicated to Kore Lavi activities, including joint field monitoring visits, has increased steadily. The conclusion of the evaluation team is that Kore Lavi efforts on institutionalization have turned a corner and now have significant momentum that will likely enable Kore Lavi to achieve the majority of its institutionalization goals by the end of the Fourth Quarter. In March 2016, the Kore Lavi team conducted an assessment of progress towards institutionalization to identify management and resource components requiring additional technical assistance and material resource support in order to accomplish Kore Lavi institutionalization objectives by the end of the project. The team engaged with key MAST stakeholders at the Central-level and departmental offices, implementing partners, and field staff to develop “institutionalization scores” to more systematically track progress. Kore Lavi Title II Program Haiti – Midterm Evaluation 87 8787 Figure 14: “Level of Institutionalization” Scoring Matrix Kore Lavi staff utilized this scoring system to evaluate institutionalization progress across several categories and at different levels of government. Categories included: 1) staffing; 2) participation; 3) procedures and standards; 4) planning; 5) coordination; 6) monitoring and administration and; 7) materials resources. The assessment found that levels of institutionalization ranged from 1.8 (Southeast) to 2.37 (MAST Central) if material resource constraints are included. Excluding material resources, institutionalization scores ranged from 2.02 (Southeast) to 2.68 (MAST Central). While these scores are all below Kore Lavi’s objective of MAST assuming “normative responsibility” of Kore Lavi social safety net activities by the end of the project, the evaluation team concurs with Kore Lavi staff that most institutional deficiencies can be addressed through improved MAST staffing commitments, technical assistance trainings, and increased allocation of operational material resources. While the assessment served a specific purpose, it does not provide insight into the ability of the Government of Haiti to assume at least partial funding responsibility for Kore Lavi activities after the end of the project. This issue is addressed below. Figure 15: Level of Institutionalization by MAST Office,March 2016 Institutionalization Assessment Levels of Institutionalization Description 5 - Operational and normative responsibilities In charge and accountable for implemetation in addition to normative responsibility 4 - Normative responsibility Provides strategic orientation. In charge of supervision, accountable for respect of procedures and standards and quality of program deliverables. 3 - Involvemennt in supervision and technical decision-making Takes active role in supervision of program activities, draft reports for quality control. Takes part in shaping strategy and developing technical aspects of the program. 2 - Consultative role Sounding board for strategic and technical decisions 1 - Informed of main developments Informed of activities’ progress and key deliverables 0 - None Not involved in program Kore Lavi Title II Program Haiti – Midterm Evaluation 88 8888 The assessment found that while MAST is making good progress on Kore Lavi-related staffing and participation, most offices still lack managerial skills related to procedures and standards, planning, coordination, and monitoring and administration. However, the poorest scoring institutionalization category was material resources, which measured MAST’s ability to provide basic operational resources such as office furniture and equipment, vehicles, fuel and program materials. Figure 16: Average institutionalization score across MAST offices per seven criteria The evaluation team interviewed Kore Lavi Senior Management after the completion of the institutionalization assessment to better understand the project’s plan of action to address institutionalization shortfalls in the second half of the project. Senior Management has developed a targeted training and technical assistance plan to improve the managerial capacity of MAST staff (categories with yellow columns), and to facilitate increased material resources within project budget constraints. While material resource needs are not likely to be fully met by Kore Lavi (not even including resource transfer funding), the evaluation team concurs that significant progress can be made regarding managerial skills and capacities through targeted trainings. The evaluation team asserts that an overall institutional score between 3 and 4 for MAST is likely by the end of the project, and the potential to increase institutional capacity to the level of normative responsibility (score of 4) is plausible project’s end. See Section 7.4 for MTE recommendations to improve sustainability of Kore Lavi activities and MAST’s institutional capacity. An additional project year would greatly increase the probability of Kore Lavi successfully facilitating MASTs ability to assume normative responsibility of Kore Lavi social safety net activities. 6.5 Gender Kore Lavi has taken a holistic approach to gender integration from design to implementation and has made a conscious and good faith effort to integrate gender considerations in each of the four SOs examined for this evaluation. Kore Lavi’s holistic approach to gender stands out as a particular strength of the program and is contributing to the achievement of more gender focused results. As made evident by findings of the MTE discussed above, gender considerations were well integrated into the Kore Lavi program from design to implementation. The design and implementation have been well Kore Lavi Title II Program Haiti – Midterm Evaluation 89 8989 informed by gender needs and priorities of Haiti, particularly, those of the targeted beneficiary communities identified by the baseline survey and more particularly, findings of a gender analysis conducted by CARE in close collaboration with MCFDF, as well as recommendations provided by MCFDF. The integration of gender was achieved through two inter-related strategies: (1) gender equality and GBV training and (2) the integration of gender equality promoting and women’s empowerment activities in almost all components of the program. Both strategies appear to have been very effective. The gender equality and GBV training using the cascade training and sensitization approach has been an effective way of getting the word out on the benefits of gender equality to the society and to household food security and nutrition in particular. The integration of gender equality and women’s empowerment promoting activities into program components, particularly the two social safety net components of Kore Lavi, effectively promoted program gains to women and their households. The VSLAs are another effective mechanism to promote women’s economic and social empowerment. There exists already a large literature on the gender impacts of community-based savings and credit associations, which point to a number of positive gender impacts of participation.24 While this evaluation is not intended to add to this literature, the women VSLA members participating in the MTE uniformly perceived positive benefits from VSLA membership. In addition to the other benefits VSLA membership might provide, one in particular that stood out was the opportunities it created for women to assume leadership positions. Notwithstanding, in light of the program requirement that 70% of VSLA members must be women, women are actually underrepresented in VSLA leadership, particularly in the higher-level leadership positions of President and Secretary. Due in part to the program’s efforts, Women are also participating actively in CADEPs and, have ascended to leadership positions on the CADEPs’ Board of Directors, including in several cases senior Board positions, such as Secretary General, President or Treasurer. The program did a good job bringing its GOH gender counterpart into the program at an early enough stage to ensure their active involvement and buy-in. This is well demonstrated by the involvement of MCFDF and final endorsement of the gender related recommendations, along with gender training resources, which were developed in close collaboration with the Kore Lavi team. The close collaboration with MCFDF, moreover, both brings in local knowledge expertise on gender in Haiti and promotes institutional participation and buy-in, both of which are integral to any eventual takeover by the GOH and to the sustainability of Kore Lavi’s contribution to gender sensitive programming in Haiti. With that said, the program’s collaboration with MCFDF has experienced some setbacks owing to erroneous perceptions and expectations among certain MCFDF staff, particularly around the issue of funding. While this led to a temporarily stall in the relationship, MCDFD has recently taken steps to reinvigorate the relationship by appointing a new focal person within the Ministry. Insufficient time has passed, however, to determine what impact this new focal person will have. Finally, the absence of an MCFDC representative on the program Steering Committee has introduced certain inefficiencies in how the Steering Committee and program can anticipate or react to gender issues. Given the program’s obvious commitment to gender issues, the absence of a woman’s advocate on the Steering Committee stands out as a curious omission. 24 See, for example, http://www.microfinancegateway.org/. Kore Lavi Title II Program Haiti – Midterm Evaluation 90 9090 6.6 Monitoring, Evaluation, Accountability and Learning Kore Lavi has prioritized information management and has demonstrated a clear institutional interest in improving knowledge and learning. Some of the approaches taken by the program in this area, however, appear to rely on compartmentalized and overly technical definitions of information leading to a heavy focus on external reporting and limiting the attention devoted to the internal use of data. Also, data accuracy problems persist at an administrative level with the registration of SO3 beneficiaries. A more flexible and less reactive approach could result in a more engaged level of information sharing across the consortium. The MEAL system is passive, not active, in seeking engagement with the program beyond collecting core IPTT indicators and is overly concerned with SO2 and SO3. While these perceptions may not be strictly accurate, they are important and suggest the importance of prioritizing internal communication and relationship-building efforts across the consortium members to explain and secure feedback on existing information and learning systems and processes. Although ostensibly indicators of activity, if not impact, Kore Lavi IPTT indicators are used overwhelmingly for external reporting and are not used for internal decision-making or planning purposes. The Kore Lavi Web Application, together with the resources added in response to data quality issues, offer the program a good opportunity to reallocate M&E assets for internal learning purposes. Another method to put more focus on internal learning is to include additional indicators to the IPTT, or to create a parallel system of internally tracked learning indicators. An example includes the 5-point scale developed by the SO4 team to measure counterpart capacity development and the transfer of capacity plans, which each member of the SO4 team is required to complete and update. Yet another method is to conduct rapid assessment exercises, such as those currently planned by the SO3 team to monitor the program’s SO3 activities. Rapid assessment activities may use either quantitative or qualitative methods – such as mini-surveys, lot quality assurance surveys (LQAS), FGDs (the SO3 team plans to use primarily qualitative methods) – but the general idea is to send a team (e.g., 1-3 persons) into an area to conduct a day or two’s worth of rapid investigations to answer a a small set of specific evaluation questions with high management utility. While such rapid assessment methods may lack statistical rigor (except for LQAS, which is designed to be statistically rigorous), they compensate for this by their speed, flexibility, low cost and emphasis on management utility. The M&E system was prioritized in the first two years of the program, and this delayed the development of the K&L system. The delay in developing an internal K&L system has meant in practice that information systems have been developed separately for each SO. Historical consequences of this general ‘silo’ approach include: (1) different information systems and beneficiary lists maintained for each SO, (2) internal learning compartmentalized according to department and SO, (3) increased risk of inaccuracy absent standardized (or standardizable) information systems, and (4) increased difficulty in creating cross-organizational learning platforms. Notwithstanding, the program does appear to have taken positive steps toward promoting greater cross￾organizational learning, quarterly workshops for senior program, SO coordinators and field managers; monthly calls with field coordinators and field officers among the SO4 team; plans for technical teams to visit each of the program departments. These processes, if used to address measures of operational quality and efficiency, rather than simple output or scale of activity, should help the program to make good progress in the remainder of the current funding cycle. Kore Lavi Title II Program Haiti – Midterm Evaluation 91 91 91 7. Recommendations This section presents a set of recommendations pertaining to each of Kore Lavi’s four SOs, gender and MEAL drawing on the findings and conclusions presented above. 7.1 SO1: National Systems for Vulnerability Targeting Strengthened Key recommendations by order of priority: 1. Until the MAST staffing issue can be addressed, Kore Lavi should continue and perhaps expand the use of high-energy interns to facilitate the continued development and operations of the Kore Lavi information system. Alternatively, Kore Lavi should increase its advocacy efforts to facilitate increased funding for Kore Lavi-related positions. 2. Consider subsidizing MAST contracting of qualified IT staff on a decreasing scale. Service contracts will allow MAST to adequately compensate qualified IT personnel for management of the Kore Lavi information system, as service contracts are not subject to MAST salary scales. By subsidizing service contracts on a decreasing scale (starting at 75%, reducing down to 25% by the end of the project), Kore Lavi can incentivize MAST to recruit and pay qualified staff competitively, and to gradually increase budget allocations for longer-term sustainability. 3. Develop and fund an alternate compensation scale for MAST information management employees, in full collaboration with MAST officials. The higher-level skill set required for information management, and importance of successfully institutionalization information management activities, justify the higher wage scale. Given potential restrictions to the direct funding of civil servants by USAID, Kore Lavi should consider alternative incentive mechanisms, such as facilitating the establishment of a social protection performance fund, that could be managed as a stand-alone public-private partnership utilizing GOH and USAID funding. 4. Engage local civil society leaders in the data collection process. Community leaders such as church members, teachers, members of community-based organizations can accompany external CNSA surveyors to improve responsiveness, facilitate identification of more remote households, and potentially to increase data accuracy. 5. A more robust assessment of targeting error is needed, including greater frequency of beneficiary verification reports. In particular, the lack of mechanisms to identify inclusion errors needs to be addressed. 6. Move forward with data collection pilots currently under consideration to test the efficacy of alternative data collection and household identification approaches. These include testing kiosk￾based self-reporting of households that are subsequently verified by Kore Lavi vulnerability surveys, and “pre-targeting” by community leaders and cluster sampling. The high cost of the census approach makes it likely that one of the above three (or other) approaches will become necessary. 7. Bring together key HDVI stakeholders to reach agreement on basic concepts related to definitions and methods in implementing the HDVI. Preferably, this effort should be led by MAST, with possible facilitation by Kore Lavi. 8. USAID and the Government of Haiti should explore options to expand the Kore Lavi targeting methodology to other regions of the country. Kore Lavi Title II Program Haiti – Midterm Evaluation 92 9292 9. USAID should promote the utilization of similar Multidimensional Deprivation Indexes (ranking based on HDVI and Weighted Sum of Deprivation) in future food assistance programs. 10. Food voucher allocation should be flexible to better match need in specific communes. While pre￾allocation may have been necessary for budgeting and call-forward purposes, the lack of flexibility in the pre-allocation approach is creating unnecessary coverage imbalances. 11. Secure adequate resources to continue the engagement of the information system management consultant (Thomas Gabrielle) beyond September 2016 to support the full adoption, ownership and operation of the Kore Lavi information system. Funding can be facilitated by adjusting the Kore Lavi budget or by advocating for funding by MAST/GOH. 7.2 SO2: Access of Extremely Vulnerable Households to Local and Nutritious Foods Increased Key recommendations by order of priority: 1. Evaluate options for improving the effectiveness of the voucher activities: The widespread sharing of food procured through the voucher activities undermines the impact of this activity on beneficiary nutrition. It would be very useful to evaluate options that would mitigate this. These options may include limiting voucher validity to certain items that would be less likely to be shared, or limiting the use to specific items that could be stored and consumed over a longer period of time, such as beans or grains. 2. Conduct a technical audit of the voucher activities: Technical problems related to the voucher activities do not appear to be systemic. Therefore, a technical audit of recurring issues related to this activity should be useful in improving its implementation. 3. Increase, and if possible, accelerate, the integration of the VSLA scheme into SSN activities under SO2, particularly by increasing the absolute number and relative share of SO2 SSN beneficiaries in VSLAs. In making this recommendation, the MTE team acknowledges the circumstances that motivated Kore Lavi to begin its VSLA activities prior to launching its other SSN activities under SO2 and that Kore Lavi is aware of this issue and is addressing it. 4. Consider conducting field research to quantify the extent to which food rations are shared, with whom and how long they last. Although reports of food sharing were common during the MTE, robust quantitative data on this phenomenon remains lacking. A similar recommendation applies to sharing of food rations received by SO3 beneficiaries. 7.3 SO3: Maternal and Child Nutritional Status Improved Key recommendations by order of priority: 1. Investigate issues with SO3 beneficiary targeting immediately, and develop a rapid action plan to address problems identified in the targeting analysis. 2. Add or strengthen post-natal planning options, especially for men. Lead Fathers so that they would be equipped to handle what might be a sensitive area of health education. SO3 beneficiaries and community health agents recommended that Kore Lavi expand its health messaging to include post- Kore Lavi Title II Program Haiti – Midterm Evaluation 93 9393 natal family planning options and advantages, especially to men. They felt that men over time have become more and more receptive to the health and nutrition messaging around infants and toddlers delivered by the Kore Lavi program, and they appreciate the expanded knowledge that the program is giving them. This appreciation opens the door to expanding health messaging in more sensitive areas such as family planning. Beneficiaries felt that men need more information, awareness and discussion. They said that many men feel that practicing birth control is somehow evil, but more often the men simply lack awareness or a sense of responsibility for pregnancies. The women would like more control over their reproductive lives, but they said they had to get their husbands and partners on board to make it work. Thus, they were recommending a “men’s focus” to family planning education, and it could be delivered, in part, through Lead Mothers and Fathers to individual families. This technical area would require careful training of Lead Mothers and Lead Fathers so that they would be equipped to handle what might be a sensitive area of health education. 3. Take advantage of existing Care groups to provide training on any subject. When asked what further training they might like, SO3 beneficiaries and Lead Mothers said “anything.” They suggested more health education on any health subject, water and sanitation, child growth and development, literacy (as many of them are functionally illiterate), math, income generation, business development, agriculture and animal husbandry. In short, they expressed a hunger for knowledge to improve their lives. The Kore Lavi project might consider adding a Life Skills course, the PHAST (Participatory Hygiene and Sanitation Transformation) methodology for learning about water, sanitation and hygiene (this program does not require any literacy from participants), and getting more beneficiaries into VSLA groups. There are many learning resources in the development community on a variety of subjects suitable for illiterate and semi-literate audiences. Usually such programs require a cascade of facilitator training similar to training of CARE groups for PM2A. Taking this on would mean the need for additional funding for Kore Lavi as well as an expansion of project objectives. However, this desire among beneficiaries is worth mentioning, as it shows not only how well Kore Lavi is appreciated by women and has inspired them, but how well the CARE Group model has worked in this context and its potential for continuing to improve women’s lives. 4. As an alternative to the above, link Care Groups to VSLAs for training or to integrate Care Group members into VSLA membership. More generally, Kore Lavi should investigate this and other opportunities for increasing integration between the different program SOs. 5. Conduct a study of workload of CHAs and SO3 Nurse Supervisors. The mid-term evaluation finding on the highly variable workloads of community health agents and SO3 Nurse Supervisors suggests that the Kore Lavi team should gather more information in this area. A place to start would be with the 22 Nurse Supervisors who oversee the work of the community health agents. The Kore Lavi program should reaffirm that the workloads of Supervisors and community health agents are logical and consistent with criteria such as geographic size, road conditions, and population density. In places where there are not enough community health agents, such that Nurse Supervisors are training and meeting with high numbers of Lead Mothers, the program should ensure that the Supervisor responsibilities are consistent with the job description and expected workload. 6. Consider adding CHAs and Supervisors to reach outlying areas or potential beneficiaries now missed. The MTE found that the Kore Lavi program does not in fact reach every woman pregnant, breastfeeding or having a child under 24 months in a catchment area. In some areas women are not enrolled because they choose not to be. Some work away from home and others simply do not want to participate. However, health centers reported cases of MAM coming from distant outlying areas where the program does not reach because it is too far for community health agents to go there. Kore Lavi Title II Program Haiti – Midterm Evaluation 94 9494 The program might consider how these outlying areas might be reached. Nurse Supervisors are given motorcycles, so they have transportation to reach some distant families if time permits. Community health agents could be given special incentives, as well as transportation funds, to identify and visit such families. When a qualifying family visits a health center because of MAM or any other reason, and it is discovered they are not in the Kore Lavi program, despite the program’s existence in that area, those families could be added for private monthly visits and perhaps food rations could be taken to them, if they are too far away to collect them. It may be that there are very few of these families and so they may be accommodated into the program with only a little extra effort. 7. Consider further compensation for CHAs and Lead Mothers, including, at a minimum, reimbursement for job-related transportation and mobile phone expenses. As mentioned in the report, community health agents repeatedly expressed a need for transportation funds and mobile phone expenses. They were very grateful to have jobs and felt their work was meaningful, but they felt that program expenses should not come from their salaries. The program might do a small study of actual program costs to community health agents to determine whether this is a valid concern. Even more than mentioning themselves, community health agents pleaded for some compensation for Lead Mothers. Programs that depend on volunteers at the community level frequently face this issue of compensation for volunteers who work at the bottom of the pyramid. The workload of Lead Mothers does not appear to be heavy or particularly time consuming, but they are key to the success of the model. The program should carry out a few focus group discussions with community health agents and with Lead Mothers on this issue to determine how important this really is to the success of the program and what they would perceive a fair compensation to be. Kitchen sets were mentioned as successful gifts to Lead Mothers in some areas, as it seems important in the culture for a woman to have such a set before she can get married. Perhaps one year of service would merit one kitchen set. Perhaps a second year of service would merit something else of value for the home. It is important that Lead Mothers are all given exactly the same compensation for periods of service. Program areas that gave a variety of different gifts to Lead Mothers said this did not work well. To the extent that this recommendation is acted on, it should be done so in coordination with the MSPP so as to ensure that such efforts are continued after the program. 8. Contact health centers to determine whether enough supplementary foods exist for all cases of MAM. Health Centers visited reported having a shortage of Plumpy Nut, also called Plumpy Sup, for their clients. They said that the supply had been reduced in recent months. Kore Lavi should follow up to find out whether this has occurred and whether it is working to the detriment of the program. In addition, it would seem an opportune time to evaluate the other special nutritional needs or antibiotic needs of MAM infants and whether the health centers have sufficient stock of these. Health centers mentioned that MAM infants usually come from the very poorest of the poor families and that such families cannot afford to buy prescriptions. MAM infants frequently have more than one health issue as a result of their malnutrition. 9. Follow-up on which areas are giving gifts, what the gifts are, and whether they are making a positive difference in the performance of Lead Mothers, as the success of this program hinges on the performance of the Lead Mothers. If the provision of training and information ends at the level of the health center and CHAs, it will not, in the evaluation team’s opinion, have the intended impact. The design of having Lead Mothers at the community level is a particular strength of this program, and Lead Mothers need to feel rewarded for their contributions.] 10. Consider implementing the family ration to supplement the beneficiary ration. It is clear that the current ration is not sufficient and thus will not move the needle in terms of Kore Lavi’s core Kore Lavi Title II Program Haiti – Midterm Evaluation 95 9595 objective to improve nutritional outcomes of SO3 beneficiaries. While there is no guarantee that the family ration will solve this problem (e.g., most of the family ration may end up being shared as well thus minimizing any incremental benefit nutritional benefit from it), short of another solution to address this problem, the family ration appears to merit further investigation. Implementing a family ration would seem to offer an excellent opportunity for a limited, and closely monitored, field test to determine its incremental nutritional impact. Only once its benefits have been confirmed, should the program move to a more generalized rollout. With regards to the concern about possible perverse effects of the family ration, the evaluation team is not convinced, absent more systematic evidence, that the potential, and short-term, receipt of an additional 11.7 kilograms of food per month exerts sufficiently strong incentives to encourage women to have more children, which implies a long￾term resource commitment that far outstrips any temporary advantage gained. In any case, this too should be monitored closely during any field test of the family ration. The evaluation team sees little downside to a controlled field test of the family ration, should Kore Lavi determine that this offers a potentially effective way around the food sharing problem. 11. Consider conducting field research to quantify the extent to which food rations are shared, with whom and how long they last. Although reports of food sharing were common during the MTE, robust quantitative data on this phenomenon remains lacking. A similar recommendation applies to sharing of food procured using paper or e-vouchers. 7.4 SO4: Haitian Institutions’ Capacity to Effectively Lead and Manage Safety Net Programming Improved Key recommendations by order of priority: 1. Conduct detailed capacity gap assessments at both the center- and departmental-levels, and develop concrete capacity development plans in order to increase the frequency and quality of trainings and to appropriately set staff expectations of Kore Lavi capacity development activities. 2. Implement targeted MAST institutional capacity trainings, especially at department levels. 3. Facilitate MAST advocacy for adequate budget support from National Government 4. USAID can facilitate increased financial resources from other donors by establishing a multi-donor platform for social safety net funding 5. USAID should conduct direct advocacy for increased GoH funding and funding from other donors for social safety-net programming. 6. USAID should accelerate the approval of the previous planned government-to-government technical assistance initiative to provide technical and financial resources for the institutional capacity development of MAST. 7. Consider adding manager-level staff to facilitate institutionalization of other Kore Lavi program components beyond the addition of the SO4 Social Accountability Manager. 8. Consider creating a second Social Accountability Manager so as to cover the full complement of SO4 initiatives. One manager could be tasked with the capacity development of CADEPs and a second manager could oversee the establishment and operations of reference desks and other feedback mechanisms. Kore Lavi Title II Program Haiti – Midterm Evaluation 96 9696 9. Consider a more open sharing of budget envelopes in order to manage MAST expectations and increase MAST confidence in Kore Lavi statements related to limited financial resources. 10. Work with USAID and the Government of Haiti to increase financial resources available for staff compensation and to increase the frequency and quality of MAST field visits. These resources need to be included in GOH and MAST budgets. 11. Assuming MAST retains the objective to transition Kore Lavi’s SSN programming under its own umbrella, it will be necessary to include in the GOH and MAST budget the financial resources to increase staff compensation (especially related to information management) and the frequency of field visits, while simultaneously improving its own operational efficiencies so that it can devote more resources to this task. 12. Other actions MAST might take to improve the effectiveness of its SSN activities are to provide some form of financial or non-financial incentives to CADEPs, such as adopting a fee-for-service model or to provide small incentive packages for high-performing CADEP members and/or CADEPs, such as note books, umbrellas, rain boots, etc. MAST’s ability to pay for these recommendations likewise depends on whether it can secure the requisite financing and/or improve its operational efficiencies to free up resources. 13. Reinforce the joint-development of six-month action plans for CADEPs and other CSOs with a greater focus on organizational development objectives and actions. 14. Expanded Community Scorecard implementation to include 360 assessments of all Kore Lavi stakeholders, including the CADEPs and local governments. 15. Implement frequent planning sessions with CADEPs to develop monthly work plans to increase CADEP ownership of activities and to set expectations appropriately. 16. Increase program focus and effort related to government capacity to respond to food emergencies. The program should revisit early warning and emergency response activities from the original program proposal. • Monitoring food security through CNSA Observatories: These activities include regular update on the food security situation at department level, based on qualitative observations made by CNSA Observatories at department level. These activities are currently supported mainly by FEWS-NET. • Monitoring the nutrition security by MSPP: These activities include regular update of growth monitoring data collected from nutrition centers and rally post, which are then aggregated at department and national levels by MSPP. Sentinel sites for nutrition monitoring are also being established under the leadership of MSPP. • Post-disasters assessments: These activities include Emergency Food and Nutrition Security Assessment (EFSA) immediately after a disaster. It also includes updating Vulnerability Maps using all relevant data and field observations available to help targeting assistance to most vulnerable communes. • Analysis of the Food Security situation and publications: Based on the food security information and data collated in the 4 components above, CNSA is leading the food security analysis with the support of its partners. An ‘analysis committee’ is to be created under the GTSAN to support a more comprehensive and participatory analysis approach. A main output of this component Kore Lavi Title II Program Haiti – Midterm Evaluation 97 9797 is to classify causes and severity of food insecurity by communes. A multi-variable vulnerability map is regularly updated. CNSA also publishes regularly bulletins and when necessary alerts on the food security situation. 7.5 Gender • Consider increasing the recruitment of Lead Fathers either in lieu of or to complement Lead Mothers. Lead fathers may have a potential to become agents of change. Their potential for this was echoed by all SO3 beneficiaries (men and women) and also by the SO3 IP and front-line staff. • Consider including a select number of Lead Mothers and Lead Fathers from across the different target departments in the pool of gender champions proposed in the gender strategy, so that they can benefit from the in-depth gender training and serve as champions in their respective communities • Scaling up the successful gender equality promoting initiatives, such as the theatre initiative developed by CADEPs in the Center Department, will need a participatory approach, starting with brainstorming sessions involving the initiators, KL staff, and CADEPs from other departments. The program should also explore ways of scaling up the use of songs as a tool for promoting gender equality. One such strategy would be to partner with popular Haitian Singers to promote the gender equality focused songs. Center and Artibonite departments, where there is active use of songs to pass-on gender messages or memorize important MCHN lessons, might, for example, serve as pilots. • Consider creating positions of gender focal points or gender champions on the ground in the different departments to monitor and follow-up on gender sensitization, including CADEP-delivered trainings, and reinforce the program’s context specific gender integration efforts across the different target communities. • Consider strengthening the gender team to mitigate the occasional voids created by the rapid turnover of the Kore Lavi Gender advisor position, by say, moving it from a one-person to a small gender team made up of an advisor and one or two assistants, budget permitting. • Consider including MCFDF on the Kore Lavi Steering Committee so as to facilitate their direct technical advice to the committee and speed up the interaction between the program and its other GOH partners. • Make gender training a priority for all program and local government partner staff not yet trained. • Consider expanding gender and GBV training and sensitization to law enforcement officers, schools and health centers in areas like the Center department where the incidence of rape and other aspects of GBV is reported to be much higher than elsewhere in the country. • To effect lasting change in gender norms, perceptions and practices, there is need to think outside the box. This is, for instance, the case with Kore Lavi’s approach of tackling gender issues that affect food security and nutrition from multiple levels/angles: • At the individual level through the training of all staff and volunteers, • At the household level through the care group network of Lead Mothers and fathers; • At the level of associations through introduction of community gender dialogue within VSLAs, • At the level of local partners (GOH, CADEPs and other CBOs) through training/sensitization and engagement with local gender experts (MCFDF) through close collaboration, learning and experience sharing. Kore Lavi Title II Program Haiti – Midterm Evaluation 98 9898 7.6 Monitoring, Evaluation, Accountability and Learning • The existing strategy for M&E/K&L development requires active implementation in order to ensure that processes across Kore Lavi are consistent and that all partners feel engaged in sharing and learning. A small cross-partner learning and knowledge group should be formed to: • Ensure the engagement of all partners at regional and national level in learning activities; • Regularly and actively solicit feedback on M&E and K&L activities, including internal communications; • Identify a core group of program-wide indicators of impact and operational efficiency in order to assess program efficiency, time/cost savings, and beneficiary service improvement, and • Develop small-scale pilots of flexible, agile methods of collecting operational data under those indicators, including and prioritizing: i. Asking field operatives and KL officers and agents to identify time savings generated by the transition to tablet use (described below) and productive ways to use the time saved, and ii. An assessment of ways to adapt the SO2 ‘Complaints’ system to SO3 operations, in order to provide feedback on service quality and efficiency, particularly registration list accuracy. • Acknowledge prime responsibility for beneficiary distribution list accuracy, relying on WFP only to confirm delivery lists within, say, 72 hours of field delivery. The program should use the field agents who are currently present at distribution to confirm beneficiary attendance and receipt. The WFP list can then be used to verify Kore Lavi data, rather than as the origination of the distribution list. If not currently the case, WFP should be given enough access to the Kore Lavi SO3 MIS to allow them to draw distribution lists within efficient time frames. • The program-wide extension of the Center Department tablet program should be prioritized and used as an opportunity to review M&E and K&L across all SOs using the forums established by the KL program management. As mentioned above, the cross-partner learning group should adopt this exercise as an opportunity to identify and incorporate field and KL officer/agent insight into time savings created by tablet use and beneficial opportunities to reallocate that time. These combined activities should help accelerate program learning and program learning practices. • The SO3 beneficiary list is a fundamental data source, and the program should prioritize a complete check of SO3 beneficiary lists for accuracy, including duplications. Current estimated error rates of 10% cannot be verified, nor remedied, without a district-level review. If the current MIS can be developed to effectively check for duplicates, including name spelling variants and related errors, then this is preferable to a manual review. However, there was no duplicate checking capacity at the time of the project visit, and the basic veracity of enrollment figures is a critical project indicator. Therefore, if necessary, a manual review should be completed as a priority. Thereafter, the project should prioritize basic accuracy-checking tools for all current MIS. • Prioritize the transition to web-based MIS where Excel spreadsheets are currently used. There is significant opportunity to use current data challenges to build MIS capacity across the program. Internal training should be given in order to explain the constraints of excel-based data storage and familiarize staff across the program with the strengths of basic MIS systems, particularly record management and basic data quality verification. This should contribute to the engagement component of the strategic recommendation above. • Consider creating a complaints system for SO3 activities similar to the one existing for SO2 activities. Such a system would have potentially important operational benefits, specifically in terms of developing opportunities for SO3 beneficiaries to provide information that could improve the operational efficiency of, say, registration and food kit distribution. Kore Lavi Title II Program Haiti – Midterm Evaluation 99 9999 • Prioritize developing the capacity of the KLWA to perform duplication checks. • SO teams should be encouraged to develop internal operational quality and efficiency indicators to measure achievement of key SO outcomes or impacts, in addition to those currently recorded in the operational KL dashboard discussed above. These indicators may be integrated into the IPTT system, or they may be tracked and reported internally, depending on donor requirements. (It is not necessary to report externally all performance indicators tracked internally.) Given that these indicators would be tracked primarily for learning purposes, as opposed to comply with external reporting requirements, the program has much greater flexibility in how these indicators are defined and measured, depending on what the management and operational teams find useful. They might be, for example, qualitative in nature and/or or they might be collected using rapid assessment methods. The key consideration in developing such learning indicators, and the methods used to collect them, is their management and operational utility. It is also important, however, to make sure that a system for collecting these indicators is either in place or can be put in place without putting excessive strain on staff or implementing partner resources, whether this involves the program’s M&E staff, departmental field teams or implementing partners (e.g., CHAs or CADEPs). • In addition to, or in lieu of, developing new, internal IPTT indicators, Kore Lavi is encouraged to plan and conduct on-going rapid assessment monitoring activities, similar to what the SO3 team is currently planning. Such rapid assessments would use a mix of qualitative and quantitative assessment methods, including, for example, FGDs, mini-surveys (short surveys with relatively few questions and administered to a relatively small group of people) or LQAS methods. LQAS offers particular potential for generating statistically rigorous information on the effectiveness of different interventions using small samples.)25 The MTE team understands that Kore Lavi’s existing resources may not allow it to undertake such rapid assessment activities. Thus, as is the case with adding IPTT indicators, undertaking any new data collection activities should be done only where existing resources allow it, and where they don’t allow it, Kore Lavi should consider the feasibility and desirability and feasibility of adding or re-allocating resources for this purpose. As a final observation here, it is easy to underestimate the incremental burden that additional data collection responsibilities put on program or implementing partner staff, and requiring them to do more than is reasonable or feasible, given their existing time and other constraints, is more likely to result in the breakdown of the system than the generation of useful learning. • Continue with and accelerate efforts to create opportunities for cross-organizational learning. There are multiple ways in which it can create such opportunities, for example workshops, cross-team field visits, newsletters, brown bag events and so forth. The important point here, however, is that whichever methods are used must be systematized with strong and visible management support. This implies in turn the need to develop a formal knowledge and learning strategy that incorporates all aspects of the program’s MEAL system. In this context, it is recommended that Kore Lavi should continue to develop the program dashboard by using the Institutionalization measures of program efficacy as a model for assessing operational efficacy. 25 The FDHS field monitoring activities claim to use LQAS methods, but in truth they do not. For LQAS to serve its purpose – generate statistically rigorous results using small samples – the LQAS methodology must be followed rigorously, which the FDHS field monitoring does not do. Kore Lavi Title II Program Haiti – Midterm Evaluation 100 100 100 Annex 1: Scope of Work (SOW) STATEMENT OF WORK (SOW) Mid-Term Evaluation (MTE) of “Kore Lavi” (Title II Development Food Assistance Program) Support to the National Program of Food Security and Nutrition A. Program description Program name: Kore Lavi - Support to the National Program of Food Security and Nutrition Program location: Department of South-East: 7 communes: Anse à Pitre, Belle-Anse, Grand Gosier, LaValle, Cote de Fer, Thiotte, Cayes-Jacmel Center department: 6 communes: Thomassique, CercaCarvajal, Cerca La Source, Thomonde, Hinche, Boucan Carré. Department of North-west: 6 communes: Baie de Henne, Bombardopolis, Mole Saint Nicolas, Jean Rabel, Bassin Bleu, Port-de-Paix. Department of Artibonite: 3 communes: Gonaïves, Anse-Rouge, Terre-Neuve Department of West (La Gonave): 2 communes: Anse-à-Galets, Pointe-à-Raquette. Program budget: US$ 79,996,200 Program start date: August 12, 2013 Program end date: September 30, 2017 Implementing agency and partners: CARE- Signatory of the cooperative agreement, Lead of the Consortium, Technical Lead of the Safety Net (Food Voucher) component; Action Contre La Faim International (ACF) - Member of the Kore Lavi Consortium, Technical Lead of the MCHN component; World Food Program (WFP) - Member of the Kore Lavi Consortium, Technical Lead on Institutionalization and Commodity Management and World Vision International (WV) - Member of the Kore Lavi Consortium, implementer of the Safety Net (Food Voucher) component in La Gonave island Donor: U.S. Agency for International Development’s (USAID) Office of Food for Peace (FFP) Evaluation type: Mid-Term Evaluation Kore Lavi Title II Program Haiti – Midterm Evaluation 101 101 101 The Program background Haiti is one of the poorest countries in the World, ranked 145 out of 169 countries on the United Nations (U.N.) 2010 Human Development Index (HDI). This ranking has seen little change over the past several year and Haiti still remains among the countries with an extremely high level of food insecurity and malnutrition. Moreover, the high incidence of climatic shocks, other hazards and the government›s limited capacities to mitigate the negative impacts of these shocks further impedes the country›s ability to gain meaningful development ground. As a result, a majority of Haitian population still faces significant challenges in all four pillars of food security: availability, access, utilization and stability of food and still stays at the high poverty level. Purpose of the program The Kore Lavi program, funded through Food for Peace (FFP) Title II Development Food Assistance Programming, is operating from August 2013 to September 2017. The overall goal of the Program is to strengthen National social safety net and improve food security and nutritional status of vulnerable populations. Kore Lavi attempts to: a)Address the urgent needs of the extremely poor households in five departments mostly affected by multiple hazards and correspondingly with higher level of vulnerability and b) Reinforce Haitian institutions› capacity to replicate and leverage, at national scale the knowledge, behavioral changes and models of social assistance prompted by the program for preventing food insecurity and malnutrition. To reach the goal, Kore Lavihas the following four Strategic Objectives (SO) and 10 Intermediary Results (IR): SO1: National systems for vulnerability targeting strengthened IR1.1: MAST-led equitable vulnerability targeting methodology developed, tested and implemented; The first strategic objective covers three main outputs (which are supposed to be achieved through coordination and joint efforts of Kore Lavi and MAST): a) to develop, test and adjust a national Household Deprivation and Vulnerability Index (HDVI), which allows the program to measure vulnerability in a multidimensional way. The index relies on specific indicators (Proxy Means) attached to each household, and is calculated through a customized algorithm. All partners have jointly worked on a standard vulnerability targeting questionnaire and analysis that was presented and reviewed in a participatory manner to improve the quality of the tool. Therefore, the HDVI index canbe used more widely than just within the context of Kore Lavi. b) to define a specific data collection methodology (socio-economic survey, community pre-targeting etc.), implemented by other organizations (CNSA, FAES, World Vision, etc.) on a door-to-door basis, whichhas allowed for the Kore Lavi program to benefit from already-existing data. c) to develop and implement a database application hosted by MAST - a tool to store, analyze and manage collected data. The main outcome of this system is to answer key questions such as how many beneficiaries (vulnerable households) are in the country or what is the geographic distribution and core characteristics of these beneficiaries. This information will allow decision makers to develop an appropriate frame of social assistance/other interventions. These 3 aspects are closely related to one another. After defining the targeting methodology and development of the algorithm (initiated by UNDP / FAES and completed by WFP / MAST) at the early stage of the program, the data collection has Kore Lavi Title II Program Haiti – Midterm Evaluation 102 102 102 been initiated in the communes targeted under the program with the support of various organizations (CNSA, FAES, World Vision, etc.). Once the data has been collected and cleaned, as well as the HDVI calculated and applied to each household, the data was consolidated in the information system (MAST database). At the request of agencies providing social services/assistance the household lists will be shared by MAST for the planning and allocation of interventions. Overall, the approach (measurement of deprivation / vulnerability, targeting methods, information systems) leads the concerted effort to create a singular beneficiary register of vulnerable populations in Haiti. From a long-term perspective, the system, which is currently functional for Kore Lavi activities, will enable MAST to own and manage the system at the national level.However, challenges still remain in the implementation of the targeting methodology, such as dependence another organizations for the availability of data, which has led to delays and limitations with respect to data accuracy. SO2: Access of extremely vulnerable households to local and nutritious foods increased IR 2.1: MAST-led, gender-responsive, food voucher-based safety net model developed and implemented IR2.2: Inclusion of local foods in the voucher-based safety net increased IR2.3: Access to complementary services for safety net households increased Under SO2 component the program: a)developed voucher transfer modalities,prioritizing use of local foods and deliver food vouchers to 18,150 extremely vulnerable households. In particular, the program defined the value of the monthly transfer of food vouchers to the most vulnerable households to be 1,100 HTG (+/- 25 USD) (all beneficiary households get the same amount of assistance). To promote good nutritional practices through a balanced diet strongly supported by fresh food and at the same time to make a direct contribution to local production, Kore Lavi uses two types of vouchers: paper vouchers (value - 400 HTG/9USD), which are allocated for the purchase of local fresh foods (fruits, vegetables, fresh meat, fish, eggs etc.) and electronic vouchers (value 700 HTG/16USD) allocated for the purchase of local staple foods (local rice, maize, sorghum, beans, flour, and oil). The fresh food is purchased in the community markets (which function every week) and the staple food is purchased in small community shops, from the vendors selected by the program. Nonetheless, not limiting local merchants’ capacity in supplying fresh foods, the following criteria were applied when selecting the vendors (mostly female merchants): their capacity to deliver a variety of fresh foods and their commitment to proposing stable and fair prices; agreement to the program requirement to sell only local products approved by the program; agreement to the program requirement of the merchants to be registered on a regular basis in the market in which they work etc. b) implemented food voucher-based school feeding pilot in six (6) selected schools. As a part of this pilot Kore Lavi serves one snack and one hot meal for school children on a daily basis with the support of local caterers trained on hygiene and nutrition practices and equipped with improved cook-stoves (using natural gas). This pilot initiative emphasizes local entrepreneurship, locally produced foods, and is managed outside the school system. The pilot intends to provide an alternative to traditional school feeding interventions in Haiti, which are primarily led by the school management and involve long distance trucking of staple foods, mostly imported (Rice, Corn flour, Beans, Oil, etc.). Therefore, for the implementation of this pilot, the program selected schools that were not included in the national school program. c) enables VSLA participation (a self-reliance mechanism supporting resilience through accumulation of savings to ensure support in the event of various shocks experienced by the households) benefiting over 180,000 VSLA household members.Kore Lavi facilitated the creation of the VSLA groups Kore Lavi Title II Program Haiti – Midterm Evaluation 103 103 103 order to help integrate as many safety net beneficiaries as possible and for as long as possible during the LOA, and to support the gradual graduation (out of the safety net) of the less vulnerable beneficiaries. SO3: Maternal and child nutritional status improved IR3.1: Household practice of appropriate nutrition behaviors to prevent malnutrition increased IR3.2: Capacity of community-based entities to promote appropriate nutrition practices to prevent malnutrition improved IR3.3: Capacity of health facilities to deliver appropriate nutritional services strengthened To achieve the third strategic objective, Kore Lavi implements an SBCC strategy in line with the preventive approach focusing on the window of opportunity in the first 1,000 days of life (PM2A). The program provides supplementary conditional rations to targeted individuals - Pregnant and Lactating Women and Children under 2 (a total of 76,567 pregnant and lactating women and 96,597 children aged 6-23 months during the life of the program). The conditions that beneficiaries must meet in order to receive food rations are: (a) to take part in primary health care activities (such as antenatal and post-natal consultation, growth monitoring and immunization) and (b) participate in the SBCC interventions of the program. The SBCC strategy is mainly implemented through a Care Group approach. A Care Group is a group of Lead Mothers who are identified by the community and regularly trained by the local community health agents (ASCP126). Each Lead Mother conducts visits to the target households (P&L women, children under 2 and their family members) in her neighbourhood on monthly bases, to share and promote key IYCF272 practices and other appropriate health and nutrition information. In order to reinforce the knowledge of the community members regarding proper health and nutrition behaviour and practices, Kore Lavi collaborates with the existing network of the community health agents of MoH while sensitizing (through various community public awareness raising campaigns such as media campaigns, nutrition fairs, special days and events etc.) key community structures, civil society organizations and traditional health workers. Moreover, to reinforce health and nutrition services provided by the health facilities, the program conducted trainings for health professionals and primary health care workers. One of the main activities currently supported at the health facility level is the management of MAM. Namely, through the network of community health agents, the program supports and facilitates active and passive screening of children to identify new cases of MAM and refer them to healthcare facilities to receive nutritional treatment. SO4: Haitian institutions’ capacity to effectively lead and manage safety net programming improved IR4.1: Institutional capacity of various levels of government to lead, coordinate and implement safety net programs reinforced IR4.2: Capacity of civil society to monitor and support safety net programs reinforced 26 Agent de Santé Communautaire Polyvalent 27 Infant and Young Child Feeding Kore Lavi Title II Program Haiti – Midterm Evaluation 104 104 104 IR4.3: Government capacity to respond to food emergencies expanded Under SO4 component the program intends to: Complement joint USAID/MAST capacity building and resource mobilization plan; Work with MAST national and department-level staff on coordination of safety net program activities; Develop MAST safety net reference documentation (policies, governance, coordination, implementation); Support CADEPs and/or other civil society organizations to develop capacity building plans and carry out social audit processes; Implement gender-sensitive annual social audits with local stakeholders; In particular, community structures (local authorities, civil society organizations) have received trainings on social accountability focusing on the application of the Community Score Card (CSC), a tool to enhance community influence on the quality, effectiveness and accountability of the services offered by Kore Lavi at community level. Kore Lavi used interactive and participatory forums to highlight specific activities with service users. Approximately 30 percent of the participants in these forums were women. This component also involves support to CNSA and IHSI to deliver timely and standardized survey assessments/early warning reports, and supports DPC to enable safety net expansion during emergencies. In this regard, Kore Lavi provided technical support to CNSA (in the form of an assigned technical resource person) to update livelihood profiles (last versions dated back to 2005). Furthermore, Kore Lavi has actively contributed to the CNSA›s involvement in the implementation of vulnerability targeting in the program target areas. Lastly, technical and financial support was provided to the spring harvest season assessment conducted by the CNSA and IPC (Integrated Phase Classification) process. Activities for promoting gender equality are integrated into each component of the program to enhance the quality, and not just the quantity, of female participation at different levels, from participation in strategic planning at the national level to more effective participation in decision-making at the household level. In addition, the program has been providing support to the Ministry for Women›s Affairs and Women›s Rights (MCFDF) to be able to promote and support gender integration in food security and social assistance programming in Haiti. Kore Lavi›s foundational theory of change is that positive and lasting change must happen in three interrelated domains for the Haitian extremely poor to escape chronic food and nutrition insecurity and recover dignity. The three domains of change are (1) the effectiveness of social safety net mechanisms and complementary services that reach the most vulnerable populations, protect their access to food while building self-reliance, (2) the scale and depth of behavior and social change that are needed to tackle the under-nutrition among vulnerable women and children, and (3) the importance of institutionalization of accountability, quality and mutually reinforcing social protection programs under the leadership of MAST. To achieve overall goal, strategic objectives and planned results Kore Lavi operates through the following management structure/scheme: • At the strategic oversight level, a Steering Committee (led by MAST, including Ministry of Health, USAID, Heads of Consortium Agencies, and the COP). • At the management level, an Executive Team (comprising the Chief of Party (COP), 2 Deputy Chief of Party (CARE & WFP), ACF Deputy Director for Programs); a Program Management Team-PMT (Executive Team and Strategic Objective Leaders), and a Program Forum (PMT, Field Coordinators, and Functional Managers - Finances, M&E, Gender, Communications, and Accountability). Kore Lavi Title II Program Haiti – Midterm Evaluation 105 105 105 • At the technical oversight and implementation level: • At the performance management level: Kore Lavi M&E and Knowledge Management unit (led by Knowledge and Learning Coordinator (KLC), including M&E Data Specialist (responsible for maintaining/ updating a centralized program database, as well as consolidating Program M&E data on a quarterly, semiannual and annual basis);4 Knowledge/Learning Field Officers (one per department - responsible for quality of data and maintaining the departmental achieves of supporting documentation) and 8 data entry officers (2 per department). B. Evaluation Objectives Kore Lavi Mid-Term Evaluation (MTE) has the following objectives: • To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. • To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with Haitian Government (MAST, MSPP). • To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). • To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. • To recommend the future orientation (adjustments to the ToC, project design, resource allocation, or implementation) and emphasis of the program during its remaining time. Program Component GoH Partners ACF CARE WFP WV SO1: Vulnerability Targeting MAST Technical Lead & Implementation SO2: Food Voucher-based Safety Net MAST Technical Guidance on Fresh Market Vouchers, Implementation in Two Depts. (North-West and Artibonite) Technical Lead & Guidance on E-Vouchers, Implementation in Two Depts. (South-Eat, Center) Implementation in one department (West/La Gonave island) SO3: Maternal and Child Health and Nutrition MSPP Technical Lead &Implementation in Two Depts. (North-West and Artibonite) Implementation in Two Depts. (South-Eat, Center) Commodity Management SO4/IR 4.1 MAST Stewardship MAST Technical Lead & Implementation SO4/IR 4.2 CADEPs & CSOs CADEPs & Local CSOs Implementation in Two Depts. (North-West and Artibonite) Implementation in Two Depts. (South-Eat, Center) Technical Lead SO4/IR 4.3 DPC, CNSA & IHSI Technical Lead & Implementation Gender-cross cutting MCFDF Implementation in Two Depts. (North-West and Artibonite) Technical Lead & Implementation in Two Depts. (South-Eat, Center) Kore Lavi Title II Program Haiti – Midterm Evaluation 106 106 106 C. Methodology The consultant will be responsible for defining and carrying out the overall evaluation approach. This will include specification of the techniques for data collection and analysis, structured field visits and interactions with beneficiaries and evaluation team. Evaluation tools, methodology and findings should be reviewed and validate with various stakeholders. The evaluation will include an assessment of each of the program components (SOs) – vulnerability targeting, food voucher based safety net, maternal and child health and nutrition and institutional stewardship, as well as an institutional capacity assessment of the implementing partners and local government (at central and regional levels) to identify the progress made. Both, qualitative and quantitative data is recommended to be used when focusing mainly on the qualitative method. The qualitative data will be collected through: a) review of the key program documents and studies (Kore Lavi proposal, results framework, gender analyses, formative research etc.)b) interviews (individual and/or focus group discussions) with key informants, and c) observations of the program in the field; The quantitative data will be collected through: a)review and assessment of the quantitative data available on program performance from the baseline survey results; b) review of the existing M&E data stored at the central and regional levels and c) small scale quantitative survey or other measures as deemed necessary by the consultant. The data and information must be collected separately for the following categories: a) individuals benefiting from one program component only (for example, only from food voucher safety net component or only from health and nutrition component); b) individuals benefiting from several program components (for example, food voucher safety net and VSLA groups; both food voucher safety net and maternal and child health and nutrition; or combination of food voucher safety net, VSLA groups and maternal and child health and nutrition); c) individuals who live in targeted communities but have not directly benefited from any project activity(or who were excluded from the program after determining that they did not meet the criteria, looking for evidence of inclusion and exclusion errors, as well as to capture their impressions of the activities’ implementation and relevance); d) representatives of the relevant government agencies (at both central and regional levels) and civil society organizations; e) Other constituencies and stakeholders not directly involved in the program who may have experienced, or may be expected to experience, its impacts. Data, information and analyses should be desegregated by sex and geographic areas. Kore Lavi Title II Program Haiti – Midterm Evaluation 107 107 107 D. Focus Areas and Key Evaluation Questions The MTE will, using the methodology described above, and bearing in mind the following focus areas, review the status of each program outcome (strategic objective): Design • How relevant/appropriate is the program concept and design (particularly MAST-led institutionalization of a nation-wide social transfers scheme that is poverty/deprivation-targeted, food-voucher based, and that is linked to food-assisted 1000 days approach to prevent malnutrition) to the current Haiti’s economic, socio-political context and the food security condition across Kore Lavi portfolio? • How well it (program design) matches the needs, priorities and contexts of the target communities and beneficiaries? • To what extent the program activities and outputs contribute (is consistent) to intended effects and impact as declared in the Project Results framework? • Implementation/Management and Coordination • How effective is the program general implementation and management in terms of quality and efficiency of inputs and activities, including (but not limited to): adherence to work plans and budgets; management of food commodities; routine monitoring of the program activities; financial management? • Institutional set-up throughout the Steering Committee, Program Management Team-(PMT), Program Forum, and Program Technical Working Groups and the degree to which they have encouraged full involvement of the consortium members and key government stakeholders (MAST and MSPP); • To what extent have the management arrangements (planning, operational decision making, problem solving, data/information management, reporting,…), integrated teams, technical/component leadership been adequate, as well as monitoring and backstopping support given by all parties concerned? • Co-operation (programmatic and organizational) among Kore Lavi partners (CARE, WFP, ACF, MAST, MSPP, USAID/Haiti) and effectiveness of this partnership. • To what degree has the collaboration with key social protection stakeholders, implementing agencies (UN, World Bank, other NGOs,…) been effective? Progress • What is the achievement, to date, of the program activities, outputs and outcomes as detailed in the Program Results Framework and the Program Implementation Plan? • What are the major factors which have facilitated or impeded the progress of program implementation? • For each component, which activities and implementation processes are more or less acceptable to members of the target institutions, communities and why? How well do implementation processes adhere to the underlying strategies, principles and established methodology and processes? • Was the vulnerability targeting methodology properly applied? (including community participation, exclusion/inclusion errors, gender/diversity issues, vulnerability database, local government / stakeholder participation and verifiable indicators in relation to targeting). Kore Lavi Title II Program Haiti – Midterm Evaluation 108 108 108 • To what extent has e-voucher system been an effective, useful and efficient mechanism for the timely delivery of subventions to the safety net beneficiaries? What are the factors that could promote or block the broader use of the e-voucher system? • How effective has the VSLA approach been in the achievement of the planned result (to mobilize capital and facilitate graduation of beneficiaries from the safety net)? • How successful has the school feeding pilot (as one of the mechanisms of social protection) been in increasing school attendance, promoting greater private sector and women participation? How replicable and sustainable the pilot is likely to be? • To what extent has the SBCC activities oriented the targeted HHs to make appropriate choices regarding nutrition and health? • How have the relevant Haitian institutions (MAST, CADEPs, CNSA, DPC etc.) been strengthened in their ability to a) coordinate and manage social assistance programming b) monitor safety net programs c) respond to food emergences? • What has been the level of ownership of the program (commitment of resources/adjustment of programming and budget / participation) by MAST, and other Haitian government institutions (in particular, MSPP)? • How effective has the program gender strategy been in promoting/improving gender equality? • What role, if any have cross cutting issues (gender, social accountability) played in shaping Kore Lavi interventions and influencing progress towards achieving planned results Effects • What difference (changes in practices, attitude, behavior, knowledge) did interventions of the program make to the beneficiaries (considering the difference between what has happened with and what would have happened without the program)?What are the major factors which will facilitate or discourage observed changes? To which extent observed changes are intended (e.g. corresponding to hypotheses of the program theory of change)? • What are the major factors which will advance or deter to scale up the methodologies and systems implemented by the program (vulnerability targeting, safety net, ASCP network, Care Group approach) nationally after the program ends. • What are the major factors which have facilitated or impeded the participation of the private sector (vendors, micro financial institutions) in the social safety net programs? • The likely sustainability of program interventions; • Learning from first half of the program and Recommendations • What are the key strengths and weaknesses of the program? • What are the key/critical emerging learning that the program should deepen to improve quality of interventions, further disseminate to inform/influence the practice of social protection in Haiti and FFP programs? Kore Lavi Title II Program Haiti – Midterm Evaluation 109 109 109 • What the program should modify to improve implementation and achieve intended results? • Any changes/adjustments in the results framework • Any changes in implementation strategies, approaches • Any changes in resource allocation • Any changes in management/coordination mechanisms E. Evaluation team composition and team members’ roles The MTE team is expected to consist of a Team Leader (external to Kore Lavi and all agencies involved in the implementation of the program), Technical Specialists and person responsible for logistics. The composition and expertise of the team should correspond to the evaluation purpose and the technical sectors and cross-cutting areas which will be assessed. The Team Leader - must possess in-depth experience and expertise of the program evaluation procedures and techniques (mixed methods of assessment preferable), particularly of programs funded by USAID/FFP in developing countries. Knowledge and experience in food security and nutrition programming, Gender analysis and Governance is highly desirable. If the Team Leader does not have experience in some of the above-mentioned sectors, the Technical Specialist with the required experience and expertise must be included in the evaluation team. The Team Leader, working closely with the M&E unit and M&E Technical Group of the program, will have the overall responsibility to organize, plan and lead the evaluation, as well as to develop and submit the final evaluation report (assuring that all findings, conclusions and recommendations presented in the report are logically based on evidences found through the assessment and have a high level of confidence). Fluency in English and French is required. The Technical Specialists(no more than 8specialists in total) - are expected to have a post graduate degree and possess experience and expertise in at least one of the following sectors: livelihoods (including comprehensive understanding of vulnerability targeting, food voucher safety net and economic components of development programs), MCHN (experiences in PM2A and Care Group approaches would be highly recommended), commodity management, governance, gender, M&E and Knowledge management system, program management and partnership. The Technical Specialists will be responsible to elaborate the methodology, plan and lead the process of primary/secondary data collection and analyses (related to each technical sector), as well as to form conclusions and recommendations. Logistics manager - will lead and manage all logistics issues of evaluation (including and not limited to staffing, training venues, transportation, logistical arrangements, accommodation, etc). Note: Consultant must include all anticipated logistics related arrangements as part of the proposed budget and should bear the responsibility of managing sub-contracts, rents or payroll of affected staff. Detailed budget should form integral part of the technical proposal. Kore Lavi Title II Program Haiti – Midterm Evaluation 110 110 110 F. Responsibilities of Kore Lavi The Kore Lavi program will be responsible to: • Provide background information/insights on the implementation context; • Provide a list of all target communities; • Provide sufficient sources of secondary data (including but not limited to approved proposal narrative and relevant attachments and documentation, program reports (quarterly, annual), Pipeline and Resource Estimate Proposals (PREPs), baseline report and reports from all conducted research/ studies(e.g., formative research, gender analyses, market analyses etc.), detailed implementation plans, program monitoring/reporting tools and manuals, program commodity management tools and reports,program beneficiary and activity data bases and descriptions, maps containing appropriate information on program target areas etc.) • Provide appropriate administrative/logistical support (including but not limited to provide contact details for key informants, provide information on security issues and local protocols, organize meeting between evaluation team and consortium members/donor at the initial and final stages of the evaluation process etc.) • Other relevant information and support as needed. G. Deliverables The evaluation team should present to Kore Lavi the following deliverables: • Detailed implementation/work plan with a calendar of key activities and milestones, along with the roles, responsibilities, anticipated outputs and associated levels of effort of each of the evaluation team members; • Evaluation methodology plan - including data collection instruments (English, French, and Creole versions) and protocol, data entry template/ screen used, data analysis plan, data sets, lists of sites visited, and lists of informants; • Transcribed data/ information. • Draft report (in English and French, based on a formal and agreed report framework) of preliminary analysis and key findings - the program staff and FFP will provide comments on the draft report, and the evaluation team will finalize the report in view of these comments; • Presentation - for presenting and discussing the draft final report interactively, the evaluation team will facilitate a one-day concluding workshop for the program stakeholders; • Final report(in English and French) with accompanying executive summary - should include the strengths and limitations of the evaluation methods and how these may have influenced the findings. It should be logically structured, contain clearly separated evidence-based findings, conclusions, lessons and recommendations, and should be free of information that is not relevant to the overall analysis. The report should respond in detail to the key focus areas described above. It should include a set of specific recommendations formulated for the program, and identify the necessary actions required to be undertaken, who should undertake those and possible time-lines (if any). Kore Lavi Title II Program Haiti – Midterm Evaluation 111 111 111 H. Timeframe The Mid-Term Evaluation of the Kore Lavi program will take place within October, 2015 to February 2016. A tentative timeframe(see table below) including stages of preparation and implementation of the evalua￾tion can be adjusted subsequent to consultations with USAID/FFP and evaluation team28. Table 1 - Timeframe of MTE Activity Timeframe Review of the final Statement of Work of MTE by the responsible FFP M&E Specialist April-May, 2015 Approval of the final Statement of Work of MTE by the AOR of the program May, 2015 Recruitment and procurement of evaluators May-July, 2015 Secondary data review by consultant and MTE work plan preparation (includ- )ing evaluation tools August 2015 Review and approval of the MTE work plan by the responsible FFP M&E Spe￾cialist and AOR of the program September, 2015 Data collection October -November, 2015 Data analysis &drafting of report November, 2015 - January, 2016 Presentation of the draft final report to the program stakeholders one day workshop, January, 2016 Feedback from USAID/FFP & finalization of MTE report February, 2016 I. Ethical Guidelines The Evaluation Team members should adhere to the following ethical guidelines292 : Systematic Inquiry: Evaluators conduct systematic, data-based inquiries. Competence: The evaluation team possesses the education, abilities, skills and experience appropriate to undertake the tasks proposed in the evaluation. Evaluators practice within the limits of their professional training and competence, and decline to conduct evaluations that fall substantially outside those limits. The evaluation team collectively demonstrates cultural competence. Integrity/Honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process. Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders. Evaluators regard informed consent for participation in evaluation and inform participants and clients about the scope and limits of confidentiality. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation. 28 Due to significant political challenges occurred in the country and upcoming Presidential and Parliamentary elections in October, 2015 the timeframe of the Mid-Term Evaluation can be updated. 29 A more detailed description can be found at http://www.eval.org/p/cm/ld/fid=51 Kore Lavi Title II Program Haiti – Midterm Evaluation 112 112 112 Annex 2: Evaluation Team Profile and Biographies Gary Woller – Team Leader Dr. Woller has over 15 years of international development experience in monitoring and evaluation of complex, multi-million dollar projects covering a variety of development topics, such as making markets work for the poor (M4P), microfinance, financial sector reform, MSME development, youth workforce development, community economic development, democracy and governance, nutrition and food security, HIV/AIDs, energy services, and climate change. Dr.Woller has served as the lead expert in developing and designing performance monitoring systems for several bi-lateral and multi-lateral funded projects. In this role, he managed and coordinated the overall approach to performance monitoring, selected performance indicators, designed data collection and reporting processes, and implemented data quality assessments. Dr. Woller has also served as team lead for numerous mixed-methods impact evaluations using a diverse set of quantitative and qualitative methods and evaluation designs, including randomized controlled trials (RCTs), regression discontinuity designs, quasi-experimental designs, pre-post designs, reflexive comparisons designs, and non-experimental designs (e.g., post-project evaluation). He has also designed and implemented numerous performance evaluations utilizing primarily qualitative data collection methods. During his evaluation work, Dr. Woller has conducted hundreds of key informant interviews and facilitated dozens of focus group discussions with a diverse range of stakeholders groups. Dr. Woller has a B.A. in Economics and MBA from Brigham Young University and a Ph.D. in Political Science from the University of Rochester. He speaks English and Spanish. Bryan Rhodes – SO1 & SO4 Specialist Mr. Rhodes has been providing innovative solutions to private sector and development clients for almost 20 years. After a successful career as an investment analyst, Bryan has specialized in agribusiness and access to finance development since 2005. Mr. Rhodes has served as the AgroEnterprise Development Specialist for CRS in Afghanistan from 2005 to 2007, and as Chief of Party for the USAID-funded Afghanistan Small and Medium Enterprise Development (ASMED) program, where Mr. Rhodes led the development of six primary value chains including horticulture, livestock, gemstones, carpets and handicrafts, and women￾owned small businesses. Since joining Absolute Options in 2011, Mr. Rhodes has led over 30 value chain and financial feasibility assessments, including studies in Zimbabwe, the Democratic Republic of Congo, Tanzania, Ghana, Guatemala, El Salvador, Honduras, Nicaragua, and Mexico. His value chain assessments include a diverse range of sectors including coffee, cacao, livestock, banana and plantains, high-value horticulture, rice, beans, maize, and agro-forestry systems. He has extensive experience with the dairy and livestock sectors in India, Bangladesh, and Afghanistan. Mr. Rhodes completed his graduate studies in Political Economy of Development at the University of California, Los Angeles. He speaks Spanish and English. Kore Lavi Title II Program Haiti – Midterm Evaluation 113 113 113 David Rinck – SO2 Specialist Mr. Rinck has over 18 years of proven leadership experience in agricultural development programming, with technical specializations in program evaluation and development, agro-enterprise and enterprise development, agriculture, and monetization. He has designed and managed USDA FFPr programs and implemented commodity monetizations under the Title II and FFPr programs in Afghanistan, Angola, Bangladesh, Benin, Burkina, Cameroon, El Salvador, Eritrea, Ethiopia, Kenya, Liberia, Madagascar, Malawi, Mali, Mozambique, Nicaragua, Pakistan, Sierra Leone, Tanzania, Zambia, and Zimbabwe. As a result of his extensive experience with USDA food assistance programs, he was specifically selected as the Senior Evaluator for a 2013-2014 USDA FFPr Program Evaluation implemented by AO and Booz Allen Hamilton. Prior to AO, Mr. Rinck spent five years as a Regional Food for Peace Officer at the USAID East Africa regional mission in Nairobi, Kenya. Between 2001-6, he served as Regional Technical Adviser for Monetization and Agro-Enterprise Development in Eastern and Southern Africa for Catholic Relief Services (CRS), where he represented CRS in an Agro-Enterprise Learning Alliance with the Centro International de Agricultura Tropical (CIAT) to develop smallholder values chains. In this capacity, he also developed and implemented the multi-year Market Assistance Program (MAP) in Zimbabwe to strengthen coarse grains value chains (sorghum and millet). This program was recognized by USAID as a successful and innovative model to support commercial agro-industry and provide food in challenging markets: http://www.usaid.gov/stories/ zimbabwe/fp_zimbabwe_food.html He has undertaken over 55 evaluations of monetization and food programs, and contributed to the updated USAID Monetization Field Manual developed by Fintrac in 2012. David holds a Master’s Degree from the University of Chicago, and a degree in Agricultural Economics from the University of California at Davis. He has also studied economics, languages and sociology at many locations worldwide, including Beijing University, as well as in the Middle East at Damascus, Beirut, and Sana’a. He speaks Arabic, French, Portuguese and Spanish. Mayling Simpson – SO3 Specialist Dr. Simpson has over 30 years of proven leadership experience in infant feeding, maternal and child health and water and sanitation. She has lived and worked in Iran, Philippines, Nepal, Switzerland, Serbia, Ethiopia and Kenya. She conducted over six years of infant feeding field research in Iran and the Philippines and produced several publications. She was a senior technical advisor on water, sanitation and hygiene (WASH) for World Bank, World Health Organization and Catholic Relief Services, serving several countries in Asia and Africa from 1979 to 2010. She was a leading researcher and writer of the PHAST approach to community participation in water supply and sanitation and a lead author of publications on Ecological Sanitation. She also was one of CRS’ East Africa’s lead specialists on maternal and child health, HIV/AIDS and water resource management. She has conducted several project evaluations on water and sanitation, including an OFDA project in Ethiopia, and safety-net projects using volunteers, including the WALA project in Malawi. Dr. Simpson holds a PhD from the University of North Carolina at Chapel Hill in Medical Anthropology. She has spent most of her career in development work but has also taught at UNC Chapel Hill, Johns Hopkins University, Colorado Mountain College and University of Belgrade as a Fulbright Scholar. She speaks English, French, Farsi and some Spanish. Kore Lavi Title II Program Haiti – Midterm Evaluation 114 114 114 Grace Fisiy – Gender Specialist Dr. Fisiy is a gender, agriculture and M&E specialist with over 20 years-experience promoting gender equality and women’s economic empowerment in the agriculture and food security sectors across Africa and Asia via her work with MCC, DFID, UNDP, IFC/World Bank, Grameen Foundation, the Mitchell Group, Women for Women International and the Government of Cameroon. She has extensive experience planning, designing, managing, implementing and evaluating projects that involve gender integration and women’s empowerment, sustainable agricultural development and food security, agricultural value chains, Making Markets Work for the Poor (M4P), public and private partnerships, agricultural policy reform, and capacity and leadership building. Dr. Fisiy holds a PhD in Development Studies with a focus on Gender and Agrarian Change and an MA in Agricultural and Rural Development both from Eramus University’s International Institute of Social Studies in The Hague. She is fluent in English, French and West African Creole. Matthew Innes – MEAL Specialist Mr. Inns is an evaluation expert specializing in development and implementation of information systems. His specializations include measurement of project design and management, performance analysis, measurement, evaluation and technology selection, and systems development. Mr. Innes has developed and implemented M&E systems for a wide range of agricultural agencies and companies, including Bloomberg Philanthropy Relationship Coffee Institute, Sustainable Food Labs, Sustainable Harvest Inc., ACDI/VOCA, Ben & Jerry’s and Green Mountain Coffee. His geographic experience spans Africa, Latin America and the Caribbean, and Oceania. Most recently, he served as the M&E Adviser to the USAID-funded Competitiveness and Trade Expansion (COMPETE) program in Tanzania, and enumerator and staff training expert for the Rwanda Specialty Coffee project. His education includes a BA in business from the University of Maryland, an MBA with a focus on sustainable supply chain management from Portland State University Business School as well as a Diploma in Law (J.D. equivalent) from Inns of Court School of Law, London, UK, and a Masters in Law from Tulane Law School in New Orleans. He has also undertaken PhD coursework in operations management, supply chain strategy, multivariate regression, factorial modeling, and multiple regression analysis. He is fluent in French, Spanish, and Arabic. Kore Lavi Title II Program Haiti – Midterm Evaluation 115 115 115 Annex 3: List of Key Informants Interviewed Organization/Name Title CARE Jean-Michel Vigreux Country Director, Steering Committee Tamara Shukakidza Deputy Country Director Juvenal Afurika COP Roseval Supreme DCOP Operations Laurore Antoine Safety Net (SO2) Technical Coordinator Murat Gilot VSLA Manager (SO2) Jude Lafrance Social Accountability Manager Fenold Clerval Commodity Manager Marie Ardys Jean Baptiste Gender Coordinator Lia Dididze Knowledge And Learning Coordinator Cesar W. Dieudonne M&E Specialist Seneq Pierre-Martelly Communication Specialist Menoucheka Vieux SO3 Manager Health Systems Magalie Benjamin Field Officer Coordinator (Center) Charcot Rosembert Field Office Coordinator (Southeast) Marie Elphine Loquillon SO3 Coordinator (Southeast) ACF Helene Queau Country Director, Steering Committee Mathieu Vernusse DCOP / Deputy Country Director Delphin Sula Health And Nutrition (SO3) Technical Coordinator Natasha Jean Baptiste Health and Nutrition Mobile Team Manager Quarante Neils Field Office Coordinator (Artibonite) Josue Limage Field Manager Gonaïves (Artibonite) Jean Joinel VSLA Technical Officer (Artibonite) Michel Kerby SO4 Technical Officer (Artibonite) Jesumene Raphaël Health and Nutrition Manager (Artibonite) Roseanna Christian Nurse Supervisor (Artibonite) Junia Charleus Nurse Supervisor (Artibonite) Nicolas Fegeant Field Office Coordinator (Northwest) Margot Clavier Field Office Coordinator (Northwest) Bresil Schnader Social Protection Agent (Artibonite) Beuplan Wadson Social Protection Agent (Artibonite) WFP Cedric Charpentier Head of Program Felix Veronneau DCOP Institutionalization / SO4 Lead Thomas Calvot SO1 Targeting Coordinator Thomas Gabriel Technical Advisor of Information Systems Murielle Bonostro Responsible For Distributions Wendy Bingham Deputy Country Director Harry Jerome Senior Logistics Assistant Serge Raphael Logistics Assistant Kore Lavi Title II Program Haiti – Midterm Evaluation 116 116 116 Marc Henry Joseph Logistics Assistant World Vision Lesley Michaud Deputy Director Operations Jean Luis Pierre Food and Cash Project Manager Jdhymi Dulaurier M&E Director (La Gonave) Wendy Salomon M&E Officer (La Gonave) Salomen Fausten Knowledge Information Officer (La Gonave) Jude Suffrin Safety Net (SO2) Technical Officer (La Gonave) MAST Mathieu Remedor Director Pierre Ricot Odney Planning Unit Director, Steering Committee Jean Robert Brutus Social Protection Consultant, Steering Committee Rozéfort Deslandes Technician - Planning unit Blanc Rolner Technician - Planning unit Adler Elcine Technician - Planning unit Gary Thias Deputy Coordinator Social Protection Clermont Saint-Vil Departmental Director (Northwest) Wesaur Saint-Victor Departmental Director (Artibonite) Jeanel Derazin Kore Lavi Focal Point (Southeast) MSPP Joseline Marhone Pierre UCPNANU Director, Steering Committee Emmanuel Ted Lazarre Department Director Health (Southeast) Pierre Rony Department Director (Northwest) MCFDF Marie Elise Gelin Director General Myriam Fetiere Coordinator / Kore Lavi Focal Point CNSA Abnel Desamours Technical Director Jean Carrel Norceidi Food Security Analyst FAES Dr. Hans Muller-Thomas Anne Arthur World Bank Anaise Fabius Consultant Digicel Godwin Tamakloe Operations Manger-Mobile Financial Services Roxana Zabaleta CADEC Joseph Fenelus Advisor (Northwest) Port Internationale Dieter Schmid Director of Operations Health Centers Mondestin Joseph Prinston Nurse Thomassique Health Center (Northwest) Sultan Semilien Nurse Bayonnaise Health Center (Artibonite) Louise Immacula Casemir Nurse La Valle Health Center (Southeast) Lead Fathers Exil Jolem Lead Father (Northwest) Kore Lavi Title II Program Haiti – Midterm Evaluation 117 117 117 Annex 4: SO4 Institutionalization Dashboard - September 2015 Vert = Satisfaisant (100% ou +) Jaune= Amélioration souhaitée (51-99%) Rouge= Urgence (0-50% d'accomplissement) Institutionnalisation (OS4) Nombre de postes néssaires pour Kore Lavi Postes comblés au MAST Pourcentage des postes comblés Nombre de jours de visites conjointes Nombre d'heures de travail conjoint Commentaires Directeur du programme Odney Pierre Ricot 1 1 Responsable du système d'info 0 0 Opérateur base de données 0 0 Soutien en développement informatique (appui 30%) 0 0 Soutien en TI (appui 10%) 0 0 2.4 0 0 0 Superviseur ciblage de la vulnérabilité 0 0 Responsable des enquêtes Blanc Rolner / Rozéfort Deslandes 4 35 Coordo et appui technique enquêtes (appui 75%) CNSA 0 4 Coordo et appui technique analyses et stats (appui 25%) 0 0 3.0 1.8 4 39 Coordonnateur coupons alimentaires Adler Elciné 1 1 Kathelen Douyon 0 0 Gary Thias 0 0 1 2 0 0 Coordonnateur bureaux de référence et renforcement A déterminer 1 0 Coordonnateur suivi et évaluation Marmotel Fils Sémelfort 1 0 0% 0 Une discussion aura à être entamée avec le MAST pour définir cette fonction au MAST central. Besoin de relancer l'institutionnalisation du suivi et évaluation. OS1/Système d'information - MAST central 0% Pour la mise en oeuvre des activités relatives au système d'information, absence de personnel du MAST (cependant, intérêt au niveau de la CAS). Le départ de Félix Duvelson, Responsable du système d'information au MAST, a laissé la passation en plan. Faible participation de gestionnaires à la formation prévue sur le système d'information. Pas de soutien du département informatique du MAST. 0 200% Coordonnateur santé materno-infantile et nutrition Implication périodique du MAST dans les rencontres avec le MSPP. Question importante: à quel point le MAST veut-il/doit￾il s'impliquer dans ce volet ? OS3/Santé materno-infantile et nutrition (SMIN) - MAST central OS4/Bureaux de référence et organisations communautaires de base (OCB) Suivi et évaluation 100% 0 0 0 TABLEAU DE BORD - PROGRAMME KORE LAVI (A l'intention de la Direction Générale du MAST) Mois Septembre 2015 Mesures de performance La collaboration qui existe entre le personnel du MAST et celui de Kore Lavi est prometteuse. Le processus d'appropriation suit son cours et les gens sont de plus en plus motivés a s'impliquer activement dans des activités de terrain. OS1/Enquêtes - MAST central OS2/Coupons alimentaires - MAST central 100% 68 58% Bon engagement et engouement du MAST pour les activités en lien avec le ciblage (participation à des réunions de réflexion sur des approches méthodologiques, participation à des missions, formations, etc.). Lien avec IHSI pour appui non encore concrétisé. 8 Direction - MAST central 100% 14 Le Directeur de l'UEP continue à mener chaque Groupe technique démontrant l'importance qu'il attache à ces rencontres. Il a également présidé à la rencontre de fin d'année de Kore Lavi. Il travaille toujours à faire débloquer l'affectation du personnel pour le système d'information. Kore Lavi Title II Program Haiti – Midterm Evaluation 118 118 118 Vert = Satisfaisant (100% ou +) Jaune= Amélioration souhaitée (51-99%) Rouge= Urgence (0-50% d'accomplissement) Institutionnalisation (OS4) Nombre de postes néssaires pour Kore Lavi Postes comblés au MAST Pourcentage des postes comblés Nombre de jours de visites conjointes Nombre d'heures de travail conjoint Commentaires Directeur du bureau départemental Wesler Saint-Victor 0 8 OS2 - Coupons alimentaires Jean-Louis Messac 2 6 OS2 - Groupes AVEC Kenny Morose 0 0 OS3 - SMIN Ketty Fénélon 0 0 OS4 - Bureaux de référence & OCB Thony Sanon 4 49 Suivi et évaluation 0 0 6 5 6 63 Directeur du bureau départemental Wiltonn Pierre 2 6 OS2 - Coupons alimentaires Lochener Etienne 4 15 OS2 - Groupes AVEC Aroldson Jean 2 6 OS3 - SMIN Mitania Barthelmy 3 10 OS4 - Bureaux de référence & OCB Louis Casibo 5 10 Suivi et évaluation Louis Casibo 2 4 6 6 18 51 Directeur du bureau départemental Clermont Saintvil 0 10 OS2 - Coupons alimentaires Sergo Auguste / Patrick Jn Milcent 0 10 OS2 - Groupes AVEC Robertson Edmond 0 10 OS3 - SMIN 0 0 OS4 - Bureaux de référence & OCB Clermont Saintvil / Elifrant Elysée 1 10.5 Suivi et évaluation 0 0 6 4 1 40.5 Directeur du bureau départemental Mathieu Remedor 0 0 OS2 - Coupons alimentaires Jerome Frantz / Zamy Jhonnys 1 10 OS2 - Groupes AVEC Charles Magdala / Serge Ciguene 0 0 OS3 - SMIN Fevil Lourdy 3 26 OS4 - Bureaux de référence & OCB Deraczin Jeanel 1 2.5 Suivi et évaluation 0 0 6 5 5 38.5 Nombre de rencontres MAST durant le mois Qualité de l'engagement avec le MAST pour Kore Lavi MSPP 1 Amélioration souhaitée CNSA/MARNDR 2 Satisfaisant MCFDF 1 Amélioration souhaitée FAES/MCPE 1 Satisfaisant Commentaires Le protocole MAST-MSPP est toujours en attente. Cependant, les relations entre les deux Ministères sont cordiales. Des représentants de chaque institution sont normalement présents dans les rencontres liées au composant OS3, aucune n'a cependant eu lieu en septembre. Un délégué du MSPP a assisté à la rétrospective de l'année 2 de Kore Lavi avec le MAST. Au niveau départemental, un accord a été signé avec la Direction Sanitaire de l'Artibonite. Cependant, peu de collaboration avec les autres départements. En appui au protocole MAST-MARND, la CNSA a complété en août l'enquête de ciblage pour Port-de-Paix. Un représentant de la CNSA était présent à la présentation du système d'information et un délégué a aussi assisté à la rétrospective de l'année 2 de Kore Lavi avec le MAST. La CNSA pourrait cependant être plus impliquée dans les groupes techniques et les réflexions stratégiques de Kore Lavi liées à ses activités. Le MCFDF a été généralement absent aux groupes techniques de Kore Lavi au niveau central en année 2. La nomination d'un point focal du MCFDF pour Kore Lavi devrait améliorer cette situation à partir d'octobre 2015. La Directrice Générale et le point focal Kore Lavi/MCFDF étaient présentes à la rétrospective de l'année 2 parle Comité d'Orientation de Kore Lavi, ce qui est de bon augure pour la suite. Au niveau départemental, des activités sont organisées de manière périodique en soutien à Kore Lavi. FAES a participé à la présentation du système d'information de Kore Lavi ce mois-ci. Le partage d'information continue, mais l'équipe Kore Lavi aurait avantage à pousser davantage pour que des représentants du FAES assistent aux groupes techniques pertinents. Nord-Ouest - Direction départementale 67% Urgence: bail non renouvelé pour le local du MAST. La prise de conscience du MAST est réelle pour Kore Lavi, mais pas encore concrétisée. Difficile de fournir un travail de qualité dans les conditions actuelles (bureaux, budget pour frais de fonctionnement généraux, manque de RH, …) reste un élément de blocage. La réalisation d'un plan d'institutionnalisation pour l'année 3 est une bonne avancée, il faut maintenant le mettre sur pied. Besoin de vis-à-vis MAST en santé materno-infantile/nutrition et en suivi et évaluation. Sud-Est - Direction départementale 83% Volonté manifeste du MAST de participer aux différentes activités. Tous les cadres n'ont le même niveau d'engagement dans ce processus d'institutionalisation. L'équipe Kore Lavi doit travailler beaucoup plus avec le MAST et renforcer ses liens avec le programme. Aujourd'hui le MAST est dans l'incapacite de faire seul certaines visites de supervision faute de moyens logistiques. Le Directeur du MAST, Mathieu Remedor, est un bon collaborateur. Deraczin Jeanel et Fevil Lourdy sont aussi très impliqués dans les suivis. Besoin de vis￾à-vis MAST en suivi et évaluation. Artibonite - Direction départementale 83% Le Directeur du MAST a répété à plusieurs reprises son intérêt pour intervenir dans les décisions et activités concernant Kore Lavi. Il s'agit maintenant d'accroître le travail conjoint avec les vis-à-vis du MAST. Besoin de vis-à-vis MAST en suivi et évaluation. Centre - Direction départementale 100% Flexibilité et disponibilité de la part du MAST reflétées dans le nombre de rencontres conjointes. Besoin d'accentuer l'appropriation intégrale du modèle de filet social. Désir de plus de directives du MAST Central pour Kore Lavi; manque de communication. TABLEAU DE BORD - PROGRAMME KORE LAVI (A l'intention de la Direction Générale du MAST) Mois Septembre 2015 Mesures de performance Kore Lavi Title II Program Haiti – Midterm Evaluation 119 119 119 Annex 5: Kore Lavi Operations Dashboard - September 2015 Vert = Satisfaisant (100% ou +) Jaune= Amélioration souhaitée (51-99%) Rouge= Urgence (0-50% d'accomplissement) Ciblage et SI (OS1) Cible (à ce jour) Résultat actuel Documents de référence validés par le MAST 4 3 Nombre de ménages dans la base de données 150,000 130,937 Infrastructures, matériel, logiciel, base de données 100% 80% Standardisation des processus, format et gestion des données 100% 51% Coupons alimentaires (OS2) Cible (à ce jour) Résultat actuel Nombre de bénéficiaires 17,700 14,852 Pourcentage des bénéficiaires ayant reçu des coupons sur 12 mois ou + A définir 21% Nombre de vendeurs de produits frais et secs 710 789 Nombre de groupes d'épargne AVEC 540 700 Pourcentage des membres du filet faisant partie des groupes AVEC 30% 18% Documents de référence validés par le MAST A définir A définir SMIN (OS3) Cible (à ce jour) Résultat actuel Nombre de bénéficiaires des distributions 71,559 71,577 Nombre de distributions effectuées sans retard 100% 75% Qualité des kits 100% 96% Nombre de ménages atteints par les groupes de soins 74,426 41,090 Bureaux de référence et OCB (OS4) Cible (à ce jour) Résultat actuel Nombre de bureaux de référence en place 16 10 Nombre de plans de renforcement de CADEP en place 14 6 Commentaires Il reste 6 bureaux de référence à mettre sur pied d'ici janvier 2016. Les 8 plans manquants devraient être complétés d'ici mars 2016. Les activités d'appui au programme Kore Lavi vont néanmoins continuer dans ces communes. TABLEAU DE BORD - PROGRAMME KORE LAVI ( A l'Intention de la Direction Générale du MAST) Commentaires Commentaires Il faut noter que la quantité de ménages bénéficiaires enrôlés ne prend pas en compte les zones urbaines (à savoir Port-de-Paix et Gonaïves urbain). Si on soustrait l'effectif pour ces deux communes (1,200) de la cible totale, cela nous donnera un pourcentage d'à peu près 90% de couverture. La cible devrait être atteinte lors de la finalisation des enquêtes des zones urbaines de Gonaïves et Port-de-Paix à la fin du 1er trimestre de l'année 3 (décembre 2015). 5 étapes sur 10 ont été complétées, la 6e est maintenant en cours. Les conditions d'utilisation de l'algorithme sont toujours en cours de validation. Les documents suivants ont déjà été validés: INPV - Protocole d'élaboration de l'algorithme vulnérabilité, Manuel de collecte et codebook INPV, Stratégie et plan de travail pour la base de données Kore Lavi. Mois Septembre 2015 Les ordinateurs, le serveur sont installés. La première version de l'application et la base de données du système sont créées. Il faut maintenant compléter le ménage complet des données entrées dans la base de données. En ajoutant les 66% qui ont reçu entre 6 et 11 transferts, 87% des bénéficiaires du filet ont touché des transferts depuis au moins 6 mois. Plus de 110 millions de gourdes ont été versées en coupons pour produits frais et secs. Mesures de performance Les données sont celles compilées en août. Le nombre de bénéficiaires rejoints excède la cible de départ. Il faut maintenant améliorer la qualité des distributions. Sur base de la dernière analyse des checklists. En cours de redressement avec des procédures mises en place pour améliorer le service aux bénéficiaires. Cette approche de groupes de soins est une nouvelle approche et a nécessité un long travail préparatoire initial. L'approche a été lancée en début d'année 2 et a été progressivement consolidée. La cible n'a malheureusement aps pu être atteinte mais elle est actuellement en cours de renforcement. Ce faible pourcentage est dû au fait que les activites AVEC ont ete lancées bien avant l'établissement du filet. Désormais, les nouveaux groupes seront composés majoritairement de membres du filet. On continue à enrôler d'autres vendeurs dans le programme (surtout les vendeurs de produits secs) pour réduire la distance (en km) parcourue par les bénéficiaires pour échanger leurs coupons. La version officielle du guide opérationnel n'a pas été encore soumise pour validation. Cependant, le panier alimentaire, la valeur et la fréquence de distribution des coupons et les critères de sélection des vendeurs ont déjà été approuvés par le MAST. Commentaires Il y a 20 784 bénéficiaires dont 76% de femmes. Le portefeuille d'épargne s'élève à environ 31 millions de gourdes et le volume des crédits en cours à 26 millions de gourdes. Il importe de noter que parmi les 700 groupes existants, 43% ont bouclé le premier cycle de maturité et ont déjà relancé le deuxième cycle. Sur 102 sites enquêtés, 5 doléances ont été collectées par rapport à la qualité des sachets. Kore Lavi Title II Program Haiti – Midterm Evaluation 120 120 120 Annex 6: SO1 Discussion Guides DISCUSSION GUIDE SO1: National Systems for Vulnerability Targeting Strengthening Mid-Term Evaluation (MTE) “Kore Lavi” (Title II Development Food Assistance Program) Support to the National Program of Food Security and Nutrition CARE USA NOVEMBER 2015 1.0 SO1 Background IR1.1: MAST-led equitable vulnerability targeting methodology developed, tested and implemented. SO1 has three main outputs: 1. Develop, test and adjust a national Household Deprivation and Vulnerability Index (HDVI); 2. Define a specific data collection methodology; 3. Development and implement a database application host by MAST. Kore Lavi Mid-Term Evaluation (MTE) has the following objectives: • To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. • To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with Haitian Government (MAST, MSPP). • To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). • To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. • To recommend the future orientation (adjustments to the ToC, project design, resource allocation, or implementation) and emphasis of the program during its remaining time. 2.0 Interview Guide DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: Kore Lavi Title II Program Haiti – Midterm Evaluation 121 121 121 1. What was/is the role of your organization in the design, roll-out or implementation of the HDVI? 2. What was particularly effective in the design and roll-out of the HDVI? 3. What was particularly challenging in the design and roll-out of the HDVI? 4. Describe specific activities undertaken by Kore Lavi staff during this process? 5. How can the implementation of HDVI be improved? 6. What elements of HDVI do you find most useful? 7. What data collection methodology is currently used for HDVI? 8. What is particularly useful about this data collection methodology? 9. What is particularly challenging about this data collection methodology? 10. Is the HDVI data collection methodology consistently used across geographies and beneficiary types/ groups? 11. What is your experience with the household score card/household vulnerability scoring? 12. What is working well with the household scoring methodology? What can be improved? 13. Is MAST working effectively towards a singular beneficiary registry? If not, what are the constraints that may be impeding progress towards this stated objective? 14. What is the process of translating raw data into HDVI scores into stored data? What data validation processes are utilized by the project and/or MAST? 15. Have cleaned household lists been shared by the project and or MAST with key stakeholders including public providers of social services, CSOs, NGOs? 16. What challenges does/did the project face in transferring the HDVI database to MAST? 17. What specific activities might facilitate the efficient operation of the HDVI database by MAST staff and management? 18. How does the HDVI matches the needs, priorities and contexts of the target communities and beneficiaries (Program Design)? 19. If applicable, how useful have you found your participation in or interaction with the following: Kore Lavi Steering Committee; Program Management Team; Program Forum and; Program Technical Working Groups (implementation/management and collaboration)? 20. What has been the level of ownership of the program (HDVI-specific) (commitment of resources/ adjustment of programming and budget / participation) by MAST, and other Haitian government institutions (in particular, MSPP) (Progress)? 21. What are the major factors which will advance or deter to scale up the methodologies and systems implemented by the program (vulnerability targeting) nationally after the program ends (Effects). 22. What are the major factors that have facilitated or impeded the participation of the private sector (vendors, micro financial institutions) in the social safety net programs (Effects)? 23. What are the key strengths and weaknesses of the program related to the HDVI (Learning)? 24. What are the key/critical emerging learning that the program should deepen to improve quality of interventions, further disseminate to inform/influence the practice of social protection in Haiti and FFP programs (Learning)? Kore Lavi Title II Program Haiti – Midterm Evaluation 122 122 122 Annex 7: SO2 Discussion Guides DISCUSSION GUIDE Mid-Term Evaluation (MTE) “Kore Lavi” (Title II Development Food Assistance Program) CARE USA NOVEMBER 2015 SO2: Access of extremely vulnerable households to local and nutritious foods increased IR 2.1: MAST-led, gender-responsive, food voucher-based safety net model developed and implemented IR2.2: Inclusion of local foods in the voucher-based safety net increased IR2.3: Access to complementary services for safety net households increased Under SO2 component the program: a) developed voucher transfer modalities, prioritizing use of local foods and deliver food vouchers to 18,150 extremely vulnerable households. In particular, the program defined the value of the monthly transfer of food vouchers to the most vulnerable households to be 1,100 HTG (+/- 25 USD) (all beneficiary households get the same amount of assistance). To promote good nutritional practices through a balanced diet strongly supported by fresh food and at the same time to make a direct contribution to local production, Kore Lavi uses two types of vouchers: paper vouchers (value - 400 HTG/9USD), which are allocated for the purchase of local fresh foods (fruits, vegetables, fresh meat, fish, eggs etc.) and electronic vouchers (value 700 HTG/16USD) allocated for the purchase of local staple foods (local rice, maize, sorghum, beans, flour, and oil). The fresh food is purchased in the community markets (which function every week) and the staple food is purchased in small commu￾nity shops, from the vendors selected by the program. Nonetheless, not limiting local merchants’ capacity in supplying fresh foods, the following criteria were applied when selecting the vendors (mostly female merchants): their capacity to deliver a variety of fresh foods and their commitment to proposing stable and fair prices; agreement to the program requirement to sell only local products approved by the pro￾gram; agreement to the program requirement of the merchants to be registered on a regular basis in the market in which they work etc. b) implemented food voucher-based school feeding pilot in six (6) selected schools. As a part of this pilot Kore Lavi serves one snack and one hot meal for school children on a dai￾ly basis with the support of local caterers trained on hygiene and nutrition practices and equipped with improved cook-stoves (using natural gas). This pilot initiative emphasizes local entrepreneurship, locally produced foods, and is managed outside the school system. The pilot intends to provide an alternative to traditional school feeding interventions in Haiti, which are primarily led by the school management and involve long distance trucking of staple foods, mostly imported (rice corn flour, beans, oil, etc.). Therefore, for the implementation of this pilot, the program selected schools that were not included in the national school program. c) enables VSLA participation (a self-reliance mechanism supporting resilience through accumulation of savings to ensure support in the event of various shocks experienced by the households) benefiting over 180,000 VSLA household members. Kore Lavi facilitated the creation of the VSLA groups order to help integrate as many safety net beneficiaries as possible and for as long as possible during the LOA, and to support the gradual graduation (out of the safety net) of the less vulnerable beneficiaries. Kore Lavi Title II Program Haiti – Midterm Evaluation 123 123 123 Interview Data: DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: Fresh Foods (Paper) and Staple Foods (E-Vouchers) 1. What types of information/interviews have you previously participated in (or implemented in the case of program staff)? 2. What do you perceive as the objectives and goal of the voucher activity? 3. How did the program select communities/markets vendors/shops where the vouchers can be redeemed? Where there any challenge in identifying communities/markets vendors/shops? 4. How were the households that receive the vouchers selected? 5. Have the paper vouchers been distributed on time? What is the timeframe for distributions? Have there any problems associated with this timeframe that could be mitigated by changing the timeframe? 6. Has the funds for the e-vouchers been available in a timely manner? What is the timeframe for purchases? Have there any problems associated with this timeframe that could be mitigated by changing the timeframe? 7. Have the market vendors been able to consistently supply local fresh foods (fruits, vegetables, fresh meat, fish, eggs etc.)? Have there been some items that have been unavailable? If so, which and when? 8. Have prices for the local fresh foods been consistent and affordable using the paper vouchers? If not, for which fresh foods have prices varied? Have there been substitutes available? 9. Have the shops been able to consistently supply local staple foods (local rice, maize, sorghum, beans, flour, and oil)? Have there been some items that have been unavailable? If so, which and when? 10. Have prices for the local staple foods been consistent and affordable using the e-vouchers? If not, for which fresh foods have prices varied? Have there been substitutes available? 11. To the best of your knowledge, has the cash sales volume of certain items increased as a result of the paper vouchers or e-vouchers? If there has been a change in cash sales volume, to what extent if this caused by increased purchases by voucher recipients? 12. To the best of your knowledge, has the cash price of certain items changed (increased or decreased) as a result of the vouchers? If so, have producers reacted to this change by increasing or decreasing production of certain products, changing the crops they produce? 13. Have any voucher beneficiaries commented on new foods or changes in the quantities of certain foods that they consume as a result of receiving the vouchers? If so, which foods? Have any voucher beneficiaries commented on the effect of any changes in their diets brought about through the voucher program? Kore Lavi Title II Program Haiti – Midterm Evaluation 124 124 124 Annex 8: SO3 Discussion Guides DISCUSSION GUIDES Maternal and Child Health & Nutrition and Social Safety Net Mid-Term Evaluation (MTE) of “Kore Lavi” (Title II Development Food Assistance Program) - Support to the National Program of Food Security and Nutrition Period of Performance: October 22 - December 15, 2015 The key informant interviews and focus group discussions will inform the mid-term evaluation about how well the Intermediate Results of SO3 were met. • SO3: Maternal and child nutritional status improved • IR3.1: Household practice of appropriate nutrition behaviors to prevent malnutrition increased • IR3.2: Capacity of community-based entities to promote appropriate nutrition practices to prevent malnutrition improved • IR3.3: Capacity of health facilities to deliver appropriate nutritional services strengthened Evaluation Objectives Kore Lavi Mid-Term Evaluation (MTE) has the following objectives: • To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. • To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with Haitian Government (MAST, MSPP). • To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). • To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. • To recommend the future orientation (adjustments to the ToC, project design, resource allocation, or implementation) and emphasis of the program during its remaining time. Theory of Change Kore Lavi’s foundational theory of change is that positive and lasting change must happen in three in￾terrelated domains for the Haitian extremely poor to escape chronic food and nutrition insecurity and recover dignity. The three domains of change are (1) the effectiveness of social safety net mechanisms and Kore Lavi Title II Program Haiti – Midterm Evaluation 125 125 125 Discussion Guide Location: Number of people in group: # males # females Date: Time: 1. Do you know why you were called here today to meet? 2. Do you know if you are part of a program? 3. What is the name of that program? 4. Do you know why you were chosen to participate in that program? 5. Has the program helped you in any way? 6. Have you learned anything new from the program that you did not know before? If yes, what are those things? 7. Do you feed yourself or your under-two-year-old differently than you did before joining the program? If so, what do you do differently? 8. Have you ever been visited by a Lead Mother? If so, what did she do during her visit? What did you learn from her? 9. How many pre-natal visits did you make to the local health clinic with your most recent pregnancy? 10. Did you receive any food rations during those visits? 11. Did you receive any nutritional advice during those visits? 12. Did you take your baby to health clinics to be weighed or get immunized or for sickness? 13. Did you get any advice on nutrition for yourself or your baby during those visits? 14. Have you attended any community activity in the past year that taught something about nutrition for you or your children? 15. What role do the men in your family play in providing and preparing food? 16. How long did you breastfeed your last baby? Or how long do you plan to breastfeed your baby? 17. Are there any obstacles to breastfeeding your very young babies (less than 6 months old) and not feeding them other foods, only breastmilk? 18. If you feed them other foods before they are six months old, what do you feed them? Who feeds them? 19. Are you or were you able to eat four times a day when you were/are pregnant? If not, why did you not eat four times a day? 20. Are you or were you able to eat eggs, meats, milk or cheese during your pregnancy and during breastfeeding? If not, why could you not eat those things? 21. Has being in this program helped you to improve your diet in any way, such as the food supplements or the advice given by Lead Mothers or health clinic workers? 22. If you had any advice for this program, what would it be? Kore Lavi Title II Program Haiti – Midterm Evaluation 126 126 126 complementary services that reach the most vulnerable populations, protect their access to food while building self-reliance, (2) the scale and depth of behavior and social change that are needed to tackle the under-nutrition among vulnerable women and children, and (3) the importance of institutionalization of accountability, quality and mutually reinforcing social protection programs under the leadership of MAST. Persons to be interviewed KIIs • ACF • CARE • WFP • MSPP • CADEPs FGDS • Lead Mothers • Community Health Agents • SO3 beneficiaries, including Preventative Distribution and parents of MAM Children Discussion Guide Organization: Name of person interviewed: Gender: Position of person interviewed: Date: Time: IR3.1: Household practice of appropriate nutrition behaviors to prevent malnutrition in￾creased Program activities: 1. What activities did your organization carry out to increase appropriate nutrition behaviors in households, aimed at pregnant and lactating mothers and children under 2 years? 2. What do you consider to be the more successful activities and why? 3. What do you consider to be the less successful activities and why? 4. Did the mothers you reached take part in ante-natal and post-natal consultations, growth monitoring and immunizations? Do you have records to show this? 5. How many Lead Mothers did you train? Is there a training manual? Are there materials they used with mothers or left with mothers? 6. How many mothers did Lead Mothers visit each month? Do you have records to show this? 7. What else did Lead Mothers do to promote better nutrition behaviors? Did they visit markets where mothers worked? Are there records to show their other activities? 8. How many health workers did your program train? Kore Lavi Title II Program Haiti – Midterm Evaluation 127 127 127 Program messages: 1. What were the key messages that you targeted to mothers, fathers, grandparents or other caregivers? 2. Who was involved in crafting these messages? 3. How were these messages pre-tested? 4. How were these messages delivered? That is, what were the channels of communication for each message so that they reached the target audience? (examples: Lead mothers’ household visits, nutrition demonstrations, radio, print materials, information campaign events, women’s groups, men’s groups, pre-natal clinics, well-baby check-ups) 5. How did you follow up with households to find out whether recommended nutrition behaviors were practiced? (Was there a baseline study of practices?) 6. What were the findings of these studies? 7. As a result of the findings, did you need to adjust any message or channel of delivery? If so, please explain. 8. Would you suggest any changes in how to promote better practices in household nutrition? IR3.2: Capacity of community-based entities to promote appropriate nutrition practices to prevent malnutrition improved 1. Which community-based entities under your supervision promoted appropriate nutrition practices? 2. What exactly did they do? 3. Who were the main outreach people responsible for promotion of nutrition? 4. Who trained these people and with what materials? 5. Did they keep a record of how many people they served? 6. How could community-based entities be strengthened to promote better practices in household nutrition? IR3.3: Capacity of health facilities to deliver appropriate nutritional services strengthened 1. Which health facilities promoted appropriate nutrition practices? 2. What exactly did they do? 3. Who were the main outreach people responsible for promotion of nutrition? 4. Who trained these people and with what materials? 5. Did they keep a record of how many people they served? 6. What is the strength of having health facilities deliver nutritional services? 7. How might nutritional services at health facilities be strengthened? Annex 9: SO4 Discussion Guides Kore Lavi Title II Program Haiti – Midterm Evaluation 128 128 128 Annex 9: SO4 Discussion Guides DISCUSSION GUIDE SO4: Haitian institutions’ capacity to effectively lead and manage safety net programming improved Mid-Term Evaluation (MTE) “Kore Lavi” (Title II Development Food Assistance Program) Support to the National Program of Food Security and Nutrition CARE USA NOVEMBER 2015 1.0 SO4 Background IR4.1: Institutional capacity of various levels of government to lead, coordinate and implement safety net programs reinforced; IR4.2: Capacity of civil society to monitor and support safety net programs reinforced; IR4.3: Government capacity to respond to food emergencies expanded. Kore Lavi Mid-Term Evaluation (MTE) has the following objectives: • To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. • To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with Haitian Government (MAST, MSPP). • To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). • To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. • To recommend the future orientation (adjustments to the ToC, project design, resource allocation, or implementation) and emphasis of the program during its remaining time. 2.0 Key Informant Interview Guide DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: Kore Lavi Title II Program Haiti – Midterm Evaluation 129 129 129 1. How does the program match the needs, priorities and contexts of the target communities and beneficiaries (Program Design)? 2. If applicable, how useful have you found your participation in or interaction with the following: Kore Lavi Steering Committee; Program Management Team; Program Forum and; Program Technical Working Groups (implementation/management and collaboration)? 3. What has been the level of ownership of the program (commitment of resources/adjustment of programming and budget / participation) by MAST, and other Haitian government institutions (in particular, MSPP) (Progress)? 4. What are the major factors which will advance or deter to scale up the methodologies and systems implemented by the program nationally after the program ends (Effects). 5. What are the major factors which have facilitated or impeded the participation of the private sector (vendors, micro financial institutions) in the social safety net programs (Effects)? 6. What are the key strengths and weaknesses of the program related to capacity development of the Haitian government institution (Learning)? 7. What are the key/critical emerging learning that the program should deepen to improve quality of interventions, further disseminate to inform/influence the practice of social protection in Haiti and FFP programs (Learning)? 8. How have the relevant Haitian institutions (MAST, CADEPs, CNSA, DPC etc.) been strengthened in their ability to a) coordinate and manage social assistance programming b) monitor safety net programs c) respond to food emergences (Progress)? 9. How effective has the program gender strategy been in promoting/improving gender equality? 10. Has coordination between national-level and department-level MAST staff improved/worsened as a result of Kore Lavi activities? 11. MAST: What are the strengths and weaknesses of Kore Lavi efforts to facilitate MAST development of social safety net reference documentation (policies, governance, coordination, implementation)? 12. CADEP/CSO: What worked well in Kore Lavi’s efforts to help your organization develop and implement capacity development plans and to carry out social audits? What activities or interventions in this regard didn’t work as well? How can this activity be improved? 13. What are the strengths and weaknesses of the implementation of the Community Score Card? What elements of social audit training from Kore Lavi was particularly useful? What elements have been/ should be improved? 14. Were the Kore Lavi Participatory Forums useful? Why or why not? To what degree did women participate in Public Forums? What activities or topics have been discussed most at Public Forums? 15. CNSA/IHSI 16. Describe your interaction with Kore Lavi staff and activities. 17. What Kore Lavi activities, advice, or interventions have been particularly useful? Kore Lavi Title II Program Haiti – Midterm Evaluation 130 130 130 18. What Kore Lavi activities, advice, or interventions have not been particularly useful? 19. What influence has the program had on the frequency and quality of CNSA assessments and early warning reports? Collaboration with DPC (to enable safety net expansion during emergencies)? 20. What has been the strengths and weaknesses of CNSA’s involvement with the implementation of vulnerability targeting? Kore Lavi Title II Program Haiti – Midterm Evaluation 131 131 131 Annex 10: Gender Discussion Guides DISCUSSION GUIDES GENDER – Cutting Across the Four “Kore Lavi” Program Components Mid-Term Evaluation (MTE) “Kore Lavi” (Title II Development Food Assistance Program) Support to the National Program of Food Security and Nutrition CARE Haiti NOVEMBER 2015 These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information Kore Lavi Mid-Term Evaluation (MTE) has the following objectives: • To assess and highlight factors affecting effective, quality and efficient implementation of the program activities/interventions. • To assess the efficiency and effectiveness of program organization, management and coordination mechanisms, including quality and usefulness of partnership among consortium members and with Haitian Government (MAST, MSPP). • To assess the progress (strengths and weaknesses) of the program (per each component) against stated outputs and outcomes to date (including an assessment of the relevance of the outputs and outcomes though a gender lens). • To assess early evidence of changes in behavior and practices (both intended and unintended) in the target communes, and compare these with the changes that were expected to be promoted by the program activities. Identify factors in the implementation or context that hold back or promote the observed and intended changes. • To recommend the future orientation (adjustments to the ToC, project design, resource allocation, or implementation) and emphasis of the program during its remaining time. Background: Gender – Cross Cutting Activities for promoting gender equality are integrated into each component of the program to enhance the quality, and not just the quantity, of female participation at different levels, from participation in strate￾gic planning at the national level to more effective participation in decision-making at the household level. In line with the recommendation of the Ministry of Women’s Affairs and Women’s Rights (MCFDF), a key focus of the program has been the training of implementation staff and community structures in gender sensitivity programing. The program has also been providing support to MCFDF to enhance their ability to promote and support gender integration in food security and social assistance programming in Haiti. Kore Lavi Title II Program Haiti – Midterm Evaluation 132 132 132 While the gender aspects of “Kore Lavi” cut across all the program components, the technical and imple￾mentation oversight are assured by the Ministry of Women’s Status and Women’s Rights (MCFDF)1 , ACF and CARE. These implementing and Coordination partners would be an entry point for the KIIs; and so would also be the Bureau of Nutrition (MSPP) given their important role in the nutrition component of the program. This KII and FGD guide attempts to bring out all the gender integration and other relevant aspects of this MTE that cut across all the four components represented by Strategic Objectives (SO) 1-4. The document structured along the different SOs captures the evaluations questions relevant to gender issues for each of the SOs. The KIIs and FGDs could be administered as standalone or integrated as gender related questions in other KIIs and FGDs. The main concern with the former option would be that: i) the crosscutting and integrated nature of gender issues might get lost, and ii) the exchange of ideas and lessons learnt from an integrated approach for both the MTE team and interviewees may also be lost; making gender issues a true standalone as against the integration that we strive to promote. This said, from the structure of the program, particularly the consolidation of the technical and implemen￾tation oversight of the gender aspects under a three partner team, it is likely that even if an integrated approach is adopted, it may be necessary to have some standalone gender-focused interviews with this team (ACF, CARE, MCFDF). 1.0 SO1 – National Systems for Vulnerability Targeting Strengthened IR1.1: MAST-led equitable vulnerability targeting methodology developed, tested and implemented. IR1.1 has three main outputs: 1. Develop, test and adjust a national Household Deprivation and Vulnerability Index (HDVI); 2. Define a specific data collection methodology; 3. Develop and implement a database application host by MAST SO1 - Key Informant Interview Guide Likely persons to be interviewed • Gender Lead Team and IPs: CARE, WFP • Other stakeholders: MCFDF, MAST, • Local experts, Opinion Leaders with local knowledge of gender issues DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: 23 Ministere de la Condition Feminine et Droit de la Femme? Kore Lavi Title II Program Haiti – Midterm Evaluation 133 133 133 These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning the interview, describe relevant the Kore Lavi SO1 intervention and ensure that the re￾spondent is familiar with it. 1. Was the development of the vulnerability targeting methodology (Household Deprivation and Vulnerability Index, or HDVI) informed by input from a gender-focused institution such as MCFDF? Would it have made a difference if such an expert were part of the team? 2. What are the specific gender equality promoting activities/strategies integrated into the development, design and implementation of the HDVI? 3. Did the staff involved in developing and implementing the HDVI receive training on gender sensitivity? If ‘Yes,’ please describe. If ‘No,’ please explain why not. 4. How would you assess the gender sensitivity training (e.g., useful or needs improvements)? 5. What are ways to improve this training? 6. What elements of the HDVI methodology and the HDVI information management system do you find most useful from a gender perspective? 7. What are the gender implications of the key strengths and weaknesses of the HDVI intervention? 8. From a gender perspective, what are the key/critical emerging learnings from the HDVI intervention related to improving the quality and effectiveness of program interventions and informing the practice of social protection in Haiti? Within Food for Peace programs more generally? 2.0.SO2 – Access of Extremely Vulnerable Households to Local and Nutritious Foods In￾creased IR 2.1: MAST-led, gender-responsive, food voucher-based safety net model developed and implemented IR2.2: Inclusion of local foods in the voucher-based safety net increased IR2.3: Access to complementary services for safety net households increased Under this component the program: 1. Developed voucher transfer modalities, prioritizing use of local foods and deliver food vouchers to 18,150 extremely vulnerable households. The voucher program is made up of two types of vouchers – i) paper vouchers (value 400 HTG/9USD), allocated for the purchase of local fresh foods (fruits, vegetables, fresh meat, fish, eggs etc.), and i i ) electronic vouchers (value 700 HTG/16USD) allocated for the purchase of local staple foods. 2. Implemented a food voucher-based school feeding pilot in six (6) selected schools. 3. Enables VSLA participation (a self-reliance mechanism supporting resilience through accumulation of savings to ensure support in the event of various shocks experienced by the households) benefiting over 180,000 VSLA household members .Kore Lavi facilitated the creation of the VSLA groups order Kore Lavi Title II Program Haiti – Midterm Evaluation 134 134 134 to help integrate as many safety net beneficiaries as possible and for as long as possible during the LOA, and to support the gradual graduation (out of the safety net) of the less vulnerable beneficiaries 4. Works in collaboration with private sector partners: i) project selected vendors of the staple foods (mostly female merchants); and ii) private local caterers trained on hygiene and nutrition practices and equipped with improved cook-stoves for the school feeding pilot. Kore Lavi Title II Program Haiti – Midterm Evaluation 135 135 135 SO2– Key Informant Interview Guide Likely persons to be interviewed • IPs and their staff: ACF, CARE, WV • GoH: MAST, MCFDF • Leaders of beneficiaries organizations (VSLA, …) • Local experts, Opinion Leaders with local knowledge of gender issues DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning the interview, describe relevant the Kore Lavi SO2 intervention and ensure that the re￾spondent is familiar with it. 1. What are the key gender-related strengths and weaknesses of this intervention? 2. Was the design and implementation of this intervention informed by identified gender/social needs and priorities in Haiti, and more specifically those of the targeted beneficiaries and their communities? If ‘Yes,’ please explain why. If ‘No,’ please explain why not. 3. Did the staff involved in the design and/or implementation of the activities under this component receive training on gender sensitivity programing? If ‘Yes,’ please describe. If ‘No,’ please explain why not. a. How would you assess the gender sensitivity training (e.g., useful or needs improvements)? Explain. b. Any suggestions on ways to improve this training? 4. What are the specific gender equality promoting activities/strategies integrated into this intervention? Are these activities effectively addressing identified gender and social concerns? If ‘Yes,’ please describe how. If ‘No,’ please explain why not. 5. Has the selection of beneficiaries for these activities been gender equitable? If ‘Yes,’ please describe Kore Lavi Title II Program Haiti – Midterm Evaluation 136 136 136 how. If ‘No,’ please explain why not. a What ongoing barriers exist for achieving greater gender equity in this intervention? b. What are possible ways to overcome these barriers? 6. Have gender related factors played a role in facilitating or impeding the extent and levels of participation of private sector actors, such as vendors and MFIs, in this intervention? If ‘Yes,’ what are these factors and what roles have they played? If ‘No,’ why not? 7. To what extent have the pre-existing gender and other social factors affected the intervention’s effectiveness? 8. How have the intervention’s activities themselves affected the pre-existing the same gender and other social factors? 9. What are the intervention’s achievements to date in terms of its impacts on it women beneficiaries specifically and on other gender outcomes more generally? 10.Do these achievements demonstrate that the program is on track, out-performing or under￾performing with regards to its gender outcomes and impacts? 11.What are the major gender or other social factors that have facilitated or impeded the process of the intervention’s implementation and its gender outcomes and impacts? 12.From a gender perspective, what are the key/critical emerging learnings from this intervention related to improving the quality and effectiveness of program interventions and informing the practice of social protection in Haiti? Within Food for Peace programs more generally? Kore Lavi Title II Program Haiti – Midterm Evaluation 137 137 137 SO2. Focus Group Discussion Guide Likely persons to be interviewed (mainly in gender specific groups of 6-10 members): • Selected food vendors • Local caterers • VSLA beneficiaries • SO2 beneficiaries DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF GROUP OR ORGANIZATION IF APPLICABLE: These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning any discussion, ask the participants to identify and describe the Kore Lavi SO2 activities in which they participated (voucher, school feeding and VSLA). Voucher Beneficiaries - men and women: 1. Did management of voucher program go as planned? – Were vouchers delivered in a timely manner, to appropriate beneficiaries, and through identified mechanisms? a. Was the management and delivery of the vouchers equal with regards to men and women? b. If ‘No,’ what difference existed and how did this affect the outcomes? c. How can the process be made more equitable? 2. Which of the voucher delivery mechanisms were better suited to your needs (paper vouchers, electronic vouchers tied to selected vendors, voucher-based school feeding)? Explain why. a. Do the needs of women differ than the needs of men where it comes to the voucher delivery mechanism? b. If ‘Yes,’ how? c. How can the delivery mechanism be changed so as to meet the needs of both women and men better? Kore Lavi Title II Program Haiti – Midterm Evaluation 138 138 138 3. What aspects of the voucher system worked best for you and others in your community; what aspects did not work so well? Explain. a. Are there aspects of the voucher system that affected women different than men? b. If ‘Yes,’, what were they and how did they affect the outcomes? c. How can aspects of the voucher system be changed to improve the outcomes for both women and men? 4. What are your suggestions on how the not so useful/effective aspects could be improved to meet your expectations and promote a scale-up of the voucher program nationwide, particularly with regards to female beneficiaries? Food Vendors: 5. How did you get in the food voucher program? a. Was the selection process equal with regards to female and male food vendors? b. If ‘No,’ what difference existed and how did this affect the outcomes? c. How can the process be made more equitable? 6. What expectations did you have from it and were these expectations met? a. Do the expectations of female vendors differ than the needs of male vendors? b. If ‘Yes,’ how? c. How can the delivery mechanism be changed so as to meet the needs of both women and men better? 7. What aspect of the program did you find to be most useful? Why? a. Are there aspects that were more useful to female clients? b. If ‘Yes,’ how? 8. What aspect was not so useful to you or your clients? Why? a. Are there aspects that were less useful to female clients? b. If ‘Yes,’ how? 9. Can the food voucher program be changed so as to improve the process and outcomes for both female vendors and female clients? If ‘Yes,’ how? 10. What are your suggestions on how the not so useful/effective aspects could be improved to meet your expectations and promote a scale-up of the voucher program nationwide, particularly with regards to female vendors and female clients? Kore Lavi Title II Program Haiti – Midterm Evaluation 139 139 139 VSLA Beneficiaries 11. How/why did you get into the VSLA program? a. Did the selection process show equal consideration for both men and women, young and old? If ‘Yes,’ please explain. If ‘No,’ why not? b. How can the selection process be changed to make it more equitable? 12. Are you a member of a single sex association or mixed one? a. Why did you choose to be in this type (mixed or single sex) of group? A b. re there differences between a single sex association and a mixed one in terms of, for example, women’s participation and opportunities? c. If ‘Yes,’ what are they? 13. What expectations did/do you have from the program? How have these been met? a. Do expectations differ between men and women members? b. If ‘Yes,’ how? c. What are the implications of this for women’s participation and the benefits they receive from membership? 14. Has involvement in, and benefits from, the program enhanced your self-sufficiency and status within the household and community? a. Do you receive more respect and consultation in decision-making matters from your spouse/ partner as a result of the benefits derived through the VSLA? b. Are VSLA benefits a useful strategy towards self-sufficiency and graduation from this program? c. Are there graduates that you know of this far, both women and men? How have they faired since graduation? Why do you think that is? 15. Would you say that this program has help promote/improve equality between men and women, young and old as compared to the past? Please explain. 16. Overall, what would you say are the key strengths and weakness of the program with regards to its impact on women both overall and in relation to men? How can the weaknesses be improved? Kore Lavi Title II Program Haiti – Midterm Evaluation 140 140 140 3. SO3: Maternal and child nutritional status improved IR3.1: Household practice of appropriate nutrition behaviors to prevent malnutrition increased IR3.2: Capacity of community-based entities to promote appropriate nutrition practices to prevent mal￾nutrition improved IR3.3: Capacity of health facilities to deliver appropriate nutritional services strengthened Under this component the program: 1. Implements an SBCC strategy in line with the preventive approach focusing on the window of opportunity in the first 1,000 days of life (PM2A). The program provides supplementary conditional rations to targeted individuals - Pregnant and Lactating Women and Children under 2 (a total of 76,567 pregnant and lactating women and 96,597 children aged 6-23 months during the life of the program). 2. Implemented through a Care Group approach. A Care Group is a group of Lead Mothers who are identified by the community and regularly trained by the local community health agents (ASCP1). 3. Collaborates with the existing network of the community health agents of MoHto reinforce the knowledge of the community members regarding proper health and nutrition behaviour and practices. 4. Carries out public sensitization (through various community public awareness raising campaigns such as media campaigns, nutrition fairs, special days and events etc.) key community structures, civil society organizations and traditional health workers. 5. Conducts trainings for health professionals and primary health care workers. One of the main activities currently supported at the health facility level is the management of MAM. Namely, through the network of community health agents, the program supports and facilitates active and passive screening of children to identify new cases of MAM and refer them to healthcare facilities to receive nutritional treatment. Kore Lavi Title II Program Haiti – Midterm Evaluation 141 141 141 SO3. Key Informant Interview Guides Likely persons to be interviewed • IPs and their staff: ACF, CARE, WFP • GoH: MoH, MSPP, MCFDF • CADEPs & other CSOs • Local experts, Opinion Leaders with knowledge on local knowledge of gender issues DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning the interview, describe relevant the Kore Lavi SO3 intervention and ensure that the re￾spondent is familiar with it. 1. What are the key gender-related strengths and weaknesses of this intervention? 2. Was the design and implementation of this intervention informed by identified gender/social needs and priorities in Haiti, and more specifically those of the targeted beneficiaries and their communities? If ‘Yes,’ please explain why. If ‘No,’ please explain why not. 3. Did the staff involved in the design and/or implementation of the activities under this component receive training on gender sensitivity programing? If ‘Yes,’ please describe. If ‘No,’ please explain why not. a. How would you assess the gender sensitivity training (e.g., useful or needs improvements)? Explain. b. Any suggestions on ways to improve this training? 4. What are the specific gender equality promoting activities/strategies integrated into this intervention? Are these activities effectively addressing identified gender and social concerns? If ‘Yes,’ please describe how. If ‘No,’ please explain why not. 5. Has the selection of beneficiaries for these activities been gender equitable? If ‘Yes,’ please describe Kore Lavi Title II Program Haiti – Midterm Evaluation 142 142 142 how. If ‘No,’ please explain why not. a. What ongoing barriers exist for achieving greater gender equity in this intervention? b. What are possible ways to overcome these barriers? 6. Have gender related factors played a role in facilitating or impeding the extent and levels of participation of private sector actors, such as vendors and MFIs, in this intervention? If ‘Yes,’ what are these factors and what roles have they played? If ‘No,’ why not? 7. To what extent have the pre-existing gender and other social factors affected the intervention’s effectiveness? 8. How have the intervention’s activities themselves affected the pre-existing the same gender and other social factors? 9. What are the intervention’s achievements to date in terms of its impacts on it women beneficiaries specifically and on other gender outcomes more generally? 10. Do these achievements demonstrate that the program is on track, out-performing or under￾performing with regards to its gender outcomes and impacts? 11. What are the major gender or other social factors that have facilitated or impeded the process of the intervention’s implementation and its gender outcomes and impacts? 12. From a gender perspective, what are the key/critical emerging learnings from this intervention related to improving the quality and effectiveness of program interventions and informing the practice of social protection in Haiti? Within Food for Peace programs more generally? Kore Lavi Title II Program Haiti – Midterm Evaluation 143 143 143 SO3. Focus Group Discussion Guide Likely persons to be interviewed • Lead mothers • SO3 beneficiaries • Community health agents (including where possible, a healthcare professional) DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF GROUP OR ORGANIZATION IF APPLICABLE: These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning any discussion, ask the participants to identify and describe the Kore Lavi SO3 activities in which they participated. Community health agents: 1. How would you rate your collaboration with Kore Lavi staff relative to the health and nutrition activities? a. Have any issues arose in this collaboration with respect to how the program addresses gender, such as how it interacts with different genders or how its policies affect different genders? What were these issues? b. How did these issues affect your collaboration with Kore Lavi and the results of that collaboration? 2. You have been involved in training Lead Mothers, contributing to Kore Lavi’s community health and nutrition sensitization campaigns, and to the screening of children for MAM. Which of these activities in your view are more and less effective in promoting the achievement of the maternal health and nutrition activities? Explain. a. In each of these activities, what important issues exist or arose that have important implications for how women participate in and benefit from the activities? b. To what extent do these activities take into account, or fail to take into account, these issues? Kore Lavi Title II Program Haiti – Midterm Evaluation 144 144 144 c. What have these gender issues affected the outcomes of these activities? 3. In your opinion, what are the key strengths and weakness of this program, particularly in relation to working with women? Do you have any suggestions on how the weaknesses can be turned around to enhance the achievement of the goals of the program, again particularly with regards to women? Lead Mothers: 1. How/why did you get selected as a Lead Mother? What do you think about the selection process in terms of how it affected you and others? 2. Who were your trainers and to what extent did the training the training you received match your situation in terms of your availability and mobility constraints (for example, days of training, training venue, distance and timing)? 3. To what extent did you participate in the design and planning of the training schedule? 4. Did the training meet both your needs and priorities and the needs and priorities of your fellow trainees and the communities served? 5. How many target houses do you have on your list? How often do you visit and advise them? To what extent are your visits tied to (1) the needs, priorities and availability of your target households and (2) your needs, priorities and availability? 6. What factors influence the participation or non-participation of target households? Examples might include the size of family, marital status, women’s household and work responsibilities, status of women inside the household, or the women’s income earning situation? Please explain how. 7. What changes within the household, particularly, in terms attitudes and behaviors, are you observing within target households as a result of your work? 8. What factors are influencing the observed changes within target households? Examples might include the size of family, marital status, women’s household and work responsibilities, status of women inside the household, or the women’s income earning situation? Please explain how. 9. In your opinion, what are the key strengths and weakness of this program? Do you have any suggestions on how the weaknesses can be turned around to enhance the achievement of the goals of the program? In answering this question, please consider your earlier answers with regards to the factors influencing participation and observed changes. SO3 Beneficiaries: 1. What aspect(s) of this program did you participate in (nutritional supplements, food rations, Lead Mother training, health trainings, public awareness raising campaigns, all of them)? 2. Which of the activities are more beneficial to your needs and priorities and which ones are not so useful/practical? Why? 3. More generally, to what extent did the activities take into account, or fail to take into account, the needs and priorities of women in your situation? a. What are these needs and priorities? Kore Lavi Title II Program Haiti – Midterm Evaluation 145 145 145 b. How specifically did the activities either take into account, or fail to take into account, these needs and priorities? 4. What benefits have you derived from your participation in these activities? a. How were these benefits affected by how the activities either did or did not take into account the needs of priorities of women in your situation? 5. Do have any suggestions on how the program can be improved to better serve you and other beneficiaries? In answering this question, please think about your answers to the above questions about how the activities either did or did not take into account the needs and priorities of women in your situation. Kore Lavi Title II Program Haiti – Midterm Evaluation 146 146 146 -SO4- Institutional capacity of various levels of government to lead, coordinate and im￾plement safety net programs reinforced IR4.1: Institutional capacity of various levels of government to lead, coordinate and implement safety net programs reinforced; IR4.2: Capacity of civil society to monitor and support safety net programs reinforced; IR4.3: Government capacity to respond to food emergencies expanded. Under this component the program intends to: 1. Complement joint USAID/MAST capacity building and resource mobilization plan; 2. Work with MAST national and department-level staff on coordination of safety net program activities; 3. Develop MAST safety net reference documentation (policies, governance, coordination, implementation); 4. Support CADEPs and/or other CSOs to develop capacity building plans and carry out social audit processes; 5. Implement gender-sensitive annual social audits with local stakeholders; 6. Train community structures (local authorities, CSOs) on social accountability focusing on the application of the Community Score Card (CSC), a tool to enhance community influence on the quality, effectiveness and accountability of the services offered by Kore Lavi at community level. 7. Provide support to CNSA and IHSI to deliver timely and standardized survey assessments/early warning reports, and supports DPC to enable safety net expansion during emergencies. 8. Kore Lavi also used interactive and participatory forums to highlight specific activities with service users, approximately 30 % of participants in these forums were women. SO4. Key Informant Interview Guide Likely persons to Interview • IPs and their staff: ACF, CARE, WFP • GoH: MAST, MCFDF • CADEPs and other CSOs and local authorities • Other local collaborators: CNSA, • Public forum facilitators DATE: INTERVIEWER: COMMUNE: DEPARTMENT: Kore Lavi Title II Program Haiti – Midterm Evaluation 147 147 147 Respondent Information: NAME: MALE/FEMALE? TITLE: NAME OF ORGANIZATION: These discussions will include an introductory discussion of the purpose of the study and the intended use of information. The MTE team will obtain the consent of participants for the use of the information. Before beginning the interview, describe relevant the Kore Lavi SO4 intervention and ensure that the re￾spondent is familiar with it. 1. What are the key gender-related aspects, strengths and weaknesses of the intervention related to capacity development of Haitian government institutions? 2. To what extent do the Kore Lavi program activities in which your organization is involved match the needs, priorities and contexts of the target communities and beneficiaries, particularly with regards to women and youths? 3. MCFDF on Gender Sensitivity Training: Can you tell us about your experiences with the development and implementation of the gender integration monitoring tools for government institutions working on Safety Net programs? a. What were the tools developed? b. What has been the outcome of using these tools, particularly with regards to how the government institutions are changing how they do things with regards to gender integration? c. What worked best and what did not work so well? What are your suggestions for improving the process and outcomes with regards to gender integration in Safety Net program? d. Overall, how would you describe the relevance, usefulness and impact of your collaboration with “Kore Lavi” on this gender sensitivity training process? 4. How would you describe the institutional setup of Kore Lavi as far as gender sensitivity and inclusiveness are concerned? 5. What are your suggestions for improving the gender aspects of this program? For instance, what difference might the presence of MCFDF at the oversight and management level had? 6. To what extent has the collaboration between the different stakeholders involved in this and other project components contributed to improving the gender outcomes of this programme? How has it done so? 7. From a gender perspective, what are the key/critical emerging learnings from this intervention related to improving the quality and effectiveness of program interventions and informing the practice of Kore Lavi Title II Program Haiti – Midterm Evaluation 148 148 148 social protection in Haiti? Within Food for Peace programs more generally? 8. What role have cross cutting issues—such as gender and social accountability—played in shaping this and other Kore Lavi interventions and influencing progress towards achieving planned results? 9. What worked well in Kore Lavi’s efforts to help your organization develop and implement capacity development plans and to carry out social audits? a. To what extent were these audits gender sensitive? b. What were the strategies used to promote gender sensitivity? c. Which strategies in this regard worked well and which ones didn’t work as well? Why or why not? 10. Overall, how effective has the program gender focus been in promoting/improving gender inclusiveness within key governmental and non-governmental institutions within Haiti? 11. To what extent has the CNSA assessments and disaster early warning reports to enable safety net expansion during emergencies exhibited gender sensitivity? Please explain. Kore Lavi Title II Program Haiti – Midterm Evaluation 149 149 149 Annex 11: MEAL Discussion Guides CARE HAITI 2015 KII Interview and Small Focus Group Guidelines Introductions and Sequence Ask for the interviewee’s personal experience and qualifications in the field, and then ask him or her to explain the overall approach to Data / IS or M&E as appropriate. Explain that there will be an opportunity to review the sequence of activities shortly, and that the interview will then focus on the sequence of activities. Give the interviewee good time to establish / identify their own competencies before proceeding. Use open (not leading) questions to set a conversational tone. Our experience is that a less formal introduction makes it easier to focus on sequence and detail later in the discussion. Generally, the interview should be informational: avoid responses which suggest that you’ve come to a conclusion or finding about the program. Answer questions about the process openly, but, when appropriate, refer the interviewee to his or her organization project contact for further information. Personal Experience and Qualifications The purpose of this section is to allow the interviewee to establish the relevance of their own presence to the interview and assessment process. Emphasize that there is no minimum qualification: this is an opportunity to present their professional experiences in the context of the field. Field workers, including managers, often have personal insight into operations and indicator development that can inform a deeper assessment of M&E and Data / IS fields. If specific areas are introduced, return to that section in detail later in the interview. Overview & Strategy This section extends the personal introduction to the more general management of the work in the target organization. Most interviewees will be able to give a short narrative of recent changes or challenges, including areas which can be developed under ‘sequence’ below. This should prepare the interviewee to discuss strategy comfortably. If appropriate, ask for details of management contacts who can describe strategy at a general level. Processes This stage needs more careful management; if necessary, mention that it’s important to move on as if necessary you can follow up in more detail after the interview. When making notes and directing conversation, focus on organizing the information into the sequences set out in the assessment forms, and make particular note of successes, responses to challenges, and the Kore Lavi Title II Program Haiti – Midterm Evaluation 150 150 150 importance of local field conditions (including labor and security) that require adjustment in approach. Where processes are combined, follow the interviewee’s description and flow as appropriate. If the interviewee is unable to comment on a particular area because of lack of exposure to the work or any other reason, note the fact and explanation, and move on. If he or she is unable to give detail, ask for an example of the work being discussed. Discussing methodology (both at M&E level and as part of Data / IS) can be time-consuming. Avoid developed conversations about specific technical detail, and use the general question sets below to redirect or draw out summaries as appropriate. Data & IS Prompts Identification How do you identify relevant data? Who decides what is relevant? Capture How is data recorded? Do you use pen and paper or electronic collection? If electronic, what tool(s) and platform(s) do you use to record the data? Describe your training process for data collectors. Transport How is data moved from the recording tool to safe storage? How do you verify that the data arrives at the storage in an identical state? How do you address errors and missing data? Storage How are data records kept and maintained? How are they checked, and how often? How do you address errors and missing data? Access Who access data, and how do they access it? How is it shared? How is it used by those who share it? Analysis How is data processed when analyzed? Is it returned to the data storage system afterwards? How do you store cleaned and uncleaned data sets? M&E Prompts Measure & indicator use Who identifies and develops indicators? How are they decided? Metric development and use How are metrics developed? Who develops them? Who uses metrics and how do they use them? For example, are external reports used for internal decision-making? Field contingencies (see learning) Field data collection and operation can present unexpected challenges: how do you approach those challenges? If an assessment plan is impossible to implement as designed, how do you approach that problem? Who decides on the best approach? Methodology Describe your methodology in this area. How is methodology developed and reviewed? How does it account for field contingencies? Kore Lavi Title II Program Haiti – Midterm Evaluation 151 151 151 Analysis Describe the analysis process, particularly the way that the impact of the project is measured. How do you identify errors and outliers, and what do you do with them? Communication How are results and reports communicated? How are challenges to effective evaluation communicated? Are reports regularly discussed within the project? Can you give a recent example of results that influenced a decision? Learning (all processes) Describe your training processes in this field. How do you ensure that error levels are reduced? Describe how results are shared internally and externally. How does the project share new learnings about results and internal findings? Can you give a recent example of results that influenced a practice or process? Kore Lavi Title II Program Haiti – Midterm Evaluation 152 152 152 CARE HAITI 2015 Monitoring and Evaluation Systems Assessment INTERVIEWEE Name Location Organization Project reference: OVERVIEW Internal system assessment Constraints and responses Comments and needs NOTES/ACTIONS ON-SITE COMPTENCIES Areas and level of proficiency External and other support and overview Comments and needs NOTES/ACTIONS PROCESSES Component Description Comments and needs Strategy Data / IS assessment? Yes/no? Kore Lavi Title II Program Haiti – Midterm Evaluation 153 153 153 1. Measure & indicator use 2. Metric development and use 3. Field contingencies (see learning) 4. Methodology 5. Analysis 6. Communication 7. Learning (all processes) NOTES/ACTIONS Kore Lavi Title II Program Haiti – Midterm Evaluation 154 154 154 CARE HAITI 2015 Data and Information Systems Assessment INTERVIEWEE Name Location Organization Project reference: OVERVIEW Internal system assessment Constraints and responses Comments and needs NOTES/ACTIONS ON-SITE COMPETENCIES Areas and level of proficiency External and other support and overview Comments and needs NOTES/ACTIONS PROCESSES Component Description Comments and needs Strategy Security & integrity Kore Lavi Title II Program Haiti – Midterm Evaluation 155 155 155 Name Location Organization Project reference: OVERVIEW 8. Identification (see M&E) 9. Capture 10. Transport 11. Storage 12. Access 13. Analysis (see M&E) NOTES/ACTIONS